We investigated the ability of 40 left-hemisphere brain-lesioned individuals with various diagnoses of aphasia to temporally synchronize the audio of a spoken word to its congruent video using a maximum-likelihood adaptive psychophysical procedure. We found a statistically significant effect of aphasia type, not explained by lesion volume, on measures of audiovisual (AV) synchrony. Brain-lesioned individuals with no symptoms of aphasia, and those with conduction aphasia performed on the synchrony task more similarly to age-matched neurotypical controls, whereas those with anomic aphasia performed the poorest. In addition, we examined the correlation between this ability and AV integration (fusion) and observed a significant correlation between measures of AV synchrony and fusion. An ROI analysis of stroke lesion maps showed that damage to the left posterior temporal regions adversely affected AV processing, although whole-brain univariate lesion-symptom mapping analyses did not yield any significant results. These findings contribute to a better understanding of the functional relationship between different AV processes in multimodal integration and their underlying cortical networks in the human brain.

Understanding the neural basis of recovery from stroke is a major research goal. Many functional neuroimaging studies have identified changes in brain activity in people with aphasia, but it is unclear whether these changes truly support successful performance or merely reflect increased task difficulty. We addressed this problem by examining differences in brain activity associated with correct and incorrect responses on an overt reading task. On the basis of previous proposals that semantic retrieval can assist pronunciation of written words, we hypothesized that recruitment of semantic areas would be greater on successful trials. Participants were 21 patients with left-hemisphere stroke with phonologic retrieval deficits. They read words aloud during an event-related fMRI paradigm. BOLD signals obtained during correct and incorrect trials were contrasted to highlight brain activity specific to successful trials. Successful word reading was associated with higher BOLD signal in the left angular gyrus. In contrast, BOLD signal in bilateral posterior inferior frontal cortex, SMA, and anterior cingulate cortex was greater on incorrect trials. These data show for the first time the brain regions where neural activity is correlated specifically with successful performance in people with aphasia. The angular gyrus is a key node in the semantic network, consistent with the hypothesis that additional recruitment of the semantic system contributes to successful word production when phonologic retrieval is impaired. Higher activity in other brain regions during incorrect trials likely reflects secondary engagement of attention, working memory, and error monitoring processes when phonologic retrieval is unsuccessful.

In the visual modality, perceptual demand on a goal-directed task has been shown to modulate the extent to which irrelevant information can be disregarded at a sensory-perceptual stage of processing. In the auditory modality, the effect of perceptual demand on neural representations of task-irrelevant sounds is unclear. We compared simultaneous ERPs and fMRI responses associated with task-irrelevant sounds across parametrically modulated perceptual task demands in a dichotic-listening paradigm. Participants performed a signal detection task in one ear (Attend ear) while ignoring task-irrelevant syllable sounds in the other ear (Ignore ear). Results revealed modulation of syllable processing by auditory perceptual demand in an ROI in middle left superior temporal gyrus and in negative ERP activity 130–230 msec post stimulus onset. Increasing the perceptual demand in the Attend ear was associated with a reduced neural response in both fMRI and ERP to task-irrelevant sounds. These findings are in support of a selection model whereby ongoing perceptual demands modulate task-irrelevant sound processing in auditory cortex.

The role of sensory-motor systems in conceptual understanding has been controversial. It has been proposed that many abstract concepts are understood metaphorically through concrete sensory-motor domains such as actions. Using fMRI, we compared neural responses with literal action (Lit; The daughter grasped the flowers ), metaphoric action (Met; The public grasped the idea ), and abstract (Abs; The public understood the idea ) sentences of varying familiarity. Both Lit and Met sentences activated the left anterior inferior parietal lobule, an area involved in action planning, with Met sentences also activating a homologous area in the right hemisphere, relative to Abs sentences. Both Met and Abs sentences activated the left superior temporal regions associated with abstract language. Importantly, activation in primary motor and biological motion perception regions was inversely correlated with Lit and Met familiarity. These results support the view that the understanding of metaphoric action retains a link to sensory-motor systems involved in action performance. However, the involvement of sensory-motor systems in metaphor understanding changes through a gradual abstraction process whereby relatively detailed simulations are used for understanding unfamiliar metaphors, and these simulations become less detailed and involve only secondary motor regions as familiarity increases. Consistent with these data, we propose that anterior inferior parietal lobule serves as an interface between sensory-motor and conceptual systems and plays an important role in both domains. The similarity of abstract and metaphoric sentences in the activation of left superior temporal regions suggests that action metaphor understanding is not completely based on sensory-motor simulations but relies also on abstract lexical-semantic codes.

Verbal working memory (VWM), the ability to maintain and manipulate representations of speech sounds over short periods, is held by some influential models to be independent from the systems responsible for language production and comprehension [e.g., Baddeley, A. D. Working memory, thought, and action . New York, NY: Oxford University Press, 2007]. We explore the alternative hypothesis that maintenance in VWM is subserved by temporary activation of the language production system [Acheson, D. J., & MacDonald, M. C. Verbal working memory and language production: Common approaches to the serial ordering of verbal information. Psychological Bulletin, 135, 50–68, 2009b]. Specifically, we hypothesized that for stimuli lacking a semantic representation (e.g., nonwords such as mun ), maintenance in VWM can be achieved by cycling information back and forth between the stages of phonological encoding and articulatory planning. First, fMRI was used to identify regions associated with two different stages of language production planning: the posterior superior temporal gyrus (pSTG) for phonological encoding (critical for VWM of nonwords) and the middle temporal gyrus (MTG) for lexical–semantic retrieval (not critical for VWM of nonwords). Next, in the same subjects, these regions were targeted with repetitive transcranial magnetic stimulation (rTMS) during language production and VWM task performance. Results showed that rTMS to the pSTG, but not the MTG, increased error rates on paced reading (a language production task) and on delayed serial recall of nonwords (a test of VWM). Performance on a lexical–semantic retrieval task (picture naming), in contrast, was significantly sensitive to rTMS of the MTG. Because rTMS was guided by language production-related activity, these results provide the first causal evidence that maintenance in VWM directly depends on the long-term representations and processes used in speech production.

Recent studies suggest that the left superior temporal gyrus and sulcus (LSTG/S) play a role in speech perception, although the precise function of these areas remains unclear. Here, we test the hypothesis that regions in the LSTG/S play a role in the categorization of speech phonemes, irrespective of the acoustic properties of the sounds and prior experience of the listener with them. We examined changes in functional magnetic resonance imaging brain activation related to a perceptual shift from nonphonetic to phonetic analysis of sine-wave speech analogs. Subjects performed an identification task before scanning and a discrimination task during scanning with phonetic (P) and nonphonetic (N) sine-wave sounds, both before (Pre) and after (Post) being exposed to the phonetic properties of the P sounds. Behaviorally, experience with the P sounds induced categorical identification of these sounds. In the PostP > PreP and PostP > PostN contrasts, an area in the posterior LSTG/S was activated. For both P and N sounds, the activation in this region was correlated with the degree of categorical identification in individual subjects. The results suggest that these areas in the posterior LSTG/S are sensitive neither to the acoustic properties of speech nor merely to the presence of phonetic information, but rather to the listener's awareness of category representations for auditory inputs.

In previous functional neuroimaging studies, left anterior temporal and temporal-parietal areas responded more strongly to sentences than to randomly ordered lists of words. The smaller response for word lists could be explained by either (1) less activation of syntactic processes due to the absence of syntactic structure in the random word lists or (2) less activation of semantic processes resulting from failure to combine the content words into a global meaning. To test these two explanations, we conducted a functional magnetic resonance imaging study in which word order and combinatorial word meaning were independently manipulated during auditory comprehension. Subjects heard six different stimuli: normal sentences, semantically incongruent sentences in which content words were randomly replaced with other content words, pseudoword sentences, and versions of these three sentence types in which word order was randomized to remove syntactic structure. Effects of syntactic structure (greater activation to sentences than to word lists) were observed in the left anterior superior temporal sulcus and left angular gyrus. Semantic effects (greater activation to semantically congruent stimuli than either incongruent or pseudoword stimuli) were seen in widespread, bilateral temporal lobe areas and the angular gyrus. Of the two regions that responded to syntactic structure, the angular gyrus showed a greater response to semantic structure, suggesting that reduced activation for word lists in this area is related to a disruption in semantic processing. The anterior temporal lobe, on the other hand, was relatively insensitive to manipulations of semantic structure, suggesting that syntactic information plays a greater role in driving activation in this area.

The generation of regular and irregular past tense verbs has been an important issue in cognitive science and has been used to advance different models of the organization of language in the brain. The dual-system view holds that the regular past tense forms are generated by a rule while irregular forms are retrieved from memory. The single-system view, on the other hand, holds that both forms are generated by a single integrated system and differ only in their reliance on factors such as phonology and semantics. We conducted an event-related fMRI study to examine the activation patterns associated with the generation and reading of regular and irregular past tense forms, in addition to the reading of their stems. Regular and irregular past tense generation activated similar brain regions compared to the reading of their respective stems. The areas activated more for irregular generation compared to regular generation included inferior frontal, precentral, and parietal regions bilaterally. This activation can be interpreted as reflecting the greater attentional and response selection demands of irregular generation. Compared to irregular generation, regular generation activated a small region in the left superior temporal gyrus when the regular and irregular past tense forms were mismatched on phonological complexity. No areas were more activated for regulars than irregulars when the past tense forms were matched on this variable. This suggests that the activation specific to regulars was related to the higher phonological complexity of their past tense forms rather than to their generation. A contrast of the reading of regular and irregular past tense forms was consistent with this hypothesis. These results support a single-system account of past tense generation.

Sine wave replicas of spoken words can be perceived both as nonphonetic auditory forms and as words, depending on a listener's experience. In this study, brain areas activated by sine wave words were studied with fMRI in two conditions: when subjects perceived the sounds spontaneously as nonphonetic auditory forms (“naïve condition”) and after instruction and brief practice attending to their phonetic attributes (“informed condition”). The test items were composed such that half replicated natural words (“phonetic items”) and the other half did not, because the tone analogs of the first and third formants had been temporally reversed (“nonphonetic items”). Subjects were asked to decide whether an isolated tone analog of the second formant (T2) presented before the sine wave word (T1234) was included in it. Experience in attending to the phonetic properties of the sinusoids interfered with this auditory matching task and was accompanied by a decrease in auditory cortex activation with word replicas but not with the acoustically matched nonphonetic items. Because the activation patterns elicited by equivalent acoustic test items depended on a listener's awareness of their phonetic potential, this indicates that the analysis of speech sounds in the auditory cortex is distinct from the simple resolution of auditory form, and is not a mere consequence of acoustic complexity. Because arbitrary acoustic patterns did not evoke the response observed for phonetic patterns, these findings suggest that the perception of speech is contingent on the presence of familiar patterns of spectral variation. The results are consistent with a short-term functional reorganization of auditory analysis induced by phonetic experience with sine wave replicas and contingent on the dynamic acoustic structure of speech.

Task-induced deactivation (TID) refers to a regional decrease in blood flow during an active task relative to a “resting” or “passive” baseline. We tested the hypothesis that TID results from a reallocation of processing resources by parametrically manipulating task difficulty within three factors: target discriminability, stimulus presentation rate, and short-term memory load. Subjects performed an auditory target detection task during functional magnetic resonance imaging (fMRI), responding to a single target tone or, in the short-term memory load conditions, to target sequences. Seven task conditions (a common version and two additional levels for each of the three factors) were each alternated with “rest” in a block design. Analysis of covariance identified brain regions in which TID occurred. Analyses of variance identified seven regions (left anterior cingulate/superior frontal gyrus, left middle frontal gyrus, right anterior cingulate gyrus, left and right posterior cingulate gyrus, left posterior parieto-occipital cortex, and right precuneus) in which TID magnitude varied across task levels within a factor. Follow-up tests indicated that for each of the three factors, TID magnitude increased with task difficulty. These results suggest that TID represents reallocation of processing resources from areas in which TID occurs to areas involved in task performance. Short-term memory load and stimulus rate also predict suppression of spontaneous thought, and many of the brain areas showing TID have been linked with semantic processing, supporting claims that TID may be due in part to suspension of spontaneous semantic processes that occur during “rest” (Binder et al., 1999). The concept that the typical “resting state” is actually a condition characterized by rich cognitive activity has important implications for the design and analysis of neuroimaging studies.

The ease by which movements are combined into skilled actions depends on many factors, including the complexity of movement sequences. Complexity can be defined by the surface structure of a sequence, including motoric properties such as the types of effectors, and by the abstract or sequence-specific structure, which is apparent in the relations amongst movements, such as repetitions. It is not known whether different neural systems support the cognitive and the sensorimotor processes underlying different structural properties of sequential actions. We investigated this question using whole-brain functional magnetic resonance imaging (fMRI) in healthy adults as they performed sequences of five key presses involving up to three fingers. The structure of sequences was defined by two factors that independently lengthen the time to plan sequences before movement: the number of different fingers (1-3; surface structure) and the number of finger transitions (0-4; sequence-specific structure). The results showed that systems involved in visual processing (extrastriate cortex) and the preparation of sensory aspects of movement (rostral inferior parietal and ventral premotor cortex (PMv)) correlated with both properties of sequence structure. The number of different fingers positively correlated with activation intensity in the cerebellum and superior parietal cortex (anterior), systems associated with sensorimotor, and kinematic representations of movement, respectively. The number of finger transitions correlated with activation in systems previously associated with sequence-specific processing, including the inferior parietal and the dorsal premotor cortex (PMd), and in interconnecting superior temporal-middle frontal gyrus networks. Different patterns of activation in the left and right inferior parietal cortex were associated with different sequences, consistent with the speculation that sequences are encoded using different mnemonics, depending on the sequence-specific structure. In contrast, PMd activation correlated positively with increases in the number of transitions, consistent with the role of this area in the retrieval or preparation of abstract action plans. These findings suggest that the surface and the sequence-specific structure of sequential movements can be distinguished by distinct distributed systems that support their underlying mental operations.

Whole-brain functional magnetic resonance imaging (MRI) was used to examine the neural substrates of internally (endogenous) and externally (exogenous) induced covert shifts of attention. Thirteen normal subjects performed three orienting conditions: endogenous (location of peripheral target predicted by a central arrow 80% of the time), exogenous (peripheral target preceded by a noninformative peripheral cue), and control (peripheral target preceded by noninformative central cue). Behavioral results indicated faster reaction times (RTs) for valid than for invalid trials for the endogenous condition but slower RTs for valid than for invalid trials for the exogenous condition (inhibition of return). The spatial extent and intensity of activation was greatest for the endogenous condition, consistent with the hypothesis that endogenous orienting is more effortful (less automatic) than exogenous orienting. Overall, we did not observe distinctly separable neural systems associated with the endogenous and exogenous orienting conditions. Both exogenous and endogenous orienting, but not the control condition, activated bilateral parietal and dorsal premotor regions, including the frontal eye fields. These results suggest a specific role for these regions in preparatory responding to peripheral stimuli. The right dorsolateral prefrontal cortex (BA 46) was activated selectively by the endogenous condition. This finding suggests that voluntary, but not reflexive, shifts of attention engage working memory systems.