Abstract

Can linguistic semantics affect neural processing in feature-specific visual regions? Specifically, when we hear a sentence describing a situation that includes motion, do we engage neural processes that are part of the visual perception of motion? How about if a motion verb was used figuratively, not literally? We used fMRI to investigate whether semantic content can “penetrate” and modulate neural populations that are selective to specific visual properties during natural language comprehension. Participants were presented audiovisually with three kinds of sentences: motion sentences (“The wild horse crossed the barren field.”), static sentences, (“The black horse stood in the barren field.”), and fictive motion sentences (“The hiking trail crossed the barren field.”). Motion-sensitive visual areas (MT+) were localized individually in each participant as well as face-selective visual regions (fusiform face area; FFA). MT+ was activated significantly more for motion sentences than the other sentence types. Fictive motion sentences also activated MT+ more than the static sentences. Importantly, no modulation of neural responses was found in FFA. Our findings suggest that the neural substrates of linguistic semantics include early visual areas specifically related to the represented semantics and that figurative uses of motion verbs also engage these neural systems, but to a lesser extent. These data are consistent with a view of language comprehension as an embodied process, with neural substrates as far reaching as early sensory brain areas that are specifically related to the represented semantics.

INTRODUCTION

The organization of semantic knowledge in the human brain is one of the most challenging questions in cognitive neuroscience (Binder, Desai, Graves, & Conant, 2009; Kemmerer, Gonzalez Castillo, Talavage, Patterson, & Wiley, 2008; Martin & Caramazza, 2003; Tyler & Moss, 2001; Warrington & McCarthy, 1994). Some researchers believe that higher order semantics relies primarily on amodal representations that are relatively encapsulated from perceptual and motor processes (e.g., Mahon & Caramazza, 2008; Caramazza & Mahon, 2006; Caramazza & Shelton, 1998; Fodor, 1983). Other researchers argue that instead, semantics and concepts are tightly interconnected with and grounded in the neural systems underlying sensory and motor processes—a view that has recently been discussed under the umbrella term, “embodiment” or “embodied cognition” (e.g., Martin, 2007; Pulvermüller, 2005; Glenberg & Kaschak, 2002; Barsalou, 1999; Damasio, 1989).

The embodied cognition view has been supported by theoretical arguments and experimental findings that indicate “simulation” may be a basic mechanism through which the nervous system processes information (Zwaan & Taylor, 2006; Jeannerod, 2001). The visual perception of others' body movements and actions has been a paradigm case in this field. When we see other individuals' actions, brain areas in frontal cortex that are involved in planning and executing our own movements are activated (Pavlova, Bidet-Ildei, Sokolov, Braun, & Krageloh-Mann, 2009; Rizzolatti & Craighero, 2004; Saygin, Wilson, Hagler, Bates, & Sereno, 2004), and inversely, damage to these areas can cause impairments in the perception of others' body movements (Saygin, 2007; Pobric & Hamilton, 2006). The hypothesis that the brain may carry out a partial internal simulation of seen body movements is further supported by experiments that manipulated visual and motor experience with the movements perceived (Calvo-Merino, Grezes, Glaser, Passingham, & Haggard, 2006; Casile & Giese, 2006).

In recent years, numerous studies have suggested that similar principles apply to the processing of conceptual and linguistic information. The majority of experiments have again focused on the motor domain. For example, the semantic processing of action verbs relating to different body parts evokes somatotopically specific activity in motor regions (Raposo, Moss, Stamatakis, & Tyler, 2009; Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006; Tettamanti et al., 2005; Hauk, Johnsrude, & Pulvermuller, 2004). These studies are complemented by additional data suggesting that language processing relies on motor representations that are congruent with the semantics of the stimuli (Boulenger, Mechtouff, et al., 2008; Grossman et al., 2008; Pulvermuller, Hauk, Nikulin, & Ilmoniemi, 2005; Saygin, Wilson, Dronkers, & Bates, 2004; Tranel, Kemmerer, Damasio, Adolphs, & Damasio, 2003). Similar findings are increasingly reported outside the motor system—for example, in gustatory (Simmons, Martin, & Barsalou, 2005), olfactory (Gonzalez et al., 2006), and visual modalities (Simmons et al., 2007; Pulvermuller et al., 2006; Zwaan, Stanfield, & Yaxley, 2002; Chao, Haxby, & Martin, 1999).

We wondered whether natural language comprehension may entail a rough “perceptual simulation” of specific visual features expressed in language. In particular, we used fMRI to explore whether BOLD responses in motion-sensitive cortex in human posterior lateral temporal cortex (PLTC), henceforth MT+,1 is modulated by motion events expressed in sentences. Importantly, we used sentences containing motion verbs that were used either literally (i.e., to describe a situation with physical motion, “I drove from Modesto to Fresno”) or figuratively (i.e., fictive motion, “The highway runs from Modesto to Fresno”; Talmy, 1996, 2000).

A number of behavioral studies have raised the possibility that during the processing of motion information expressed in language, motion-sensitive neural mechanisms (e.g., direction selective MT cells) are automatically engaged. Kaschak et al. (2005) reported that simultaneously presented visual directional motion (e.g., upward) affected RTs to sentences describing congruent movement (e.g., “The balloon ascended into the clouds”). Meteyard, Zokaei, Bahrami, and Vigliocco (2008) have shown that near-threshold visual motion interacts with lexical decision on congruent motion words. The same authors also reported that sensitivity, response bias, and RT in directional motion perception from random dot kinematograms were affected by aurally presented motion verbs (Meteyard, Bahrami, & Vigliocco, 2007).

The physical presence of visual motion is not a necessary condition for evoking neural responses in motion-sensitive cortical areas. Imagined motion, implied motion, and still images with representational momentum can engage motion selective neural mechanisms (Winawer, Huk, & Boroditsky, 2008; Slotnick, Thompson, & Kosslyn, 2005; Krekelberg, Dannenberg, Hoffmann, Bremmer, & Ross, 2003; Kourtzi & Kanwisher, 2000; Senior et al., 2000). Processing linguistic stimuli related to actions or motion can also activate regions in PLTC that are close to MT (Kable, Kan, Wilson, Thompson-Schill, & Chatterjee, 2005; Noppeney, Josephs, Kiebel, Friston, & Price, 2005; Wallentin, Lund, Ostergaard, Ostergaard, & Roepstorff, 2005; Kable, Lease-Spellmeyer, & Chatterjee, 2002; Damasio et al., 2001; Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995). Indeed, using a paradigm in which subjects acquired new perceptual–lexical associations, Revill, Aslin, Tanenhaus, and Bavelier (2008) found that PLTC (including anterior MT+) is automatically activated during the real-time processing of motion semantics.

However, it has not been possible to unambiguously conclude from these studies that language modulates motion-sensitive visual cortex. Many prior studies were not focused on motion, but rather on action versus object semantics, and the conceptual processing of action information (see also, Tranel, Manzel, Asp, & Kemmerer, 2008; Tyler, Randall, & Stamatakis, 2008; Tranel, Martin, Damasio, Grabowski, & Hichwa, 2005). Thus, although they are related, these studies were not designed to specifically address the question of whether the processing of motion information in language involves modulation of neural activity in primary motion-sensitive brain areas. Furthermore, because MT+ was not always functionally localized in these studies and given the multimodal, patchy organization of temporal cortex (Beauchamp, 2005), it was not possible to know whether activations were actually in visual motion-sensitive regions for a given individual. In fact, Kable et al. (2002) noted that the region that showed modulation was anterior to MT+ as identified with a localizer. Bedny, Caramazza, Grossman, Pascual-Leone, and Saxe (2008) failed to find modulation of (functionally localized) MT+ by motion semantics as subjects made semantic relatedness judgments of verbs and nouns and have argued that activations in PLTC reflect the retrieval of concepts or grammatical information rather than perceptual mechanisms that are recruited during language processing. It is therefore possible that activations found in previous studies are in regions other than MT+, and the behavioral effects observed between linguistic and motion processing have their neural source outside this region.

The present study was designed to address whether motion-related semantics in fact modulate neural activity in MT+. Using fMRI, we measured activation in individually localized MT+ ROIs in both hemispheres as participants were presented with sentences with or without motion event semantics. Given the multimodal nature of lateral temporal cortex, the proximity of language-related areas as well as intersubject variability in the location of functional visual areas (Dumoulin et al., 2000), it was important to localize MT+ functionally within each individual. To provide an ROI that was not expected to be modulated by motion events, we also individually localized the FFA, a region in inferior temporal cortex that shows higher activity for viewing faces compared with other classes of visual stimuli (Kanwisher, McDermott, & Chun, 1997).

We presented participants with three kinds of sentences: motion sentences, which included a motion verb as well as carrying motion semantics (e.g., “The deer jumped over the brook”); static sentences, which did not depict any motion (e.g., “The deer slept next to the brook”); and fictive motion sentences (Talmy, 1996, 2000), which featured a motion verb but the overall semantics of the sentence did not denote motion (e.g., “The bridge jumped over the brook”).

The addition of the fictive motion condition allows us to address important questions regarding embodied representations for semantics (Aziz-Zadeh & Damasio, 2008). Previous research has shown that effector-specific activations that are evoked in motor cortex during language processing are no longer apparent if the same words are used in metaphors (Aziz-Zadeh et al., 2006) or idioms (Raposo et al., 2009). These findings suggest that the overall semantics conveyed rather than the processing of the individual words is critical in these effects.

On the other hand, a strong version of the embodied cognition view would suggest that even the figurative use of motion or action words would engage simulation mechanisms, and supporting data have recently been reported (Boulenger, Hauk, & Pulvermuller, 2008; Glenberg et al., 2008). Also consistent with this prediction, behavioral and eye tracking studies indicate that processing fictive motion automatically entails a mental simulation of motion (Richardson & Matlock, 2007; Matlock, 2004). Using fMRI, Wallentin et al. (2005) found that processing fictive motion activated PLTC. Similarly, Chen, Widick, and Chatterjee (2008) reported increased activation in temporal cortex for sentences with predicate metaphors (Chen et al., 2008). Rüschemeyer, Brass, and Friederici's (2007) observation of an increased response in PLTC for abstract verbs with motor stems compared with those with abstract stems (e.g., verzeichnen/to denote vs. bezaubern/to charm) is also consistent with these studies (Rüschemeyer et al., 2007). However, in these experiments, the regions identified appeared anterior and/or superior to reported coordinates of MT+, which was not functionally mapped in the individual participants.

To summarize, we ask first whether motion sentences modulate neural responses specifically within participants' own functionally mapped MT+ ROIs. If no MT+ modulation is observed, this will suggest that processing motion information in language does not involve a neural simulation specific enough to activate motion-sensitive visual cortex—or as with any null result, the effect might be too small to be measured with the present methods and sample size. If comprehending motion information in natural language does modulate MT+, this would support the hypothesis that processing motion semantics involves simulation of the visual perception of motion. Further, if we find an equal modulation of MT+ by fictive motion and by motion sentences, this result would suggest that the response in MT+ is due simply to the use of a motion verb and not indexed to the overall semantics conveyed by the sentence. On the other hand, if fictive motion sentences pattern like static sentences and do not modulate MT+, we may conclude that sentential semantics makes the critical contribution to the MT+ response. A third possibility is that fictive motion sentences may cause a greater MT+ response than static sentences but a weaker response than true motion sentences. This result would indicate that MT+ is responsive to the degree of motion semantics expressed within a sentence. Finally, we do not expect fictive motion sentences to modulate MT+ in the absence of modulation by motion sentences, nor do we expect any of the sentence types to differentially modulate the FFA.

METHODS

Participants

Participants (N = 16, 7 women, aged 21–33 years, mean age = 27 years) had normal vision and hearing by self-report, were right-handed, and monolingual English speakers. Informed consent was obtained according to local ethics (University of California, San Diego, and San Diego State University). Data from one subject could not be used due to excessive head movement. All subjects participated in the main experiment and the MT+ localizer, but because of timing constraints, only 11 subjects additionally completed the FFA localizer (see below).

Stimuli

Three types of sentence stimuli were presented: motion sentences contained a motion verb and described an event with movement; static sentences described an event with little or no motion; and fictive motion sentences contained a motion verb used nonliterally such that the overall semantics were similar to the static sentences. Examples are provided in Table 1.

Table 1. 

Example Sentences

Motion The deer jumped over the brook 
I drove from Modesto to Fresno. 
The delivery trucks hurry across the Green Valley. 
The sports car went from Del Mar to La Jolla Shores. 
Fictive The bridge jumped over the brook. 
The highway runs from Modesto to Fresno. 
The train tracks hurry across the Green Valley. 
The bike trail went from Del Mar to La Jolla Shores. 
Static The deer slept next to the brook. 
Modesto and Fresno are in California 
Marshes occupy part of the Green Valley. 
The sports car was parked in the center of Del Mar. 
Anomalous The bus goes from the color to the briefcase. 
The wooden fence crosses the late curfew. 
The Swiss village was nestled at the foot of the giraffe. 
Motion The deer jumped over the brook 
I drove from Modesto to Fresno. 
The delivery trucks hurry across the Green Valley. 
The sports car went from Del Mar to La Jolla Shores. 
Fictive The bridge jumped over the brook. 
The highway runs from Modesto to Fresno. 
The train tracks hurry across the Green Valley. 
The bike trail went from Del Mar to La Jolla Shores. 
Static The deer slept next to the brook. 
Modesto and Fresno are in California 
Marshes occupy part of the Green Valley. 
The sports car was parked in the center of Del Mar. 
Anomalous The bus goes from the color to the briefcase. 
The wooden fence crosses the late curfew. 
The Swiss village was nestled at the foot of the giraffe. 

Fictive motion sentences were created using the framework of Talmy (1996, 2000). Care was taken to create motion and static sentences that matched as closely as possible in length and content to the fictive motion sentences. We first generated an initial set of 303 sentences in the form of “triplets,” in which each triplet of sentences (motion, static, and fictive motion) contained approximately the same number of words, similar syntactic structure, and the same or comparable content words. This procedure was adopted so that sentences would be matched across conditions for length and for factors such as sentence structure and word frequency.

The initial 303 sentences were rated by two separate groups of native English speakers on the naturalness of the sentences and on the vividness of motion content. In the first norming survey, six native English speakers read each sentence and marked any sentence that seemed incorrect or odd in any way. For each sentence they marked, they were asked either to underline the problematic portions or to write comments. Any sentence that was marked by more than one subject was rejected and removed from the list of sentences to be used in the experiment. If a sentence was marked only once, the sentence was revised on the basis of the suggestions provided by the subject. Changes made at this stage were removal of prepositional phrases or unfamiliar words and a few semantic substitutions made in line with subjects' comments (e.g., substituting the word path for road, or deer for goat).

In the second norming survey, a separate group of 21 native English speakers were administered a questionnaire in which they were asked to indicate the vividness of the movement described in each sentence on a 5-point scale (0 = no motion, 4 = a lot of motion). As a guide, subjects were given two examples of sentences containing relatively little motion semantics (“The horse slept in the ravine next to the large rock” and “The mountain range sits between Mexico and Canada”) and two sentences containing a high amount of motion (“His horse galloped from the hills into the ravine” and “I drove rapidly to the hotel in Santa Monica”). Participants were told that the questionnaire would contain items with different levels of movement depicted in them and were asked to rate the vividness of the motion in each item. They were also encouraged to close their eyes for a moment after reading each sentence. There were seven practice sentences, including two motion, two fictive motion, and three static sentences. The remainder of the questionnaire had 280 sentences in random order. Mean motion ratings were calculated for each sentence on the basis of the results of this questionnaire.

The experimental sentences were selected such that (a) motion sentences had high motion ratings, (b) static sentences had low motion ratings, (c) fictive motion sentences came from the same triplet with a motion or static sentence that was retained in the final set (preferably both), and (d) sentences with highly variable motion ratings were excluded.

Sixty-five sentences for each condition were selected for use in the fMRI study. In this final set of sentences, motion ratings were significantly different between the static, fictive, and motion sentences (p < .0001 corrected for multiple comparisons). As shown in Supplementary Figure 1, static sentences had a mean rating of 0.26 (SD = 0.15), fictive motion sentences had a mean rating of 1.12 (SD = 0.25), and motion sentences had a mean of 2.77 (SD = 0.46). Thus, fictive sentences had an intermediate rating between static and motion sentences, but they also had ratings that were much closer to those for the static sentences. The final sentences did not differ in length (average length in words for motion sentences = 8.11; for fictive sentences = 8.17; for static sentences = 8.21; p > .1 for pairwise comparisons). The final sentences also did not differ in the average frequency of the content words (average CELEX frequency for words in motion sentences = 120.73; for fictive sentences = 116.63; for static sentences = 117.24; p > .1 for pairwise comparisons).

Figure 1. 

MT+ (A) and FFA (B) ROIs and BOLD responses. Left panel: Axial MRI images, with the approximate location of the ROIs marked with yellow crosses. Middle panel: BOLD response averaged across subjects and both hemispheres for motion (green), fictive motion (blue), and static (red) sentences plotted over time. The y-axis is percent signal change with respect to baseline. Error bars are SEM. Right panel: Mean signal change shown separately for the left and right hemispheres for motion (green), fictive motion (blue), and static (red) sentences. The y-axes are the same as middle panel. Error bars are SEM. *Significant pairwise comparisons (p < .001, corrected for multiple comparisons).

Figure 1. 

MT+ (A) and FFA (B) ROIs and BOLD responses. Left panel: Axial MRI images, with the approximate location of the ROIs marked with yellow crosses. Middle panel: BOLD response averaged across subjects and both hemispheres for motion (green), fictive motion (blue), and static (red) sentences plotted over time. The y-axis is percent signal change with respect to baseline. Error bars are SEM. Right panel: Mean signal change shown separately for the left and right hemispheres for motion (green), fictive motion (blue), and static (red) sentences. The y-axes are the same as middle panel. Error bars are SEM. *Significant pairwise comparisons (p < .001, corrected for multiple comparisons).

We also generated sentences resembling the experimental sentences in structure and content words but containing a semantic anomaly (see Table 1). These were used as the target sentences in the experimental task (see below).

We used audiovisual presentation of the sentences. There were at least three reasons that motivated this decision. First, our interest was whether motion information carried in sentences would evoke activity in MT+ during natural language processing. It is well known that neuroimaging studies of semantics can lead to different results depending on the linguistic class of the stimuli (e.g., verbs, nouns) and/or the task subjects carry out (e.g., semantic relatedness judgments, category judgments; see Devlin et al., 2002). Our choice of full, complete sentences along with audiovisual stimuli allows participants to process language for meaning in a manner that is relatively naturalistic and similar to everyday communication. Second, selectively attending stimuli in one modality can suppress the BOLD signal in sensory areas for other modalities (e.g., Lewis, Beauchamp, & DeYoe, 2000), and we wanted to avoid top–down suppression of MT+ (cf. Bedny et al., 2008), which would decrease the power to detect subtle modulations of MT+ activity. Third, audiovisual presentation allows us to compare our results at least qualitatively with data from related experiments investigating MT+ modulation by motion semantics in sign languages, where the presentation of the stimuli will necessarily include the visual modality (McCullough, Saygin, Alac, & Emmorey, 2008; MacSweeney et al., 2002).

Note that neither MT+ nor FFA would be expected to show differential responses to the sentence types simply because the stimuli are audiovisual. Although more visual motion may be present in the sentence blocks compared with baseline (see below), there is no reason to expect differential responses in MT+ or FFA between the three sentence types.

A female native speaker of English (who was blind to the goals of the study) was videotaped uttering each sentence with a neutral facial expression as she looked directly at the camera. The speaker produced the sentences in a randomized order. These recordings were then edited into video clips with duration 2.26–5.73 sec (mean = 3.89 sec). Video clips of the same speaker sitting in the same position, but remaining silent, with a small amount of natural movement (e.g., blinks, breathing, small torso movements) were used in the baseline (see below).

Procedure

fMRI data were collected using a 3-T GE Signa EXCITE scanner equipped with an eight-element phased-array head coil. In all experiments, echo-planar volumes were acquired from the whole brain with repetition time of 2000 msec, echo time of 30 msec, 3.5 mm in-plane resolution, and 4 mm slice thickness and no gap.

We used a blocked design with 30-sec blocks of sentences and 15-sec blocks of baseline. In each run, there were nine sentence and nine baseline blocks. The low-level baseline condition was included to have a common reference against which we can measure responses to the sentences and aimed to present some visual and auditory stimuli and a simple button press task. We interspersed these blocks in between each experimental block so that our comparisons of interest were always presented after the same baseline. The sentence blocks were presented in pseudorandom order, where two blocks of the same type (e.g., motion sentences) were not presented consecutively (even with a baseline block between them). On average, each experimental block contained eight sentences (range = 7–9), presented with 250-msec ISI between sentences, and an additional 200–700 msec between blocks, depending on the total duration of the sentences in each block.

The experimental task was designed simply to keep subjects alert and was intended to direct their attention to the meaning of the sentences, without focusing specifically on the motion information. Subjects were asked to press a button when they encountered a semantically anomalous sentence. These sentences were generated to be semantically and grammatically similar to the experimental sentences but were rendered anomalous through word substitution. The anomalous target sentence was never the first or the last sentence in a block. In baseline blocks, subjects monitored whether a continuous pure tone presented along with the video stimuli changed frequency. These “catch” trials occurred on average once per block (range = 0–2 times) and were balanced across conditions.

We used separate localizer scans to identify MT+ and FFA. In the MT+ localizer, subjects viewed 32-sec blocks of white dots on a black background that were either moving (flow fields) or stationary (Bavelier et al., 2001). In the FFA localizer, subjects viewed 20-sec blocks of pictures of faces, houses, scrambled faces, and scrambled houses as they performed a one-back working memory task.

The main experiment consisted of 3 runs of 273 volumes each (9.1 min). In addition, all but one of the subjects completed 131 volumes for the MT+ localizer (4.4 min). Eleven subjects participated in the FFA localizer, which was 163 volumes (5.4 min). In each run, the experimental stimuli began after three volumes to allow the magnetization to reach steady state.

For each participant, a 1 × 1 × 1.3 mm anatomical scan was also collected, usually in the middle of the scanning session. We also collected B0 field maps, which were used to correct for distortions in the functional images in the phase-encode direction (Reber, Wong, Buxton, & Frank, 1998).

Stimuli were presented using a Macintosh PowerBook G4 computer (Apple, Cupertino, CA). Video stimuli for the main experiment were presented using QuickTime (Apple, Cupertino, CA). Participants' keypresses were collected through Matlab (Mathworks, Natick, MA) with the Psychophysics Toolbox extensions (Brainard, 1997) running on a separate computer. MT+ localizer was presented using Matlab, and the FFA localizer was presented using PsyScope X Software (http://psy.ck.sissa.it).

Visual stimuli were projected onto a screen that was set up at the foot of the scanner bed, using an XGA video projector and a custom lens (Buhl Optical, Rochester, NY). Subjects viewed the video stimuli through a mirror that was placed inside the head coil and simultaneously heard the audio track through MRI-compatible headphones (Resonance Technology, Northridge, CA). They responded using an MR-safe Lumitouch button box (Photon Control, Burnaby, Canada).

All subjects completed a short practice run of the experiment outside the scanner so that they were familiarized with the kinds of sentences in the study. Before running the experiment, sound volume was adjusted individually for each subject such that the stimuli were loud enough to hear and identify over scanner noise with earplugs, but not too loud to cause discomfort.

Data Analysis

Image preprocessing and statistical analyses were performed using Analysis of Functional Neuroimages software (Cox, 1996). We used a random effects approach within the framework of the general linear model. For each subject, the B0 field maps were used to correct for distortions, and the runs were concatenated and spatially registered in three-dimensional space for head movement correction. The data were normalized by the mean at each voxel before deconvolution. The general linear model coefficients were estimated using the Analysis of Functional Neuroimages programs 3dDeconvolve along with 3dREMLfit, the latter of which allows a restricted maximum likelihood estimation of the temporal autocorrelation structure. We used a second order polynomial baseline fit and the three translation and three rotation parameters from the motion correction as nuisance regressors. We estimated and plotted percent signal change from baseline in our ROIs for the block duration plus one repetition time before and after. To statistically assess modulation of the BOLD response in our ROIs, we used mean percent signal change (obtained by averaging time points other than 0 and 1 as the hemodynamic response is not stabilized in this time range) as a dependent measure in a repeated measures ANOVA with Sentence Type (motion, fictive, static) and Hemisphere (left, right) as factors.

We designed our study to test a very specific hypothesis within independently identified ROIs (MT+ and FFA). We conducted whole-brain analyses only for completeness. For these we used a 6-mm FWHM Gaussian smoothing and transformed each subject's data into stereotaxic (Talairach) space.

ROIs were defined using our localizer scans plus anatomical constraints. The most reliable landmark for area MT is near the junction of the inferior temporal sulcus and the ascending limb of the inferior temporal sulcus (Dumoulin et al., 2000; Watson et al., 1993). To obtain the MT+ ROIs, we selected a cluster of voxels showing a strong (p < 10−10) selectivity for motion in this anatomical region. For the FFA, voxels in the fusiform gyrus responding significantly more to pictures of faces than to pictures of houses (p < 10−6) were selected (Kanwisher et al., 1997). ROIs had to be at least five voxels in size. The average ROI size was 27.28 voxels (SD = 20.2) for MT+ and 61.5 voxels (SD = 71.2) for the FFA ROI.

One subject was excluded from all analyses because of excessive head movement in multiple scans. We also could not use the MT+ localizer from another subject because of head movement. In two subjects, MT+ was clearly identified only in one hemisphere, and in four subjects, FFA was clearly identified in one hemisphere. In the end, we had a total of 13 right and 13 left hemisphere ROIs for MT+ and 9 right and 9 left hemisphere ROIs for FFA.

RESULTS

Behavioral Data

Subjects were highly accurate in performing the sentence comprehension task (mean accuracy = 0.96), indicating good attention to the stimuli and task. Subjects were even more accurate in the baseline task (mean accuracy = 0.99). The difference in accuracy between tasks was significant (paired t test p < .01), but performance did not vary between sentence types (p > .1).

fMRI Data

The mean coordinates for the ROIs were as follows: left MT+: x = −43.6 ± 5.3, y = −71.6 ± 4.5, z = 4.55 ± 5.7; right MT+: x = 42.6 ± 4.8, y = −69.0 ± 3.4, z = 2.50 ± 5.9; left FFA: x = −38.9 ± 3.4, y = −54.6 ± 7.2, z = −8.3 ± 3.5; and right FFA: x = 41.00 ± 1.9, y = −57.6 ± 5.4, z = −9.5 ± 3.3, consistent with prior work (Dumoulin et al., 2000; Tootell et al., 1995; Watson et al., 1993; Zeki et al., 1991) and with the location of the cytoarchitectonic correlate of human MT+ proposed by Malikovic et al. (2007). The mean locations of these ROIs are approximately marked in Figure 1, leftmost panel. The peaks of the MT+ ROI varied considerably between subjects (as much as 17 mm in x, 16 mm in y, and 15 mm in z directions; see also Dumoulin et al., 2000), highlighting the importance of localizing this region in individual subjects. This location was more consistent with previously published coordinates for MT rather than MST (Dukelow et al., 2001), but the ROIs likely contained voxels from both areas.

The middle panel of Figure 1 shows time courses of the BOLD signal for the three sentence types in MT+ and FFA. In general, the signal was stronger in FFA compared with MT+ (p < .0001). Activations were slightly stronger in the left hemisphere, consistent with its dominance for language processing. In MT+, BOLD responses were greatest for motion sentences bilaterally, whereas fictive motion sentences activated MT+ less than motion sentences but more than the static sentences. Static sentences led to decreased activity in MT+ bilaterally, whereas fictive motion sentences led to a small decrease in activity only in the left hemisphere. In contrast to MT+, all sentence types activated the FFA ROIs similarly.

To quantify these differences in BOLD response across sentence types, we ran a 2 × 3 repeated measures ANOVA for mean percent signal change in MT+ and FFA, with Sentence Type (motion, fictive, static) and Hemisphere (left, right) as factors. For MT+, there was a significant main effect of Sentence Type, F(2, 24) = 24.78, p < .0001 (all pairwise t tests significant, p < .001 corrected; see Figure 1). We found no main effect of Hemisphere (F < 1, p = .89). There was a significant Sentence Type × Hemisphere interaction, F(2, 24) = 11.23, p < .0001: Activation was stronger in the left hemisphere for motion sentences and in the right hemisphere for fictive motion sentences.

For the FFA, the same ANOVA yielded a significant effect of Hemisphere, F(1, 6) = 12.14, p < .05; the FFA response to all sentences was stronger in the left hemisphere. There was no effect of Sentence Type (F < 1, p = .98) nor an interaction between Sentence Type and Hemisphere (F < 1, p = .93).

It is unlikely that we failed to observe significant effects other than Hemisphere in this ROI because we had slightly fewer subjects with FFA data, since the sample size was still large enough to detect effects of similar size as those found in MT+ (for Sentence Type, 1-β = 0.67, for Sentence Type × Hemisphere, 1-β = 0.66).

For completeness, we also conducted a standard whole-brain analysis of the fMRI data, which is reported in Supplementary data.

DISCUSSION

Using a set of carefully normed sentences with and without motion semantics (motion, fictive motion, and static sentences), we studied the modulation of neural activity in human MT+ during natural language comprehension. Motion sentences activated MT+ significantly more than the fictive motion and static sentences. Fictive motion also led to a small but significant increase in response in MT+ compared with static sentences. No differential modulation was found between the sentence types in our control ROI, face-sensitive region FFA.

The modulation of MT+ by motion sentences suggests that semantic processing of visual features depicted in linguistic stimuli may involve a rough neural simulation of seeing those visual stimuli. This finding is consistent with a number of behavioral studies suggesting language can interact with low-level motion processing (Meteyard et al., 2008; Meteyard et al., 2007; Kaschak et al., 2005). Our results also complement previous neuroimaging experiments that found PLTC responds preferentially to action or motion-related stimuli (Kable et al., 2002; Damasio et al., 2001; Martin et al., 1995). Here, we were able to show that such responses are localized precisely to visual motion-sensitive areas as identified in individual subjects (see also, Revill et al., 2008).

In addition, our results mirror those of McCullough et al. (2008) for American Sign Language (ASL). In this study, ASL sentences with motion semantics (e.g., “A man goes around the corner”) were found to modulate activity in MT+ (identified by the same motion perception task used here) to a greater extent than ASL sentences with static semantics (e.g., “A man stands at the corner.”). The finding that MT+ is sensitive to motion semantics in both English and ASL (and possibly in British Sign Language, see MacSweeney et al., 2002) indicates that motion-selective visual cortex responds to the semantic content of language, regardless of the surface characteristics of how motion information is expressed, that is, by iconic movements of the hands or by noniconic audiovisual speech.

However, our results contrast with those of Bedny et al. (2008) who found no significant difference in MT+ activity for nouns or verbs with different motion ratings. As with any null result, their study may have lacked power to detect the modulations we were able to measure. Alternatively, the difference between these results might be an indication that the choice of stimuli and task influence whether modulations of visual areas by linguistic semantics can be detected. For example, although examples were not provided, the stimuli in their study were rather different from ours; they used nouns or verbs from specific semantic categories. The nouns were animals (high motion), tools (medium motion), or inanimate natural kinds (low motion). The verbs always described actions (high motion), change of state or bodily functions (medium motion), or mental states (low motion). Second, the participants were asked to make semantic relatedness judgments between pairs of words, a task that may focus attention to different and likely more abstract aspects of the stimuli (e.g., “Are both animals carnivores? Are the actions both related to sports?”). Attention to such features may lead to a decrease in signal within sensory areas and lead to less power to detect modulations by semantics. Indeed, MT+ and other visual ROIs were deactivated during this semantic judgment task (Bedny et al., 2008, pp. 11350). In sum, it appears that the choice of task and stimuli can influence the power to detect modulations of MT+ by linguistic semantics.

Our second major finding is that fictive motion sentences also led to increased activation in MT+ (and not in FFA) compared with static sentences. Although this finding is consistent with the idea that processing figurative language involves embodied representations (e.g., Glenberg et al., 2008; Boroditsky, 2000; Lakoff & Johnson, 1980), it is somewhat unexpected given results in the motor domain. Both Aziz-Zadeh et al. (2006) and Raposo et al. (2009) found modulations of individually mapped effector-specific motor regions by sentences that conveyed action semantics, but not for metaphorical or idiomatic uses of the same verbs. Conversely, a recent study found somatotopic modulation of responses in motor areas during idiom comprehension (Boulenger, Hauk, et al., 2008). Indeed, nonliteral uses of motion-related words have been shown to evoke neural responses in PLTC (Chen et al., 2008; Rüschemeyer et al., 2007; Wallentin et al., 2005). Here we showed specific modulation of MT+ by fictive motion. Furthermore, because our stimuli were novel and unlikely to have been overlearned, our results are consistent with Aziz-Zadeh & Damasio's (2008) suggestion that conventionalized uses of figurative language might be less likely to rely on embodied literal representations compared with novel uses.

The response in MT+ by fictive motion compared with static sentences was relatively (though not selectively) right lateralized, consistent with this hemisphere's known involvement in discourse level and figurative language processing (Coulson & Van Petten, 2007; Jung-Beeman, 2005; Mashal, Faust, & Hendler, 2005; Bottini et al., 1994). However, we should be cautious not to overinterpret this lateralization effect, as neuroimaging data on this topic have not been entirely consistent (e.g., Schmidt, DeBuse, & Seger, 2007; Lee & Dapretto, 2006).

Although fictive motion sentences activated MT+ significantly more than static sentences, they evoked a much smaller response compared with true motion sentences. This result shows that the overall semantics of the sentence makes a distinct and significant contribution to the MT+ response and that the modulation we are measuring here relates to semantic content rather than the mere presence of motion or action verbs. We suggest that activation in MT+ for motion sentences reflects the embodied processing of motion semantics. Behavioral studies have suggested that processing fictive motion entails a simulation process in motion processing mechanisms (Richardson & Matlock, 2007; Matlock, 2004), and the modulation of MT+ by our fictive motion sentences could reflect a neural correlate of such a simulation. An alternative possibility to explain the “in-between” response to fictive motion sentences is that the presence of the motion verbs or the processing of fictive motion semantics increases MT+ response, whereas the sentential semantics (a static scene) inhibits MT+. This hypothesis is also consistent with our observation that MT+ showed a decrease in response for the static sentences. These distinct kinds of modulation are not separable in the present fMRI data, and the specific contributions to the MT+ response as well as the possible sources of these modulatory signals must be explored in future studies.

It is unlikely for these data to reflect factors independent of the semantics of the sentences such as attention or general arousal, as there is no reason to expect attention or arousal to vary systematically between experimental blocks. Supporting this, the behavioral performance showed no difference across conditions. Furthermore, if attention was the major factor causing differences in response patterns, this would be expected to also affect FFA. Instead, in this region, we find robust signal for the stimuli but no differences between the conditions.

Finally, one might ask whether these results simply reflect visual imagery. It is not possible to answer this question unambiguously. Although researchers have studied consciously directed, willed imagery in various studies and found effects on sensorimotor areas (Munzert, Lorey, & Zentgraf, 2009; Kosslyn & Thompson, 2003), it is much more difficult to separate out effects of automatically and unconsciously engaged imagery processes (Jeannerod, 2001). Our experiment was very different to the typical “top–down” kind of visual imagery experiment, where participants are typically presented with as few sensory stimuli as possible and are asked to close their eyes and bring specific visual images to their mind's eye. Instead, our subjects were presented with audiovisual stimuli, were not asked to visualize the sentences, were focused on processing the sentences for anomalies in meaning, and were blind to our experimental hypotheses. Under these conditions, any contribution imagery makes to the neural responses is likely to be a component of language understanding in general. Because it will be very difficult to separate out the contributions of imagery to language processing (especially if mental imagery is automatically and unconsciously engaged; e.g., Bergen, Lindsay, Matlock, & Narayanan, 2007; Richardson, Spivey, Barsalou, & McRae, 2003), it may be more useful to consider whether imagery and linguistic semantics (or attention or context) modulate sensorimotor regions through similar or distinct mechanisms. Electrophysiological data may also shed light on the time course of cortical activation within peri-sylvian language regions and sensory regions associated with motion imagery and perception.

In sum, we have shown that linguistic semantics of motion modulates neural responses as early as visual areas that subserve motion processing. These data are consistent with a view of language comprehension as a distributed and embodied process, with neural substrates as far reaching as relatively early sensory and motor areas that are specifically related to the represented semantics.

Acknowledgments

The authors thank all staff at UCSD Center for fMRI and SDSU Laboratory for Language and Cognitive Neuroscience for support and assistance and Jon Driver and Frederic Dick for comments on an earlier draft. The project was supported by NIH grant R01 HD13249 (Emmorey). A. P. S. was additionally supported by a European Commission Marie Curie award FP6-025044 and a City University Research Fellowship.

Reprint requests should be sent to Dr. Ayse Pinar Saygin, Department of Cognitive Science, University of California San Diego, La Jolla, CA 92093-0515, or via e-mail: apsaygin@gmail.com.

Note

1. 

Area MT (or V5) is one of multiple regions in monkey superior temporal cortex known to be involved in motion perception (Ungerleider & Desimone, 1986). In the human brain, MT is found most commonly near the intersection of the ascending limb of the inferior temporal sulcus and lateral occipital sulcus (Dumoulin et al., 2000; Tootell et al., 1995; Watson et al., 1993; Zeki et al., 1991). However, precise human analogs of the different motion-sensitive areas in macaque temporal cortex (and possible cross-species differences) are not yet known (Nelissen, Vanduffel, & Orban, 2006; Sereno & Tootell, 2005). Standard localizers contrasting moving to static stimuli to identify human analog of MT almost certainly activate a region that includes other areas such as area MST; hence, the region is commonly referred to as MT+ unless further subdivisions are made on the basis of additional information such as retinotopy (Huk, Dougherty, & Heeger, 2002).

REFERENCES

Aziz-Zadeh
,
L.
, &
Damasio
,
A.
(
2008
).
Embodied semantics for actions: Findings from functional brain imaging.
Journal of Physiology (Paris)
,
102
,
35
39
.
Aziz-Zadeh
,
L.
,
Wilson
,
S. M.
,
Rizzolatti
,
G.
, &
Iacoboni
,
M.
(
2006
).
Congruent embodied representations for visually presented actions and linguistic phrases describing actions.
Current Biology
,
16
,
1818
1823
.
Barsalou
,
L. W.
(
1999
).
Perceptual symbol systems.
Behavioral and Brain Sciences
,
22
,
577
660
.
Bavelier
,
D.
,
Brozinsky
,
C.
,
Tomann
,
A.
,
Mitchell
,
T.
,
Neville
,
H.
, &
Liu
,
G.
(
2001
).
Impact of early deafness and early exposure to sign language on the cerebral organization for motion processing.
Journal of Neuroscience
,
21
,
8931
8942
.
Beauchamp
,
M. S.
(
2005
).
See me, hear me, touch me: Multisensory integration in lateral occipital-temporal cortex.
Current Opinion in Neurobiology
,
15
,
145
153
.
Bedny
,
M.
,
Caramazza
,
A.
,
Grossman
,
E.
,
Pascual-Leone
,
A.
, &
Saxe
,
R.
(
2008
).
Concepts are more than percepts: The case of action verbs.
Journal of Neuroscience
,
28
,
11347
11353
.
Bergen
,
B. K.
,
Lindsay
,
S.
,
Matlock
,
T.
, &
Narayanan
,
S.
(
2007
).
Spatial and linguistic aspects of visual imagery in sentence comprehension.
Cognitive Science
,
31
,
733
764
.
Binder
,
J. R.
,
Desai
,
R. H.
,
Graves
,
W. W.
, &
Conant
,
L. I.
(
2010
).
Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies.
Cerebral Cortex
,
19
,
2767
2769
.
Boroditsky
,
L.
(
2000
).
Metaphoric structuring: Understanding time through spatial metaphors.
Cognition
,
75
,
1
28
.
Bottini
,
G.
,
Corcoran
,
R.
,
Sterzi
,
R.
,
Paulesu
,
E.
,
Schenone
,
P.
,
Scarpa
,
P.
,
et al
(
1994
).
The role of the right hemisphere in the interpretation of figurative aspects of language. A positron emission tomography activation study.
Brain
,
117
,
1241
1253
.
Boulenger
,
V.
,
Hauk
,
O.
, &
Pulvermuller
,
F.
(
2008
).
Grasping ideas with the motor system: Semantic somatotopy in idiom comprehension.
Cerebral Cortex
,
19
,
1905
1914
.
Boulenger
,
V.
,
Mechtouff
,
L.
,
Thobois
,
S.
,
Broussolle
,
E.
,
Jeannerod
,
M.
, &
Nazir
,
T. A.
(
2008
).
Word processing in Parkinson's disease is impaired for action verbs but not for concrete nouns.
Neuropsychologia
,
46
,
743
756
.
Brainard
,
D. H.
(
1997
).
The psychophysics toolbox.
Spatial Vision
,
10
,
433
436
.
Calvo-Merino
,
B.
,
Grezes
,
J.
,
Glaser
,
D. E.
,
Passingham
,
R. E.
, &
Haggard
,
P.
(
2006
).
Seeing or doing? Influence of visual and motor familiarity in action observation.
Current Biology
,
16
,
1905
1910
.
Caramazza
,
A.
, &
Mahon
,
B. Z.
(
2006
).
The organisation of conceptual knowledge in the brain: The future's past and some future directions.
Cognitive Neuropsychology
,
23
,
13
38
.
Caramazza
,
A.
, &
Shelton
,
J. R.
(
1998
).
Domain-specific knowledge systems in the brain: The animate-inanimate distinction.
Journal of Cognitive Neuroscience
,
10
,
1
34
.
Casile
,
A.
, &
Giese
,
M. A.
(
2006
).
Nonvisual motor training influences biological motion perception.
Current Biology
,
16
,
69
74
.
Chao
,
L. L.
,
Haxby
,
J. V.
, &
Martin
,
A.
(
1999
).
Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects.
Nature Neuroscience
,
2
,
913
919
.
Chen
,
E.
,
Widick
,
P.
, &
Chatterjee
,
A.
(
2008
).
Functional-anatomical organization of predicate metaphor processing.
Brain and Language
,
107
,
194
202
.
Coulson
,
S.
, &
Van Petten
,
C.
(
2007
).
A special role for the right hemisphere in metaphor comprehension? ERP evidence from hemifield presentation.
Brain Research
,
1146
,
128
145
.
Cox
,
R. W.
(
1996
).
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages.
Computers in Biomedical Research
,
29
,
162
173
.
Damasio
,
A. R.
(
1989
).
Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition.
Cognition
,
33
,
25
62
.
Damasio
,
H.
,
Grabowski
,
T. J.
,
Tranel
,
D.
,
Ponto
,
L. L.
,
Hichwa
,
R. D.
, &
Damasio
,
A. R.
(
2001
).
Neural correlates of naming actions and of naming spatial relations.
Neuroimage
,
13
,
1053
1064
.
Devlin
,
J. T.
,
Russell
,
R. P.
,
Davis
,
M. H.
,
Price
,
C. J.
,
Moss
,
H. E.
,
Fadili
,
M. J.
,
et al
(
2002
).
Is there an anatomical basis for category-specificity? Semantic memory studies in PET and fMRI.
Neuropsychologia
,
40
,
54
75
.
Dukelow
,
S. P.
,
DeSouza
,
J. F.
,
Culham
,
J. C.
,
van den Berg
,
A. V.
,
Menon
,
R. S.
, &
Vilis
,
T.
(
2001
).
Distinguishing subregions of the human MT+ complex using visual fields and pursuit eye movements.
Journal of Neurophysiology
,
86
,
1991
2000
.
Dumoulin
,
S. O.
,
Bittar
,
R. G.
,
Kabani
,
N. J.
,
Baker
,
C. L.
, Jr.,
Le Goualher
,
G.
,
Bruce Pike
,
G.
,
et al
(
2000
).
A new anatomical landmark for reliable identification of human area V5/MT: A quantitative analysis of sulcal patterning.
Cerebral Cortex
,
10
,
454
463
.
Fodor
,
J. A.
(
1983
).
The modularity of mind: An essay on faculty psychology.
Cambridge, MA
:
MIT Press
.
Glenberg
,
A. M.
, &
Kaschak
,
M. P.
(
2002
).
Grounding language in action.
Psychonomic Bulletin & Review
,
9
,
558
565
.
Glenberg
,
A. M.
,
Sato
,
M.
,
Cattaneo
,
L.
,
Riggio
,
L.
,
Palumbo
,
D.
, &
Buccino
,
G.
(
2008
).
Processing abstract language modulates motor system activity.
Quarterly Journal of Experimental Psychology
,
61
,
905
919
.
Gonzalez
,
J.
,
Barros-Loscertales
,
A.
,
Pulvermuller
,
F.
,
Meseguer
,
V.
,
Sanjuan
,
A.
,
Belloch
,
V.
,
et al
(
2006
).
Reading cinnamon activates olfactory brain regions.
Neuroimage
,
32
,
906
912
.
Grossman
,
M.
,
Anderson
,
C.
,
Khan
,
A.
,
Avants
,
B.
,
Elman
,
L.
, &
McCluskey
,
L.
(
2008
).
Impaired action knowledge in amyotrophic lateral sclerosis.
Neurology
,
71
,
1388
1389
.
Hauk
,
O.
,
Johnsrude
,
I.
, &
Pulvermuller
,
F.
(
2004
).
Somatotopic representation of action words in human motor and premotor cortex.
Neuron
,
41
,
301
307
.
Huk
,
A. C.
,
Dougherty
,
R. F.
, &
Heeger
,
D. J.
(
2002
).
Retinotopy and functional subdivision of human areas MT and MST.
Journal of Neuroscience
,
22
,
7195
7205
.
Jeannerod
,
M.
(
2001
).
Neural simulation of action: A unifying mechanism for motor cognition.
Neuroimage
,
14
,
3103
3109
.
Jung-Beeman
,
M.
(
2005
).
Bilateral brain processes for comprehending natural language.
Trends in Cognitive Sciences
,
9
,
512
518
.
Kable
,
J. W.
,
Kan
,
I. P.
,
Wilson
,
A.
,
Thompson-Schill
,
S. L.
, &
Chatterjee
,
A.
(
2005
).
Conceptual representations of action in the lateral temporal cortex.
Journal of Cognitive Neuroscience
,
17
,
1855
1870
.
Kable
,
J. W.
,
Lease-Spellmeyer
,
J.
, &
Chatterjee
,
A.
(
2002
).
Neural substrates of action event knowledge.
Journal of Cognitive Neuroscience
,
14
,
795
805
.
Kanwisher
,
N.
,
McDermott
,
J.
, &
Chun
,
M. M.
(
1997
).
The fusiform face area: A module in human extrastriate cortex specialized for face perception.
Journal of Neuroscience
,
17
,
4302
4311
.
Kaschak
,
M. P.
,
Madden
,
C. J.
,
Therriault
,
D. J.
,
Yaxley
,
R. H.
,
Aveyard
,
M.
,
Blanchard
,
A. A.
,
et al
(
2005
).
Perception of motion affects language processing.
Cognition
,
94
,
B79
B89
.
Kemmerer
,
D.
,
Gonzalez Castillo
,
J.
,
Talavage
,
T.
,
Patterson
,
S.
, &
Wiley
,
C.
(
2008
).
Neuroanatomical distribution of five semantic components of verbs: Evidence from fMRI.
Brain and Language
,
107
,
16
43
.
Kosslyn
,
S. M.
, &
Thompson
,
W. L.
(
2003
).
When is early visual cortex activated during visual mental imagery?
Psychological Bulletin
,
129
,
723
746
.
Kourtzi
,
Z.
, &
Kanwisher
,
N.
(
2000
).
Activation in human MT/MST by static images with implied motion.
Journal of Cognitive Neuroscience
,
12
,
48
55
.
Krekelberg
,
B.
,
Dannenberg
,
S.
,
Hoffmann
,
K.-P.
,
Bremmer
,
F.
, &
Ross
,
J.
(
2003
).
Neural correlates of implied motion.
Nature
,
424
,
674
677
.
Lakoff
,
G.
, &
Johnson
,
M.
(
1980
).
Metaphors we live by.
Chicago
:
University of Chicago Press
.
Lee
,
S. S.
, &
Dapretto
,
M.
(
2006
).
Metaphorical vs. literal word meanings: fMRI evidence against a selective role of the right hemisphere.
Neuroimage
,
29
,
536
544
.
Lewis
,
J. W.
,
Beauchamp
,
M. S.
, &
DeYoe
,
E. A.
(
2000
).
A comparison of visual and auditory motion processing in human cerebral cortex.
Cerebral Cortex
,
10
,
873
888
.
MacSweeney
,
M.
,
Woll
,
B.
,
Campbell
,
R.
,
McGuire
,
P. K.
,
David
,
A. S.
,
Williams
,
S. C.
,
et al
(
2002
).
Neural systems underlying British Sign Language and audio-visual English processing in native users.
Brain
,
125
,
1583
1593
.
Mahon
,
B. Z.
, &
Caramazza
,
A.
(
2008
).
A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content.
Journal of Physiology (Paris)
,
102
,
59
70
.
Malikovic
,
A.
,
Amunts
,
K.
,
Schleicher
,
A.
,
Mohlberg
,
H.
,
Eickhoff
,
S. B.
,
Wilms
,
M.
,
et al
(
2007
).
Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT+: A probabilistic, stereotaxic map of area hOc5.
Cerebral Cortex
,
17
,
562
574
.
Martin
,
A.
(
2007
).
The representation of object concepts in the brain.
Annual Review of Psychology
,
58
,
25
45
.
Martin
,
A.
, &
Caramazza
,
A.
(
2003
).
Neuropsychological and neuroimaging perspectives on conceptual knowledge: An introduction.
Cognitive Neuropsychology
,
20
,
195
212
.
Martin
,
A.
,
Haxby
,
J. V.
,
Lalonde
,
F. M.
,
Wiggs
,
C. L.
, &
Ungerleider
,
L. G.
(
1995
).
Discrete cortical regions associated with knowledge of color and knowledge of action.
Science
,
270
,
102
.
Mashal
,
N.
,
Faust
,
M.
, &
Hendler
,
T.
(
2005
).
The role of the right hemisphere in processing nonsalient metaphorical meanings: Application of principal components analysis to fMRI data.
Neuropsychologia
,
43
,
2084
2100
.
Matlock
,
T.
(
2004
).
Fictive motion as cognitive simulation.
Memory & Cognition
,
32
,
1389
1400
.
McCullough
,
S.
,
Saygin
,
A. P.
,
Alac
,
M.
, &
Emmorey
,
K.
(
2008
).
Motion semantics in American Sign Language activate motion-sensitive visual areas in the human brain.
Paper presented at the Society for Neuroscience, Washington, DC.
Meteyard
,
L.
,
Bahrami
,
B.
, &
Vigliocco
,
G.
(
2007
).
Motion detection and motion verbs: Language affects low-level visual perception.
Psychological Science
,
18
,
1007
1013
.
Meteyard
,
L.
,
Zokaei
,
N.
,
Bahrami
,
B.
, &
Vigliocco
,
G.
(
2008
).
Visual motion interferes with lexical decision on motion words.
Current Biology
,
18
,
R732
R733
.
Munzert
,
J.
,
Lorey
,
B.
, &
Zentgraf
,
K.
(
2009
).
Cognitive motor processes: The role of motor imagery in the study of motor representations.
Brain Research Reviews
,
60
,
306
326
.
Nelissen
,
K.
,
Vanduffel
,
W.
, &
Orban
,
G. A.
(
2006
).
Charting the lower superior temporal region, a new motion-sensitive region in monkey superior temporal sulcus.
Journal of Neuroscience
,
26
,
5929
5947
.
Noppeney
,
U.
,
Josephs
,
O.
,
Kiebel
,
S.
,
Friston
,
K. J.
, &
Price
,
C. J.
(
2005
).
Action selectivity in parietal and temporal cortex.
Brain Research, Cognitive Brain Research
,
25
,
641
649
.
Pavlova
,
M.
,
Bidet-Ildei
,
C.
,
Sokolov
,
A. N.
,
Braun
,
C.
, &
Krageloh-Mann
,
I.
(
2009
).
Neuromagnetic response to body motion and brain connectivity.
Journal of Cognitive Neuroscience
,
21
,
837
846
.
Pobric
,
G.
, &
Hamilton
,
A. F.
(
2006
).
Action understanding requires the left inferior frontal cortex.
Current Biology
,
16
,
524
529
.
Pulvermüller
,
F.
(
2005
).
Brain mechanisms linking language and action.
Nature Reviews Neuroscience
,
6
,
576
582
.
Pulvermuller
,
F.
,
Hauk
,
O.
,
Nikulin
,
V. V.
, &
Ilmoniemi
,
R. J.
(
2005
).
Functional links between motor and language systems.
European Journal of Neuroscience
,
21
,
793
797
.
Pulvermuller
,
F.
,
Huss
,
M.
,
Kherif
,
F.
,
Moscoso del Prado Martin
,
F.
,
Hauk
,
O.
, &
Shtyrov
,
Y.
(
2006
).
Motor cortex maps articulatory features of speech sounds.
Proceedings of the National Academy of Sciences, U.S.A.
,
103
,
7865
7870
.
Raposo
,
A.
,
Moss
,
H. E.
,
Stamatakis
,
E. A.
, &
Tyler
,
L. K.
(
2009
).
Modulation of motor and premotor cortices by actions, action words and action sentences.
Neuropsychologia
,
47
,
388
396
.
Reber
,
P. J.
,
Wong
,
E. C.
,
Buxton
,
R. B.
, &
Frank
,
L. R.
(
1998
).
Correction of off resonance-related distortion in echo-planar imaging using EPI-based field maps.
Magnetic Resonance in Medicine
,
39
,
328
330
.
Revill
,
K. P.
,
Aslin
,
R. N.
,
Tanenhaus
,
M. K.
, &
Bavelier
,
D.
(
2008
).
Neural correlates of partial lexical activation.
Proceedings of the National Academy of Sciences, U.S.A.
,
105
,
13111
13115
.
Richardson
,
D.
, &
Matlock
,
T.
(
2007
).
The integration of figurative language and static depictions: An eye movement study of fictive motion.
Cognition
,
102
,
129
138
.
Richardson
,
D.
,
Spivey
,
M. J.
,
Barsalou
,
L. W.
, &
McRae
,
K.
(
2003
).
Spatial representations activated during real-time comprehension of verbs.
Cognitive Science
,
27
,
767
780
.
Rizzolatti
,
G.
, &
Craighero
,
L.
(
2004
).
The mirror-neuron system.
Annual Review of Neuroscience
,
27
,
169
192
.
Rüschemeyer
,
S.-A.
,
Brass
,
M.
, &
Friederici
,
A. D.
(
2007
).
Comprehending prehending: Neural correlates of processing verbs with motor stems.
Journal of Cognitive Neuroscience
,
19
,
855
865
.
Saygin
,
A. P.
(
2007
).
Superior temporal and premotor brain areas necessary for biological motion perception.
Brain
,
130
,
2452
2461
.
Saygin
,
A. P.
,
Wilson
,
S. M.
,
Dronkers
,
N. F.
, &
Bates
,
E.
(
2004
).
Action comprehension in aphasia: Linguistic and nonlinguistic deficits and their lesion correlates.
Neuropsychologia
,
42
,
1788
1804
.
Saygin
,
A. P.
,
Wilson
,
S. M.
,
Hagler
,
D. J.
, Jr.,
Bates
,
E.
, &
Sereno
,
M. I.
(
2004
).
Point-light biological motion perception activates human premotor cortex.
Journal of Neuroscience
,
24
,
6181
6188
.
Schmidt
,
G. L.
,
DeBuse
,
C. J.
, &
Seger
,
C. A.
(
2007
).
Right hemisphere metaphor processing? Characterizing the lateralization of semantic processes.
Brain and Language
,
100
,
127
141
.
Senior
,
C.
,
Barnes
,
J.
,
Giampietro
,
V.
,
Simmons
,
A.
,
Bullmore
,
E. T.
,
Brammer
,
M.
,
et al
(
2000
).
The functional neuroanatomy of implicit-motion perception or representational momentum.
Current Biology
,
10
,
16
22
.
Sereno
,
M. I.
, &
Tootell
,
R. B.
(
2005
).
From monkeys to humans: What do we now know about brain homologies?
Current Opinion in Neurobiology
,
15
,
135
144
.
Simmons
,
W. K.
,
Martin
,
A.
, &
Barsalou
,
L. W.
(
2005
).
Pictures of appetizing foods activate gustatory cortices for taste and reward.
Cerebral Cortex
,
15
,
1602
1608
.
Simmons
,
W. K.
,
Ramjee
,
V.
,
Beauchamp
,
M. S.
,
McRae
,
K.
,
Martin
,
A.
, &
Barsalou
,
L. W.
(
2007
).
A common neural substrate for perceiving and knowing about color.
Neuropsychologia
,
45
,
2802
2810
.
Slotnick
,
S. D.
,
Thompson
,
W. L.
, &
Kosslyn
,
S. M.
(
2005
).
Visual mental imagery induces retinotopically organized activation of early visual areas.
Cerebral Cortex
,
15
,
1570
1583
.
Talmy
,
L.
(
1996
).
Fictive motion in language and “ception”.
In P. Bloom, M. A. Peterson, L. Nadel, & M. F. Garrett (Eds.),
Language and space
(pp.
211
276
).
Cambridge, MA
:
MIT Press
.
Talmy
,
L.
(
2000
).
Toward a cognitive semantics, volume I: Conceptual structuring systems.
Cambridge
:
MIT Press
.
Tettamanti
,
M.
,
Buccino
,
G.
,
Saccuman
,
M. C.
,
Gallese
,
V.
,
Danna
,
M.
,
Scifo
,
P.
,
et al
(
2005
).
Listening to action-related sentences activates fronto-parietal motor circuits.
Journal of Cognitive Neuroscience
,
17
,
273
281
.
Tootell
,
R. B.
,
Reppas
,
J. B.
,
Kwong
,
K. K.
,
Malach
,
R.
,
Born
,
R. T.
,
Brady
,
T. J.
,
et al
(
1995
).
Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging.
Journal of Neuroscience
,
15
,
3215
3230
.
Tranel
,
D.
,
Kemmerer
,
D.
,
Damasio
,
H.
,
Adolphs
,
R.
, &
Damasio
,
A. R.
(
2003
).
Neural correlates of conceptual knowledge for actions.
Cognitive Neuropsychology
,
20
,
409
432
.
Tranel
,
D.
,
Manzel
,
K.
,
Asp
,
E.
, &
Kemmerer
,
D.
(
2008
).
Naming static and dynamic actions: Neuropsychological evidence.
Journal of Physiology (Paris)
,
102
,
80
94
.
Tranel
,
D.
,
Martin
,
C.
,
Damasio
,
H.
,
Grabowski
,
T. J.
, &
Hichwa
,
R.
(
2005
).
Effects of noun-verb homonomy on the neural correlates of naming concrete entities and actions.
Brain and Language
,
92
,
288
299
.
Tyler
,
L. K.
,
Randall
,
B.
, &
Stamatakis
,
E. A.
(
2008
).
Cortical differentiation for nouns and verbs depends on grammatical markers.
Journal of Cognitive Neuroscience
,
20
,
1381
1389
.
Tyler
,
L. K.
, &
Moss
,
H. E.
(
2001
).
Towards a distributed account of conceptual knowledge.
Trends in Cognitive Sciences
,
5
,
244
252
.
Ungerleider
,
L. G.
, &
Desimone
,
R.
(
1986
).
Cortical connections of visual area MT in the macaque.
Journal of Comparative Neurology
,
248
,
190
222
.
Wallentin
,
M.
,
Lund
,
T. E.
,
Ostergaard
,
S.
,
Ostergaard
,
L.
, &
Roepstorff
,
A.
(
2005
).
Motion verb sentences activate left posterior middle temporal cortex despite static context.
NeuroReport
,
16
,
649
652
.
Warrington
,
E. K.
, &
McCarthy
,
R. A.
(
1994
).
Multiple meaning systems in the brain: A case for visual semantics.
Neuropsychologia
,
32
,
1465
1473
.
Watson
,
J. D.
,
Myers
,
R.
,
Frackowiak
,
R. S.
,
Hajnal
,
J. V.
,
Woods
,
R. P.
,
Mazziotta
,
J. C.
,
et al
(
1993
).
Area V5 of the human brain: Evidence from a combined study using positron emission tomography and magnetic resonance imaging.
Cerebral Cortex
,
3
,
79
94
.
Winawer
,
J.
,
Huk
,
A. C.
, &
Boroditsky
,
L.
(
2008
).
A motion aftereffect from still photographs depicting motion.
Psychological Science
,
19
,
276
283
.
Zeki
,
S.
,
Watson
,
J. D.
,
Lueck
,
C. J.
,
Friston
,
K. J.
,
Kennard
,
C.
, &
Frackowiak
,
R. S.
(
1991
).
A direct demonstration of functional specialization in human visual cortex.
Journal of Neuroscience
,
11
,
641
649
.
Zwaan
,
R. A.
,
Stanfield
,
R. A.
, &
Yaxley
,
R. H.
(
2002
).
Language comprehenders mentally represent the shapes of objects.
Psychological Science
,
13
,
168
171
.
Zwaan
,
R. A.
, &
Taylor
,
L. J.
(
2006
).
Seeing, acting, understanding: Motor resonance in language comprehension.
Journal of Experimental Psychology: General
,
135
,
1
11
.