Abstract

A single word (the noun “elephant”) encapsulates a complex multidimensional meaning, including both perceptual (“big”, “gray”, “trumpeting”) and conceptual (“mammal”, “can be found in India”) features. Opposing theories make different predictions as to whether different features (also conceivable as dimensions of the semantic space) are stored in similar neural regions and recovered with similar temporal dynamics during word reading. In this magnetoencephalography study, we tracked the brain activity of healthy human participants while reading single words varying orthogonally across three semantic dimensions: two perceptual ones (i.e., the average implied real-world size and the average strength of association with a prototypical sound) and a conceptual one (i.e., the semantic category). The results indicate that perceptual and conceptual representations are supported by partially segregated neural networks: Whereas visual and auditory dimensions are encoded in the phase coherence of low-frequency oscillations of occipital and superior temporal regions, respectively, semantic features are encoded in the power of low-frequency oscillations of anterior temporal and inferior parietal areas. However, despite the differences, these representations appear to emerge at the same latency: around 200 msec after stimulus onset. Taken together, these findings suggest that perceptual and conceptual dimensions of the semantic space are recovered automatically, rapidly, and in parallel during word reading.

INTRODUCTION

Recovering the meaning of single words involves accessing a rich set of representations, for instance, the word “elephant” evokes “a big gray mammal emitting a trumpeting sound, etc….” Decades of neuroimaging findings and neuropsychological observations have attempted to shed light onto where, when, and how this information is stored and recovered in the brain (Pulvermüller, 2018; Lambon Ralph, Jefferies, Patterson, & Rogers, 2017). We recently adopted a geometrical perspective conceiving word meaning representations as points in a multidimensional space where distinct dimensions represent different features, such as color, shape, and taxonomic category. We proposed that these dimensions can be of two kinds: motor-perceptual or conceptual (Borghesani & Piazza, 2017). The former are apprehended by physically interacting with the objects or via analogical representations, such as audiovisual recordings (e.g., that elephants are grayish and big), whereas the latter cannot be directly perceived through our senses but, rather, are learned through language (e.g., that elephants are mammals or that they can be found in India). Based on previous findings, we proposed that the motor-perceptual properties are stored in the same brain circuits that encode them when we physically interact with the objects whereas the declarative conceptual properties are stored in higher-order areas of the semantic network (e.g., Borghesani et al., 2016).

Our proposal does not assign any primacy of one kind of representation over the other, and thus, it does not predict a temporal hierarchy of information retrieval across the spatially distributed system, especially when task instructions and/or context do not prioritize one kind of representation over the others. This view is in line with the hub-and-spoke model, the most influential neurocomputational account of the organization of semantic knowledge (Chen, Lambon Ralph, & Rogers, 2017). The hub-and-spoke model links semantic access to the orchestrated activity of a distributed network of brain regions encompassing modality-specific cortices as well as one transmodal center (the anterior temporal lobe [ATL]). This model does not predict any specific temporal hierarchy across systems supporting perceptual versus conceptual representations. On the contrary, it associates the retrieval of semantic representations to an early bidirectional recruitment of both the central hub and its spokes. However, alternative views of the neural dynamics underlying semantic representations have been suggested. Some researchers believe that we, first of all, access conceptual declarative knowledge and that this constitutes the bulk of word meaning. According to this view, the recovery of perceptual features is only epiphenomenal (and thus successive in time) to semantic access (e.g., Mahon & Caramazza, 2008). Others claim that word meaning is mainly resolved by retrieving perceptual features and that the recovery of declarative information occurs later in time, reflecting the integration of all other features (Pulvermüller, 2013).

To date, separate and complementary findings from a variety of time-resolved techniques provide partial and indirect support to the prediction made by these models. Chronometric TMSs and magnetoencephalography (MEG) findings indicate that ATL plays a crucial role in semantic processing 200–600 msec poststimulus onset (Teige et al., 2018; Jackson, Lambon Ralph, & Pobric, 2015; Lau, Gramfort, Hämäläinen, & Kuperberg, 2013). Although studies on the temporal features of word meaning access have traditionally focused on late-appearing ERPs, notably the N400 (Lau, Phillips, & Poeppel, 2008; Kutas & Federmeier, 2000), there is now evidence that semantic information is recovered much earlier in time. ERP effects distinguishing gross taxonomic categories have been reported around 250–270 msec (Hinojosa et al., 2001; Martín-Loeches, Hinojosa, Gómez-Jarabo, & Rubia, 2001; Dehaene, 1995). Coherently, direct recordings of ATL activity via electrocorticogram allow read-out of semantic representations already 250 msec after stimulus onset (Chen et al., 2016). A separate body of research has highlighted sensory-motor effects (i.e., somatotopically organization of cortical responses induced by words referring to different body parts) as early as 220–240 msec for written words (Hauk & Pulvermüller, 2004) and 170 msec for spoken words (Pulvermüller, Shtyrov, & Ilmoniemi, 2005). Finally, a recent MEG evidence suggests that both ATL and modality-specific spokes are recruited within 200 msec poststimulus onset to allow semantic processing of simultaneously presented pictures and words (Mollo, Cornelissen, Millman, Ellis, & Jefferies, 2017). Importantly, brain oscillations ranging from theta to gamma band appear to play a relevant role in accessing word semantic content (for a review, see Bastiaansen & Hagoort, 2006). In particular, early theta power increase (100–400 msec) has been linked to the dissociation between words explicitly referring to visual versus auditory features (Bastiaansen, Oostenveld, Jensen, & Hagoort, 2008). Similarly, semantic priming has been associated with early (250–350 msec) theta power and intertrial phase coherence (ITC) increase (Salisbury & Taylor, 2012) and with oscillatory interactions centered at 8 Hz and starting around 200 msec between the superior temporal cortex and the middle and anterior temporal cortex (Kujala, Vartiainen, Laaksonen, & Salmelin, 2012). Lastly, intracranial recordings with microelectrode arrays in the anteroventral temporal lobe have shown that the first wave of activation elicited by reading a word is an excitatory feedforward wave along the ventrotemporal visual cortex coupled with an almost simultaneous inhibitory cortical feedback (Halgren et al., 2006), a process that corresponds to a widespread reset of the phase of ongoing theta oscillations (Halgren et al., 2015). Crucially, semantic category effects (animals vs. objects) already emerge at the end of this “first pass” cycle, starting around 150 msec, as revealed by measures of both synaptic activity (local field potentials, gamma-band power, and current source density) and neuronal firing (multiunit activity and single-unit recordings; Chan, Baker, et al., 2011).

However, conclusive evidence in favor of an early simultaneous recruitment of both hub and spokes is missing: To date, no study has directly compared (within participant and using the same symbolic stimuli) the spatiotemporal dynamics of conceptual and perceptual dimensions of the semantic space. Do we access conceptual representations earlier, later, or concurrently with perceptual ones? For example, do we recall that “elephant” refers to an animal before, after, or at the same time as recalling that it is large and gray and makes a prototypical trumpeting sound? In this study, we aimed to address this question by designing an MEG experiment where we presented adult participants with written words varying orthogonally along three semantic dimensions: visuoperceptual (the average real-world size of the item referred to by the word), audioperceptual (the strength of its association with a prototypical sound), and conceptual (the taxonomic category; see Figure 1A). If semantic representations were primarily dependent on the reactivation of motor-perceptual features, then perceptual effects, emerging from the activity of sensory-motor cortices, should appear earlier than conceptual ones. Alternatively, if semantic information is primarily coded by an abstract linguistic code in transmodal hubs, conceptual information should be decoded first, localized in multimodal convergence regions. Finally, a third option is that all nodes supporting perceptual and conceptual information are activated simultaneously through an integrated involvement of associative language and sensory-motor areas.

Figure 1. 

Stimuli space investigated and experimental setting. (A) Schematic representation of the 3-D space governing the stimuli used in the experiment, along with seven exemplar words. Thirty-two words were selected to orthogonally vary along three dimensions: one conceptual (i.e., taxonomic category) and two perceptual, involving visual (i.e., implied real-world size) and auditory (i.e., implied real-world prototypical sound) stimuli. (B) Example of a sequence of stimuli: During the experiment, participants were instructed to silently read the target stimuli and to press a button, as fast and accurately as possible, at the presentation of rare odd stimuli. The odd stimuli consist of two diagnostic words referring to one of the 32 targets.

Figure 1. 

Stimuli space investigated and experimental setting. (A) Schematic representation of the 3-D space governing the stimuli used in the experiment, along with seven exemplar words. Thirty-two words were selected to orthogonally vary along three dimensions: one conceptual (i.e., taxonomic category) and two perceptual, involving visual (i.e., implied real-world size) and auditory (i.e., implied real-world prototypical sound) stimuli. (B) Example of a sequence of stimuli: During the experiment, participants were instructed to silently read the target stimuli and to press a button, as fast and accurately as possible, at the presentation of rare odd stimuli. The odd stimuli consist of two diagnostic words referring to one of the 32 targets.

Guided by the previously described studies showing early semantic effects in different aspects of the signal, we expected that brain activations relative to different semantic dimensions might emerge as both phase-locked and non-phase-locked changes in the MEG signals, primarily in the theta and alpha frequency range. To capture these effects, we analyzed stimulus-related changes in terms of both the amplitude of brain oscillations (by computing event-related spectral power) and the distribution of the phase of these oscillations across trials (by computing the ITC). These two measures form a homogeneous “time–frequency state space” that includes both the “evoked” and “induced” components of event-related EEG changes (Makeig, Debener, Onton, & Delorme, 2004) and within which any potential event-related field (ERF) effect emerges as a variable combination of event-related phase-locking (captured by ITC) and increase of oscillatory power (captured by spectral power changes). The advantage of ITC over ERF is that the former decomposes the latter into its constituent phase-locked frequency bands (Makeig et al., 2004), providing in some cases an increased sensitivity (e.g., Salisbury & Taylor, 2012; Mormann et al., 2005). Even though we expected the main effects of semantic access in theta and alpha range, given previous reports of higher frequencies involvement in semantic processing (e.g., Bastiaansen & Hagoort, 2006), we extended our analysis to beta and (for spectral power only) gamma band.

METHODS

Participants

Fifteen healthy adults (seven men, mean age = 24.57 ± 2.69 years) participated in the MEG study. Data from two additional participants were discarded because of magnetic artifacts (the subsequent MRI scan suggested the presence of dental implants). All participants were right-handed as measured with the Edinburgh Handedness Questionnaire, had normal or corrected-to-normal vision, and were French native speakers. All experimental procedures were approved by the local ethics committee, and each participant provided signed informed consent to take part in the study. Participants received a monetary compensation for their participation.

Stimuli

We aimed at recording robust and reliable neural responses to highly controlled stimuli. To this end, we selected 32 concrete written names varying orthogonally along three semantic dimensions (see Table 1): the implied real-world size (separating “small” vs. “big” items), the strength of the association with a prototypical sound (separating “noisy” vs. “silent” items), and the taxonomic category (separating “living” vs. “nonliving” items). Words were well matched across several psycholinguistic variables (i.e., number of letters, number of syllables, number of phonemes, number of morphemes, number of orthographic neighbors, number of orthographic uniqueness points, gender, and frequency of use in books and in movies retrieved from Lexique, lexique.org). These psycholinguistic variables did not differ across the two semantic categories (Mann–Whitney rank test for number of letters: u = 109.5, p = .25; number of syllables: u = 105, p = .17; number of phonemes: u = 98.5, p = .13; number of orthographic neighbors: u = 126, p = .47; number of orthographic uniqueness points: u = 106, p = .20; frequency of use in books: u = 120, p = .39; frequency of use in movies: u = 126, p = .48; chi-square test of gender: χ2 = 0.14, p = .70; chi-square test of number of morphemes: χ2 = 0.14, p = .70). Similarly, they did not differ across the visual-perceptual semantic property (Mann–Whitney rank test for number of letters: u = 103, p = .17; number of syllables: u = 121, p = .39; number of phonemes: u = 91, p = .08; number of orthographic neighbors: u = 109, p = .23; number of orthographic uniqueness points: u = 119, p = .37; frequency of use in books: u = 111.5, p = .27; frequency of use in movies: u = 103, p = .18; chi-square test of gender: χ2 = 0.14, p = .71; chi-square test of number of morphemes: χ2 = 0.14, p = .70) nor across the audio-perceptual semantic property (Mann–Whitney rank test for number of letters: u = 89.5, p = .07; number of syllables: u = 103.5, p = .15; number of phonemes: u = 91, p = .08; number of orthographic neighbors: u = 92.5, p = .08; number of orthographic uniqueness points: u = 125, p = .46; frequency of use in books: u = 104.5, p = .19; frequency of use in movies: u = 126, p = .48; chi-square test of gender: χ2 = 1.29, p = .26; chi-square test of number of morphemes: χ2 = 1.28, p = .25). These analyses were run with statistical functions provided by Python's library, SciPy (https://docs.scipy.org/doc/scipy/reference/stats.html). To verify that our classification (e.g., small animal associated with a characteristic sound) corresponded with that of our participants, we administered two behavioral questionnaires right after the MEG recordings; this was done after the scan to exclude any interference with their spontaneous word processing during the MEG experiment. In the Size Questionnaire, participants were asked to rate, on a scale from 1 to 9, the size of the object/animal each word referred to, as compared with a shoe box (i.e., implicitly responding to the question: “could this item fit in a shoe box?”). In the Sound Questionnaire, participants were asked to rate, on a scale from 1 to 9, whether the item was associated with a prototypical sound or not. The order of tasks and the words within each task were randomized across participants. The results indicated that participants' judgments were consistent with our classification: for size, the average score for items categorized as “big” in our design was 7.84 (±0.80), whereas the average for items categorized as “small” was 3.28 (±1.27). None of the items categorized as “big” had a score lower than 6, and none of the items categorized as “small” were higher than 5. Similarly, across participants, the average score for items categorized as having a prototypical sound was 7.67 (±0.77, none of them having an average score lower than 6), whereas the average for items categorized as silent was 2.43 (±1.2, none of them having an average score higher than 5).

Table 1. 
Experimental Stimuli
Words (EN)Words (FR)Length (No. of Letters)Implied Real-world SizeImplied Real-world SoundCategoryBehavioral Ratings: SizeBehavioral Ratings: Audio
MeanSDModeMeanSDMode
gorilla gorille big typical sound living 8.40 0.95 9.00 6.93 2.35 9.00 
elephant éléphant big typical sound living 9.00 0.00 9.00 7.20 2.71 9.00 
giraffe girafe big silent living 8.80 0.40 9.00 2.53 2.19 1.00 
lama lama big silent living 7.73 1.73 9.00 3.40 2.58 1.00 
marmoset ouistiti small typical sound living 5.00 1.51 5.00 6.20 2.56 9.00 
parrot perroquet small typical sound living 4.33 1.40 5.00 7.13 2.00 9.00 
scorpion scorpion small silent living 1.80 0.54 2.00 1.27 0.57 1.00 
chameleon caméléon small silent living 3.53 1.31 3.00 1.40 0.61 1.00 
cow vache big typical sound living 8.27 0.77 9.00 8.67 0.60 9.00 
ship mouton big typical sound living 7.07 1.98 9.00 8.53 0.62 9.00 
bull taureau big silent living 8.47 0.72 9.00 5.53 1.93 7.00 
chamois chamois big silent living 7.53 1.15 7.00 3.00 1.83 1.00 
cricket cricket small typical sound living 1.07 0.25 1.00 7.53 2.22 9.00 
cock coq small typical sound living 4.27 1.39 5.00 8.73 0.77 9.00 
ant fourmi small silent living 1.00 0.00 1.00 1.27 0.57 1.00 
rabbit lapin small silent living 3.33 1.58 3.00 1.80 1.11 1.00 
vacuum cleaner aspirateur 10 big typical sound nonliving 5.93 1.88 7.00 7.87 1.31 9.00 
washing machine lave-linge 10 big typical sound nonliving 6.93 2.11 9.00 7.00 1.67 9.00 
wardrobe armoire big silent nonliving 8.27 0.77 9.00 1.60 1.08 1.00 
sofa sofa big silent nonliving 7.73 1.69 9.00 1.27 0.57 1.00 
blender mixeur small typical sound nonliving 4.47 1.63 5.00 6.93 1.98 9.00 
alarm clock réveil small typical sound nonliving 2.73 1.06 3.00 8.53 1.50 9.00 
pillow oreiller small silent nonliving 4.80 1.51 6.00 1.13 0.34 1.00 
fork fourchette 10 small silent nonliving 2.13 1.96 1.00 2.07 2.05 1.00 
helicopter hélicoptère 11 big typical sound nonliving 8.93 0.25 9.00 8.00 1.15 9.00 
motorbike moto big typical sound nonliving 7.93 1.06 9.00 8.20 1.33 9.00 
bike vélo big silent nonliving 7.20 1.51 7.00 4.07 2.11 5.00 
canoe canoë big silent nonliving 7.27 2.21 9.00 1.67 1.19 1.00 
car stereo autoradio small typical sound nonliving 2.67 0.87 2.00 6.80 2.54 9.00 
rotating beacon gyrophare small typical sound nonliving 2.67 1.14 2.00 8.47 2.00 9.00 
roller roller small silent nonliving 4.47 1.78 5.00 3.67 2.41 2.00 
boots bottes small silent nonliving 4.27 1.39 5.00 3.33 2.12 3.00 
Words (EN)Words (FR)Length (No. of Letters)Implied Real-world SizeImplied Real-world SoundCategoryBehavioral Ratings: SizeBehavioral Ratings: Audio
MeanSDModeMeanSDMode
gorilla gorille big typical sound living 8.40 0.95 9.00 6.93 2.35 9.00 
elephant éléphant big typical sound living 9.00 0.00 9.00 7.20 2.71 9.00 
giraffe girafe big silent living 8.80 0.40 9.00 2.53 2.19 1.00 
lama lama big silent living 7.73 1.73 9.00 3.40 2.58 1.00 
marmoset ouistiti small typical sound living 5.00 1.51 5.00 6.20 2.56 9.00 
parrot perroquet small typical sound living 4.33 1.40 5.00 7.13 2.00 9.00 
scorpion scorpion small silent living 1.80 0.54 2.00 1.27 0.57 1.00 
chameleon caméléon small silent living 3.53 1.31 3.00 1.40 0.61 1.00 
cow vache big typical sound living 8.27 0.77 9.00 8.67 0.60 9.00 
ship mouton big typical sound living 7.07 1.98 9.00 8.53 0.62 9.00 
bull taureau big silent living 8.47 0.72 9.00 5.53 1.93 7.00 
chamois chamois big silent living 7.53 1.15 7.00 3.00 1.83 1.00 
cricket cricket small typical sound living 1.07 0.25 1.00 7.53 2.22 9.00 
cock coq small typical sound living 4.27 1.39 5.00 8.73 0.77 9.00 
ant fourmi small silent living 1.00 0.00 1.00 1.27 0.57 1.00 
rabbit lapin small silent living 3.33 1.58 3.00 1.80 1.11 1.00 
vacuum cleaner aspirateur 10 big typical sound nonliving 5.93 1.88 7.00 7.87 1.31 9.00 
washing machine lave-linge 10 big typical sound nonliving 6.93 2.11 9.00 7.00 1.67 9.00 
wardrobe armoire big silent nonliving 8.27 0.77 9.00 1.60 1.08 1.00 
sofa sofa big silent nonliving 7.73 1.69 9.00 1.27 0.57 1.00 
blender mixeur small typical sound nonliving 4.47 1.63 5.00 6.93 1.98 9.00 
alarm clock réveil small typical sound nonliving 2.73 1.06 3.00 8.53 1.50 9.00 
pillow oreiller small silent nonliving 4.80 1.51 6.00 1.13 0.34 1.00 
fork fourchette 10 small silent nonliving 2.13 1.96 1.00 2.07 2.05 1.00 
helicopter hélicoptère 11 big typical sound nonliving 8.93 0.25 9.00 8.00 1.15 9.00 
motorbike moto big typical sound nonliving 7.93 1.06 9.00 8.20 1.33 9.00 
bike vélo big silent nonliving 7.20 1.51 7.00 4.07 2.11 5.00 
canoe canoë big silent nonliving 7.27 2.21 9.00 1.67 1.19 1.00 
car stereo autoradio small typical sound nonliving 2.67 0.87 2.00 6.80 2.54 9.00 
rotating beacon gyrophare small typical sound nonliving 2.67 1.14 2.00 8.47 2.00 9.00 
roller roller small silent nonliving 4.47 1.78 5.00 3.67 2.41 2.00 
boots bottes small silent nonliving 4.27 1.39 5.00 3.33 2.12 3.00 

The original 32 words (in French) were preselected as to span two semantic categories (living and nonliving items) and to orthogonally vary along, two perceptual semantic dimensions: (implied real-world size and prototypical sound).

Testing Procedures

Participants were seated in a comfortable armchair in front of a screen. They were instructed to silently read the target stimuli (i.e., the 32 words) and to make semantic decisions only on rare odd stimuli. These odd stimuli appeared on 6% of the trials and consisted of a pair of words that were semantically related to one of the targets (e.g., “trunk, ivory” for elephant). Participants pressed the left- or right-hand button to indicate whether or not the odd stimulus was related to the previously seen target word (i.e., 1-back semantic relatedness task). The response mapping was counterbalanced within participants: Half of the participants answered yes with the left hand in the first half of the imaging runs and then yes with the right hand in the last half; the other half of the participants followed the reverse order. Importantly, the pairs of words used as odd stimuli did not contain any verb nor any reference to the category, size, or sound related to the dimensions that we investigated. Target stimuli were presented at the center of the screen (refresh rate: 60 Hz), printed in Courier New (in white), for 300 msec. They were followed by an interstimulus interval that varied randomly between 2167 and 3340 msec. Odd stimuli were presented for 1670 msec, followed by 1670 msec of blank (in red, see Figure 1B). Within a given MEG session, the participants underwent eight runs of approximately 7 min each. Breaks between runs were tailored to participants' needs. Each run contained five repetitions of each of the 32 target stimuli and 10 odd stimuli, for a total of 170 stimuli per run. Pseudorandomization ensured that, over the entire experiment, for half of the odd stimuli (i.e., 40 times), a positive answer was expected. Before testing the first participant, a photodetector was used to compute the delay between the time at which the triggers were sent to the MEG acquisition computer and the time at which the stimuli actually appeared on the screen. This delay (50 msec) was corrected during data preprocessing. Stimuli were presented with Psychopy (www.psychopy.org).

Data Acquisition

Data were collected at Neurospin (CEA-Inserm) in a dimly illuminated, sound-attenuating, and magnetically shielded room using a whole-head Elekta MEG system with 102 magnetometers and 204 orthogonal planar gradiometers (Elekta Neuromag). Before the recordings, three fiducial points (nasion, left and right preauricular areas) and about 100 supplementary points distributed over the scalp of the participants were digitalized (3-D digitizer, Polhemus Isotrak system) to coregister the MEG data with the anatomical MRI.

During the recordings, participants were seated in the upright position with their head in close contact to the dewar and were instructed to avoid any unnecessary head, body, or limb movements. At the start of each recording block, their head position was measured with four head position coils (HPI) placed over the frontal and mastoid areas and compared online with the position at the beginning of the recording. To minimize head displacements across the whole recordings, if the head moved more than 10 mm from the original position in any direction, the participant was assisted to reposition the head closer to the original position. MEG recordings were sampled at 1 kHz, hardware band-pass filtered between 0.03 Hz and 330 Hz, and active compensation of external noise (Maxshield, Elekta Neuromag) was applied. Heartbeat and horizontal and vertical eye movements were recorded simultaneously with the MEG signals with three additional pairs of electrodes for the electrocardiogram and the electrooculograms, respectively. Immediately before or after each recording session, empty room recordings of about 2 min were acquired.

Structural MRI anatomy data were collected for each participant at Neurospin (CEA-Inserm) with a 3-T Siemens Magnetom TrioTim scanner using a 32-channel head coil. Anatomical images were acquired using a T1-weighted MPRAGE sagittal scan (voxels size 1 × 1 × 1.1 mm, 160 slices, 7 min).

Data Preprocessing

The first preprocessing step utilized the software Maxfilter (Elekta Neuromag) to suppress external magnetic interference by using signal space separation, which interpolated noisy MEG sensors (identified by visual inspection of the raw data) and corrected for head movements between data blocks. Head movement correction was performed with respect to a subject-specific head position, computed as the mean head position across blocks (custom-made software, courtesy of Antoine Ducorps and Denis Schwartz, CENIR, Paris, France) and used as the reference head position for MEG/MRI coregistration. The remaining preprocessing steps were performed with Brainstorm (Tadel, Baillet, Mosher, Pantazis, & Leahy, 2011). Data were visually inspected to mark bad segments or segments that included clear motor artifacts or channel jumps/anomalies and then were low-pass filtered at 160 Hz. Heartbeat and blinking components were automatically detected by using principal component analysis and visually checked and removed by discarding the corresponding signal space projections. The stimulus trigger delay (50 msec) was corrected. Data were then epoched starting 800 msec before and ending 1200 msec after the onset of the stimuli (epochs containing previously marked bad segments were automatically rejected) and downsampled to 500 Hz (no baseline correction was applied).

For each participant, MRI data were segmented with FreeSurfer (surfer.nmr.mgh.harvard.edu), imported in Brainstorm (Tadel et al., 2011), and coregistered with MEG data by using previously digitized head surface points. The forward model was computed by using an overlapping spheres method. Noise covariance was estimated from the MEG empty room recordings. A total of 15,002 individual sources were computed with the weighted minimum-norm method using the default Brainstorm parameters.

Time–Frequency Analysis

Event-related Spectral Power

For the main analysis, spectral power was estimated by computing the time–frequency decomposition with the multitaper approach implemented in Fieldtrip (Oostenveld, Fries, Maris, & Schoffelen, 2011), with parameters adapted to two distinct frequency ranges. For the low-frequency range (4–35 Hz in 1-Hz steps), data segments were extracted from sliding time windows with a length of 500 msec between 4 and 10 Hz (frequency resolution = 2 Hz) and with a length equal to five oscillation cycles per frequency between 10 (frequency resolution = 2 Hz) and 35 Hz (frequency resolution = 7 Hz), shifted in steps of 40 msec. These parameters were chosen to optimize the frequency resolution for higher frequencies, while still keeping a limited time window for lower frequencies, to test stimulus-related effects. Data segments were tapered with a single Hanning window and Fourier-transformed. Spectral power was computed as the square amplitude of the resulting time–frequency decomposition. For the high-frequency range (34–100 Hz in 2-Hz steps), data segments were extracted from sliding time windows of 200 msec length and shifted in steps of 40 msec. A multitaper approach was applied to each window to optimize spectral concentration over the frequency of interest (Mitra & Pesaran, 1999). Frequency smoothing was set to 20% of each frequency value. With these settings, the number of tapers used ranged from 2 at 34 Hz (frequency resolution = 7 Hz) to 7 at 100 Hz (frequency resolution = 20 Hz). Spectral power was first estimated per taper and then averaged across tapers. For all frequencies, mean power from baseline ([−500 −250] msec) was removed. For the high temporal resolution analysis, spectral power was estimated by computing the time–frequency decomposition in the frequency range 5–13 Hz in 1-Hz steps; data segments were extracted from sliding time windows with a length equal to 1.5 oscillation cycles per frequency (time window = 300 msec at 5 Hz, 115 msec at 13 Hz) and shifted in steps of 10 msec. These parameters were chosen to maximize the temporal resolution while still keeping a limited frequency smoothing. Data segments were tapered with a single Hanning window and Fourier-transformed. Spectral power was computed as the square amplitude of the resulting time–frequency decomposition. For all frequencies, mean power from the baseline period ([−500 −250] msec) was removed.

Intertrial Phase Coherence

Intertrial phase coherence (ITC) was estimated for each participant and condition by computing the phase-locking factor (Tallon-Baudry, Bertrand, Delpuech, & Pernier, 1996) with the following steps: (a) The complex time–frequency decomposition at time t and frequency f of each single trial (as computed above, both for the main analysis and for the high temporal resolution analysis) was normalized by its absolute value to obtain amplitude-independent unitary vectors in the complex plane. (b) These normalized vectors were averaged across single trials to obtain a complex value related to the phase distribution of each time–frequency region around t and f. The ITC was computed as the modulus of this value. ITC ranges from 0 (purely non-phase-locked activity) to 1 (strictly phase-locked activity).

Statistical Analysis

All the statistical tests were aimed at identifying significant differences between experimental conditions and were performed with the nonparametric cluster-based statistical analysis (Maris & Oostenveld, 2007), as implemented in Fieldtrip (Oostenveld et al., 2011). This method allows statistical testing on wide time and frequency intervals with no need of a priori selection of spatial ROIs because it effectively controls the Type I error rate in a situation involving multiple comparisons by clustering neighboring channel–time–frequency pairs that exhibit statistically significant effects (test used at each channel–time–frequency point: dependent-samples t statistics) and using a permutation test to evaluate the statistical significance at the cluster level (Montecarlo method, 3000 permutations for each test). Results on statistically significant clusters are reported by specifying the polarity of the cluster (positive or negative), its p value, its temporal and spectral extent, and the time and frequency of its maximum effect (hereafter indicated as the cluster's peak), defined as the time–frequency at which the cluster statistics is at its maximum. The time course of the cluster statistics is obtained by averaging, at each time point, the channel–time–frequency point t statistics over all the channels and frequencies belonging to the cluster at that time point. Analogously, the frequency range of the cluster statistics is obtained by averaging, at each frequency bin, the channel–time–frequency point t statistics over all the channels and time points belonging to the cluster at that frequency bin. All the statistical tests were performed separately for magnetometers and combined gradiometers.

Unless otherwise stated, for all contrasts of interest, cluster-based statistical analyses corrected for multiple comparisons over time, and frequency and sensor space were applied on the whole-time window (from 0 to 600 msec) for three frequency ranges, theta and alpha (4–13 Hz), beta (13–35 Hz), and gamma (35–100 Hz), with the latter for spectral power only.

Latency Analysis

To estimate and statistically compare the onset latencies of the earliest power/ITC effects, we adapted to time–frequency measures the approach proposed in Miller, Patterson, and Ulrich (1998), to compute the onset latency of ERP components by using jackknife resampling (Miller, 1974). The jackknife-based approach provides an efficient way to perform robust statistical comparisons between onset latencies in different conditions by overcoming the unreliability of latency estimation at the single-subject level. It consists of (1) computing n “leave-one-out” grand averages, each one from a subsample of n − 1 of the n participants and (2) estimating the onset latency from each of the n grand averages. For each grand average, the peak value was computed as the maximum amplitude over the first 300 msec after stimulus onset, and onset latency was estimated with the relative criterion technique (Kiesel, Miller, Jolicoeur, & Brisson, 2008) as the time point at which the amplitude reaches 90% of the peak value. The threshold of 90% (higher than the thresholds generally used with ERPs; Kiesel et al., 2008) was chosen to take into account the relatively low temporal resolution of the power/ITC measures, that is, a lower percentage value might artificially provide early onsets possibly caused by the backward temporal spreading of the peak rather than by real effects. Differences between latencies in different conditions were statistically evaluated by using a standard paired t test and ANOVA, where the t values or the F values were adjusted for the jackknife resampling by setting tc = t/(n − 1) and Fc = F/(n − 1)2 before computing significance (Ulrich & Miller, 2001).

Source Reconstruction

To explore the anatomical sources of the statistically significant effects observed at the sensor level, spectral power and ITC were estimated at the source level by performing the following steps: (a) For each participant, source-level time series were reconstructed from single trials on the 15,002 sources obtained from the weighted minimum-norm method reconstruction in Brainstorm; (b) spectral power and ITC were estimated at the source level by using the same time–frequency decomposition and parameters used at the sensor level (main analysis), within the frequency range of the statistically significant effects identified at the sensor level (6–12 Hz); (c) spectral power data were z-scored on the baseline [−500 −250] msec (ITC was not z-scored because it is already a normalized measure); and (d) data were spatially smoothed (10 mm) and projected onto a standard anatomical template (Colin27) to perform averages across participants.

For each measure and contrast of interest, a paired t test was run at each source location, and the corresponding significant clusters (p < .05 uncorrected) are reported on a template cortex smoothed at 50%. Importantly, the t test at the source level is only used to properly describe the source distribution of the statistically significant effect established at the sensor level and not for a second statistical test at the source level; therefore, no correction for multiple comparison is required (Gross et al., 2013).

Specificity of the Main Effects

We performed a post hoc supplementary statistical analysis to assess the specificity of the main effects of each dimension in the associated space–frequency–time cluster. First, we extracted for all trials the ITC and spectral power values from the three clusters of sensors, frequencies, and time points where the earliest significant dimension-specific effects were detected. Then, in each ROI, we ran a repeated-measures ANOVA with three grouping factors (i.e., Category, Size, Sound) with two levels each (e.g., big vs. small). This way, we assessed whether it was possible, within each cluster (e.g., the one where category effect was observed), to detect information relative to the other two dimensions (e.g., size and audio).

RESULTS

Temporal Unfolding of Brain Activation during Word Reading

Overall, averaging across all stimuli, we were able to appreciate how single word reading impacts both the amplitude and the phase of brain oscillations across different frequency ranges. First, we observed a burst in ITC, which started at about 100 msec, peaked at ∼200 msec, and propagated at least until ∼350 msec. This peak in phase-locked activity was especially high in the theta frequency range, where it lasted until ∼500 msec, but affected also higher-frequency ranges, a pattern resembling the one observed in the middle layers at the laminar level in microelectrode recordings (Halgren et al., 2015). The ITC effect started in the occipital cortex but rapidly propagated anteriorly toward the temporal and parietal cortices, bilaterally but slightly more intensely in the left compared with the right temporal cortex (see Figure 2A). This pattern closely overlapped with the source distribution of the early ERFs typically associated with word reading (Marinkovic et al., 2003). Second, looking at the spectral power changes over time, we detected a transient increase in the theta band followed by a more sustained decrease in alpha band and, more marginally, beta band. The initial spectral power increase showed a similar temporal and spatial evolution to that of the ITC effect: starting at about 100 msec and peaking at ∼200 msec, progressing from occipital to temporal. Starting from about 250 msec and propagating at least until ∼600 msec, a massive posterior alpha-band desynchronization emerged (see Figure 2B). This spectral pattern is fully compatible with previous work investigating the oscillatory modulations elicited by word reading (Goto et al., 2011). After replicating the known unfolding of neural activation linked with word processing, we investigated when, where, and by which oscillatory aspect of the signal, different kinds of semantic information modulate this brain dynamics.

Figure 2. 

Oscillatory brain dynamics elicited by word reading. (A) Intertrial phase coherence (ITC) effect in the theta band (top row) and alpha band (bottom row), 200 and 400 msec after stimulus onset. Rightmost: time–frequency representation of ITC averaged across all sensors. Across all frequencies, an increase in phase coherence can be appreciated around 200 msec, with higher impact on lower ones. (B) Panels as in A for both power changes in the theta band (top row) and alpha band (bottom row), 200 and 400 msec after stimulus onset. A transient increase in power at lower bands (theta) around 200 msec is followed by a more sustained decrease of power at higher frequencies (alpha and beta).

Figure 2. 

Oscillatory brain dynamics elicited by word reading. (A) Intertrial phase coherence (ITC) effect in the theta band (top row) and alpha band (bottom row), 200 and 400 msec after stimulus onset. Rightmost: time–frequency representation of ITC averaged across all sensors. Across all frequencies, an increase in phase coherence can be appreciated around 200 msec, with higher impact on lower ones. (B) Panels as in A for both power changes in the theta band (top row) and alpha band (bottom row), 200 and 400 msec after stimulus onset. A transient increase in power at lower bands (theta) around 200 msec is followed by a more sustained decrease of power at higher frequencies (alpha and beta).

Intertrial Phase Coherence Reveals the Early Emergence of the Perceptual Dimensions

We first investigated the ITC, searching for components of the signal that could differentiate across our conditions of interest. We found evidence for an early encoding of both perceptual dimensions. Between 120 and 360 msec (peak at 240 msec), the ITC, over a cluster of left occipital sensors in the theta band (frequency range: 6–7 Hz, peak at 6 Hz, corrected p = .03), differentiated words with respect to the implied real object size: Overall, words referring to small items elicited a higher ITC than words referring to big items (Figure 3A, top and center). Source reconstruction at the peak latency showed that the cortical source of this difference in the implied size was located in a left-lateralized set of regions including the occipital lobe, the medial-temporal lobe, and inferior frontal gyrus (Figure 3A, top right). In the same early time window, we also found a highly significant effect of implied real-world sound (time range = 40–320 msec, peak at 200 msec), but at the higher frequency alpha band (frequency range = 8–12 Hz, peak at 10 Hz, corrected p = .008): Words referring to items associated with a prototypical sound elicited higher ITC as compared with those not associated with a specific sound (Figure 3A, bottom). Source reconstruction suggested that this effect was linked to the activity of left occipital and bilateral superior temporal areas extending especially in the right hemisphere to supramarginal gyrus (Figure 3A, bottom right). The effect of the associated sound also emerged in a later time window in the theta band (time range = 320–520 msec, peak at 400 msec; frequency range = 4–6 Hz, peak at 5 Hz, corrected p = .04). However, the sign of this later effect was reversed compared with the early effect: Words referring to items not typically associated with sounds elicited a higher ITC. All these effects were observed on the combined gradiometers, whereas only trending (but congruently so) effects could be appreciated in the magnetometers. No effect of semantic category was detected in this aspect of the signal. No significant effects were found in the beta band.

Figure 3. 

Rapid recovery of both perceptual and conceptual dimensions of the semantic space. (A) Effect of perceptual dimensions on ITC. Top row: time–frequency representation (left, only significant values are shown), sensor topography (middle), and source reconstruction (right, significant clusters, p < .05 uncorrected, are reported on a template cortex smoothed at 50%) of the statistically significant cluster associated with the implied real-world size effect (peak at 240 msec and 6 Hz, corrected p = .03). Lower row: plots as in the top row of the implied real-world sound effect (peak at 200 msec and 10 Hz, corrected p = .008). (B) Effect of conceptual dimension on spectral power. Panels as in A for both the early (top, peak at 200 msec and 8 Hz, corrected p = .02) and late (bottom, peak at 600 msec and 10 Hz, corrected p = .01) semantic category effects.

Figure 3. 

Rapid recovery of both perceptual and conceptual dimensions of the semantic space. (A) Effect of perceptual dimensions on ITC. Top row: time–frequency representation (left, only significant values are shown), sensor topography (middle), and source reconstruction (right, significant clusters, p < .05 uncorrected, are reported on a template cortex smoothed at 50%) of the statistically significant cluster associated with the implied real-world size effect (peak at 240 msec and 6 Hz, corrected p = .03). Lower row: plots as in the top row of the implied real-world sound effect (peak at 200 msec and 10 Hz, corrected p = .008). (B) Effect of conceptual dimension on spectral power. Panels as in A for both the early (top, peak at 200 msec and 8 Hz, corrected p = .02) and late (bottom, peak at 600 msec and 10 Hz, corrected p = .01) semantic category effects.

Spectral Power Changes Reveal the Early Emergence of the Conceptual Dimension

A strong and long-lasting effect of the conceptual dimension (i.e., semantic category) was detected in a left occipital-temporal cluster of combined gradiometers (corrected p = .01) and lasted between 80 and 600 msec (peak at 600 msec) and 4 and 13 Hz (peak at 9 Hz): words referring to animals elicited a higher power increase than those referring to nonliving items. At closer inspection, two subclusters could be clearly visualized, and thus, the analyses were repeated for two separated time windows (0–300 and 300–600 msec). This allowed the isolation of an early peak (at 200 msec and around 8 Hz, corrected p = .02) and a late peak (at 600 msec and 10 Hz, corrected p = .01; Figure 3B). The same effects were also observed on the magnetometers (peak at 560 msec, between 8 and 13 Hz, corrected p = .03). Source reconstruction (Figure 3B, right) suggested that the early effect primarily originated from a temporoparietal network of brain regions in the left hemisphere, including the left angular gyrus, whereas the second effect could be traced back to the activity of left anterior temporal and inferior frontal areas.

The two perceptual dimensions showed later and weaker, yet still significant, effects. An implied real-world size effect was detected in a left occipital cluster of magnetometers between 160 and 480 msec (peak at 400 msec, peak at 10 Hz, corrected p = .04): Words referring to small items were associated with an increase in gamma band. This effect was not observed in the gradiometers. An implied real-world sound effect was also detected in a bilateral occipital cluster of gradiometers between 200 and 600 msec (peak at 600 msec) and between 5 and 13 Hz (peak at 10 Hz, corrected p = .01). This last effect was similarly observed in the magnetometers (peak at 560 msec and 11 Hz, corrected p = .008). These effects indicated that words referring to items associated with prototypical sounds were associated with a greater decrease in alpha band. No significant effects were found for sound and category dimensions in the beta and gamma band.

Perceptual and Conceptual Dimensions Arise in Parallel

To estimate more accurately the latency of the onset of perceptual and conceptual effects and to determine whether they arise in parallel or with a rapid temporal hierarchy, we performed a second time–frequency analysis with higher temporal resolution. We pushed the temporal resolution to its limit (1.5 cycles per frequency) and restricted the analysis to the frequency interval where the earliest effects emerged (5–13 Hz). The detected effects were very similar in statistical power, time, and sensor cluster to those of the first analysis (size: corrected p = .06, latency 180–250 msec, peak at 230 msec; sound: corrected p = .003, latency 130–250 msec, peak at 220 msec; category: corrected p = .002, latency 20–300 msec, peak at 240 msec). Latency was then estimated for each condition on its associated significant cluster by using a jackknife-based approach (see Methods), which overcomes the unreliability of single-subject latency estimation by building a statistical estimate on a “leave-one-out” resampling procedure (Miller et al., 1998). This latency analysis provided evidence for an early (size: 191 ± 9 msec; sound: 191 ± 10 msec; category: 201 ± 20 msec) and fully parallel onset of the three effects (repeated-measures ANOVA on the three dimensions, corrected for jackknife: F(1.76, 24.66) = 0.18, corrected p = .81; corrected paired t test Size vs. Sound: p = .98, Size vs. Category: p = .68; Sound vs. Category: p = .64; Figure 4).

Figure 4. 

Simultaneous recovery of visual, auditory, and conceptual semantic dimensions. Average latencies of the earliest significant effects of the perceptual (ITC) and conceptual (spectral power) dimensions are not statistically different (corrected F(1.76, 24.66) = 0.18, corrected p = .81; corrected paired t test size vs. sound: p = .98, size vs. category: p = .68; sound vs. category: p = .64). Histograms represent the mean latency ± standard error (standard error corrected for the jackknife-based approach = sqrt(n − 1) * sem; Miller et al., 1998) of the difference of ITC values for small–big and sound–no sound and of the difference of spectral power values of living–nonlinving. ITC/power values are computed on the sensors and frequencies corresponding to the peak of statistical significance. Data from the “high temporal resolution” analysis.

Figure 4. 

Simultaneous recovery of visual, auditory, and conceptual semantic dimensions. Average latencies of the earliest significant effects of the perceptual (ITC) and conceptual (spectral power) dimensions are not statistically different (corrected F(1.76, 24.66) = 0.18, corrected p = .81; corrected paired t test size vs. sound: p = .98, size vs. category: p = .68; sound vs. category: p = .64). Histograms represent the mean latency ± standard error (standard error corrected for the jackknife-based approach = sqrt(n − 1) * sem; Miller et al., 1998) of the difference of ITC values for small–big and sound–no sound and of the difference of spectral power values of living–nonlinving. ITC/power values are computed on the sensors and frequencies corresponding to the peak of statistical significance. Data from the “high temporal resolution” analysis.

Specificity of the Emergence of Perceptual and Conceptual Dimensions

To assess whether the space–time–frequency cluster associated to each semantic dimension was selectively activated by that dimension, we performed a supplementary post hoc statistical analysis with a repeated-measures ANOVA with three grouping factors (i.e., Category, Size, Sound) of two levels each (e.g., big vs. small). Results indicate that all three effects are dimension-specific, as no main effect of the two other dimensions is significant in each of the three clusters: no main effect of Sound or Category (Category: F(1.00, 14.00) = 0.01, p = .92; Sound: F(1.00, 14.00) = 2.122, p = .17) in the Size cluster (200–240 msec, 6–7 Hz); no main effect of Size or Category (Size: F(1.00, 14.00) = 0.64, p = .44; Category: F(1.00, 14.00) = 0.02, p = .90) in the Sound cluster (200–240 msec, 10–11 Hz); no main effect of Size or Sound (Size: F(1.00, 14.00) = 2.42, p = .14; Sound: F(1.00, 14.00) = 0.58, p = .46) in the Category cluster (200–240 msec, 8–9 Hz). Although this analysis cannot assess to what extent (and in which areas) the spatiotemporal patterns associated to each dimension are fully separable, the result clearly indicates that the encoding of the three dimensions is partially specific and cannot be reduced to a common, generic, early semantic activation.

DISCUSSION

In this study, we investigated the spatiotemporal dynamics of the retrieval of different dimensions of word meaning during silent single word reading. We compared three semantic dimensions: two perceptual (i.e., implied real-world size and sound) and one conceptual (i.e., semantic category). This approach allowed us to directly test the hypothesis of a temporal hierarchy in the recovery of perceptual and conceptual semantic information. The task, orthogonal to the dimensions of the semantic space we investigated, ensured that the recovered representations emerged spontaneously and were not induced by the task. Moreover, capitalizing on the intrinsically multivariate nature of the MEG signal, we explored stimulus-locked effects in terms of both event-related spectral perturbations and ITC. We found that automatic retrieval of both perceptual and conceptual dimensions occurs rapidly and simultaneously because of the recruitment of different brain regions.

Early Parallel Recovery of Both Perceptual and Conceptual Dimensions

The first main result of this study is that of an early effect of all three dimensions of word meaning investigated (∼200 msec after stimulus onset). This suggests that both perceptual and conceptual dimensions of the semantic space are activated in an automatic and parallel fashion extremely early during reading.

This early timing is consistent with the view that semantic modulation already starts during the “first pass” of the feedforward stream of neural activity triggered by word reading (Chan, Baker, et al., 2011), which resets the phase of theta and alpha oscillations in multiple cortical areas (Halgren et al., 2015; see our Figure 2A) and is likely to initiate the concurrent activation of the both the semantic “hub” and its “spokes.”

Concerning the access to the conceptual dimension, while the majority of the previous studies investigating semantic processes focused on a (relatively) late time window (i.e., the N400), we are not the first to observe a very early semantic category effect. For instance, Dehaene, using EEG, detected a category-selective response differentiating animal names, famous people's names, and verbs and numerals within 250 msec after written word onset (Dehaene, 1995). Moreover, Chan and colleagues reported an early (200 msec) multivariate decoding of a semantic category (i.e., living vs. nonliving) within both written and spoken words (Chan, Halgren, Marinkovic, & Cash, 2011). Importantly, our results seem to reflect at the macroscopic scale the early (150–200 msec) semantic category selectivity observed at the laminar scale of microelectrode recordings in the inferotemporal and perirhinal cortex (Chan, Baker, et al., 2011), showing an effect in the same direction of ours (higher responses for animals than for tools).

Regarding the two perceptual dimensions, our results generally align with data stemming from the investigation of verb processing and its motor-related embodied aspect, which report somatotopically organized semantic differences as early as 150–240 msec after stimulus onset (Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008; Hauk & Pulvermüller, 2004; Pulvermüller, Harle, & Hummel, 2000). Furthermore, the effect of the auditory dimension appears to be stronger than the effect of the visual one, a finding that is coherent with what has been reported when comparing nouns referring to auditory versus visual properties (Bastiaansen et al., 2008). Interestingly, converging evidence stems from studies indicating linguistic compositional effects in the left ATL in very early time windows (from 175 to 275 msec; e.g., Bemis & Pylkkänen, 2011), specifically those suggesting that this phenomenon is modulated by the specificity of the features being integrated: constructing the concept of “tomato soup” generates more activity than the generic “vegetable soup” (Zhang & Pylkkänen, 2015; Westerlund & Pylkkänen, 2014).

We speculate that previous studies relying only on ERP/ERF may have missed early semantic effects because, although most of the evoked signal in the first 300 msec after stimulus presentation is likely dominated by the processing of low-level physical properties of the stimuli, possibly washing out more subtle semantic differences, ITC/power estimation can segregate different effects in different frequency bands, resulting in an increased sensitivity (Salisbury & Taylor, 2012; Mormann et al., 2005). N400-like semantic effects observed with ERP/ERFs probably relate to a successive step of contextual integration of word meaning, coincident with a phase in which top–down and local associations predominate (see alpha desynchronization in Figure 2B) and during which semantic effects propagate more anteriorly (see, e.g., the late conceptual effect in Figure 3B).

To our best knowledge, this study provides the first direct comparison of the neurodynamics of the recovery of both perceptual and conceptual features of concrete nouns. We failed to detect any temporal hierarchy: Even at the high temporal resolution offered by the MEG signal, perceptual and conceptual semantic information appear to be recovered at the same time, around 200 msec poststimuli onset. It should be noted that the stimuli and task chosen (i.e., semantic verification on single written word) limit the generalization of the phenomena we observe. For any given concept (e.g., piano), not only different weights are assigned to one feature over another (e.g., sound > size; Binder et al., 2016; Hoffman & Lambon Ralph, 2013), but they also need to be dynamically changed as a function of the task at hand (e.g., playing it vs. moving it). Further studies shall parametrically modulate these variables to compare their effects on the latency with which we access specific dimensions of the semantic space. As done here, care needs to be taken while selecting the stimuli as multiple psycholinguistic variables might interfere with early semantic effects, for instance, morphological effects are known to modulate brain activity within the first ∼170 msec after stimuli onset (Flick et al., 2018; Gwilliams & Marantz, 2018; Brooks & Cid de Garcia, 2015; Pylkkänen & Marantz, 2003).

Overall, we show how concepts are brought about by an automatic and rapid integration of different semantic features. This result is consistent with the predictions of the hub-and-spoke model, thus adding to the growing body of electrophysiological, behavioral, neuropsychological, and computational evidence that supports this account (Mollo et al., 2017; Chen et al., 2016; Lambon Ralph, Sage, Jones, & Mayberry, 2010; Rogers & Patterson, 2007; Rogers et al., 2004).

Spectral and Source Dissociation between Perceptual and Conceptual Dimensions

The second key finding of this research is that, while sharing similar temporal dynamics, perceptual and conceptual dimensions appear to dissociate in terms of the signal property that appears to encode them and, at least partially, the underlying generating sources.

The earliest perceptual effects appear in phase-locking changes: Around 200 msec after stimulus onset, phase coherence is modulated by the visual dimension of word meaning mainly in occipital areas and by the auditory dimension of word meaning mainly in occipitotemporal areas. On the other hand, in the same time window (∼200 msec), the conceptual effect is revealed by power changes. One tentative interpretation of this dissociation is that recovering perceptual and conceptual dimensions of word meaning might require deployment of different computational resources from dedicated brain networks. Perceptual effects may involve a partial reinstatement of brain activity elicited by the perception of the real-world aspect of interest (sound or size; Kiefer et al., 2008), an activation that, due to the specific laminar and connectivity structure of sensory areas, might rely on frequency-specific phase resetting similar to the one triggered directly by a sensory stimulus (Makeig et al., 2002) or by multisensory integration (Lakatos, Chen, O'Connell, Mills, & Schroeder, 2007). By contrast, the retrieval of conceptual features in multimodal, associative cortices might rely on modulatory nonphased locked feedback mechanisms. Findings from a recent MEG–TMS study support the idea that the retrieval of stable, complex, semantic representations in ATL is linked with neural oscillations that are not necessarily phase-locked to stimulus onset, thus best appreciated in total power changes (Teige et al., 2018). The growing availability of direct recordings of spontaneous and task-based neuronal activity will allow testing of these hypotheses by revealing the dynamics of feedforward and feedback connections in specific brain areas.

Finally, the different dimensions also seem to be partially dissociated in terms of their underlying sources. On one hand, the visual and auditory properties detected in the phase coherence changes are predominantly linked to occipital and mid and superior temporal regions, thus mostly involving modality-specific areas. On the other hand, the semantic category effect observed in power changes is linked to left mid-inferior and anterior temporal regions, traditionally associated with multimodal processes and language-related functions. This partial dissociation supports hybrid theories on the neural substrate of semantic representations that assign complementary roles to multimodal convergence areas (“semantic hubs”) and modality specific cortices (“spokes”; Lambon Ralph et al., 2017). In particular, our results align with a recent report of a synchronized engagement of amodal hub(s) and sensory-motor spokes during word reading (Mollo et al., 2017). Overall, our findings are in agreement with converging evidence from neuroimaging and neuropsychological studies: Semantic knowledge appears to be encoded in distributed (Huth, de Heer, Griffiths, Theunissen, & Gallant, 2016) yet specialized (Fernandino et al., 2015) cortical areas, with lesions to different portions of the system leading to specific semantic deficits (Binney, Embleton, Jefferies, Parker, & Lambon Ralph, 2010; Pobric, Jefferies, & Lambon Ralph, 2010).

Conclusion

In conclusion, our results indicate that different aspects of the meaning of concrete words are retrieved automatically, rapidly, and simultaneously, yet relying on different brain regions and types of neuronal signals. Visual and auditory perceptual semantic aspects (i.e., the implied real-world size and sound) are most clearly observed in terms of phase coherence changes over occipital and temporal regions, respectively. Conversely, conceptual aspects (i.e., the semantic category) are best retrieved in power changes over superior temporal cortices at early time points and anterior ventrotemporal cortices at later time points. Overall, specific perceptual and conceptual dimensions of the semantic space appear to be accessed concurrently yet differentially within the first 200 msec of word reading.

The early contribution of sensory-motor cortices to the retrieval of motor-perceptual dimensions is predicted by embodied views on semantics; however, such theories would not be able to explain the near simultaneous retrieval of the conceptual dimension in associative areas. On the other hand, both the timing and the reconstructed sources of the effects cannot be accommodated by a purely abstract theory on semantics, which considers motor-perceptual effects as postconceptual mental imagery. Thus, our results speak against both a purely embodied model and purely amodal language-based perspective on the neural substrate of semantic representations, calling for a hybrid model where symbolic inputs are followed by a rapid activation of both a transmodal hub (dedicated to the processing of conceptual dimensions) and modality-specific spokes (dedicated to the processing of motor-perceptual dimensions).

Acknowledgments

We would like to thank the LBIOM team of the NeuroSpin center for their help in participant scanning. The research was funded by INSERM, CEA, Collège de France, and University Paris VI.

Reprint requests should be sent to Valentina Borghesani, Department of Neurology, Memory and Aging Center, University of California San Francisco, 675 Nelson Rising Lane, Mission Bay Campus, San Francisco, CA 94158, or via e-mail: valentinaborghesani@gmail.com.

REFERENCES

Bastiaansen
,
M.
, &
Hagoort
,
P.
(
2006
).
Oscillatory neuronal dynamics during language comprehension
.
Progress in Brain Research
,
159
,
179
196
.
Bastiaansen
,
M. C.
,
Oostenveld
,
R.
,
Jensen
,
O.
, &
Hagoort
,
P.
(
2008
).
I see what you mean: Theta power increases are involved in the retrieval of lexical semantic information
.
Brain and Language
,
106
,
15
28
.
Bemis
,
D. K.
, &
Pylkkänen
,
L.
(
2011
).
Simple composition: A magnetoencephalography investigation into the comprehension of minimal linguistic phrases
.
Journal of Neuroscience
,
31
,
2801
2814
.
Binder
,
J. R.
,
Conant
,
L. L.
,
Humphries
,
C. J.
,
Fernandino
,
L.
,
Simons
,
S. B.
,
Aguilar
,
M.
, et al
(
2016
).
Toward a brain-based componential semantic representation
.
Cognitive Neuropsychology
,
33
,
130
174
.
Binney
,
R. J.
,
Embleton
,
K. V.
,
Jefferies
,
E.
,
Parker
,
G. J.
, &
Lambon Ralph
,
M. A.
(
2010
).
The ventral and inferolateral aspects of the anterior temporal lobe are crucial in semantic memory: Evidence from a novel direct comparison of distortion-corrected fMRI, rTMS, and semantic dementia
.
Cerebral Cortex
,
20
,
2728
2738
.
Borghesani
,
V.
,
Pedregosa
,
F.
,
Buiatti
,
M.
,
Amadon
,
A.
,
Eger
,
E.
, &
Piazza
,
M.
(
2016
).
Word meaning in the ventral visual path: A perceptual to conceptual gradient of semantic coding
.
Neuroimage
,
143
,
128
140
.
Borghesani
,
V.
, &
Piazza
,
M.
(
2017
).
The neuro-cognitive representations of symbols: The case of concrete words
.
Neuropsychologia
,
105
,
4
17
.
Brooks
,
T. L.
, &
Cid de Garcia
,
D.
(
2015
).
Evidence for morphological composition in compound words using MEG
.
Frontiers in Human Neuroscience
,
9
,
215
.
Chan
,
A. M.
,
Baker
,
J. M.
,
Eskandar
,
E.
,
Schomer
,
D.
,
Ulbert
,
I.
,
Marinkovic
,
K.
, et al
(
2011
).
First-pass selectivity for semantic categories in human anteroventral temporal lobe
.
Journal of Neuroscience
,
31
,
18119
18129
.
Chan
,
A. M.
,
Halgren
,
E.
,
Marinkovic
,
K.
, &
Cash
,
S. S.
(
2011
).
Decoding word and category-specific spatiotemporal representations from MEG and EEG
.
Neuroimage
,
54
,
3028
3039
.
Chen
,
L.
,
Lambon Ralph
,
M. A.
, &
Rogers
,
T. T.
(
2017
).
A unified model of human semantic knowledge and its disorders
.
Nature Human Behaviour
,
1
,
0039
.
Chen
,
Y.
,
Shimotake
,
A.
,
Matsumoto
,
R.
,
Kunieda
,
T.
,
Kikuchi
,
T.
,
Miyamoto
,
S.
, et al
(
2016
).
The ‘when’ and ‘where’ of semantic coding in the anterior temporal lobe: Temporal representational similarity analysis of electrocorticogram data
.
Cortex
,
79
,
1
13
.
Dehaene
,
S.
(
1995
).
Electrophysiological evidence for category-specific word processing in the normal human brain
.
NeuroReport
,
6
,
2153
2157
.
Fernandino
,
L.
,
Binder
,
J. R.
,
Desai
,
R. H.
,
Pendl
,
S. L.
,
Humphries
,
C. J.
,
Gross
,
W. L.
, et al
(
2015
).
Concept representation reflects multimodal abstraction: A framework for embodied semantics
.
Cerebral Cortex
,
26
,
2018
2034
.
Flick
,
G.
,
Oseki
,
Y.
,
Kaczmarek
,
A. R.
,
Al Kaabi
,
M.
,
Marantz
,
A.
, &
Pylkkänen
,
L.
(
2018
).
Building words and phrases in the left temporal lobe
.
Cortex
,
106
,
213
236
.
Goto
,
T.
,
Hirata
,
M.
,
Umekawa
,
Y.
,
Yanagisawa
,
T.
,
Shayne
,
M.
,
Saitoh
,
Y.
, et al
(
2011
).
Frequency-dependent spatiotemporal distribution of cerebral oscillatory changes during silent reading: A magnetoencephalograhic group analysis
.
Neuroimage
,
54
,
560
567
.
Gross
,
J.
,
Baillet
,
S.
,
Barnes
,
G. R.
,
Henson
,
R. N.
,
Hillebrand
,
A.
,
Jensen
,
O.
, et al
(
2013
).
Good practice for conducting and reporting MEG research
.
Neuroimage
,
65
,
349
363
.
Gwilliams
,
L.
, &
Marantz
,
A.
(
2018
).
Morphological representations are extrapolated from morpho-syntactic rules
.
Neuropsychologia
,
114
,
77
87
.
Halgren
,
E.
,
Kaestner
,
E.
,
Marinkovic
,
K.
,
Cash
,
S. S.
,
Wang
,
C.
,
Schomer
,
D. L.
, et al
(
2015
).
Laminar profile of spontaneous and evoked theta: Rhythmic modulation of cortical processing during word integration
.
Neuropsychologia
,
76
,
108
124
.
Halgren
,
E.
,
Wang
,
C.
,
Schomer
,
D. L.
,
Knake
,
S.
,
Marinkovic
,
K.
,
Wu
,
J.
, et al
(
2006
).
Processing stages underlying word recognition in the anteroventral temporal lobe
.
Neuroimage
,
30
,
1401
1413
.
Hauk
,
O.
, &
Pulvermüller
,
F.
(
2004
).
Neurophysiological distinction of action words in the fronto-central cortex
.
Human Brain Mapping
,
21
,
191
201
.
Hinojosa
,
J. A.
,
Martín-Loeches
,
M.
,
Muñoz
,
F.
,
Casado
,
P.
,
Fernández-Frías
,
C.
, &
Pozo
,
M. A.
(
2001
).
Electrophysiological evidence of a semantic system commonly accessed by animals and tools categories
.
Cognitive Brain Research
,
12
,
321
328
.
Hoffman
,
P.
, &
Lambon Ralph
,
M. A.
(
2013
).
Shapes, scents and sounds: Quantifying the full multi-sensory basis of conceptual knowledge
.
Neuropsychologia
,
51
,
14
25
.
Huth
,
A. G.
,
de Heer
,
W. A.
,
Griffiths
,
T. L.
,
Theunissen
,
F. E.
, &
Gallant
,
J. L.
(
2016
).
Natural speech reveals the semantic maps that tile human cerebral cortex
.
Nature
,
532
,
453
458
.
Jackson
,
R. L.
,
Lambon Ralph
,
M. A.
, &
Pobric
,
G.
(
2015
).
The timing of anterior temporal lobe involvement in semantic processing
.
Journal of Cognitive Neuroscience
,
27
,
1388
1396
.
Kiefer
,
M.
,
Sim
,
E.-J.
,
Herrnberger
,
B.
,
Grothe
,
J.
, &
Hoenig
,
K.
(
2008
).
The sound of concepts: Four markers for a link between auditory and conceptual brain systems
.
Journal of Neuroscience
,
28
,
12224
12230
.
Kiesel
,
A.
,
Miller
,
J.
,
Jolicoeur
,
P.
, &
Brisson
,
B.
(
2008
).
Measurement of ERP latency differences: A comparison of single-participant and jackknife-based scoring methods
.
Psychophysiology
,
45
,
250
274
.
Kujala
,
J.
,
Vartiainen
,
J.
,
Laaksonen
,
H.
, &
Salmelin
,
R.
(
2012
).
Neural interactions at the core of phonological and semantic priming of written words
.
Cerebral Cortex
,
22
,
2305
2312
.
Kutas
,
M.
, &
Federmeier
,
K. D.
(
2000
).
Electrophysiology reveals semantic memory use in language comprehension
.
Trends in Cognitive Sciences
,
4
,
463
470
.
Lakatos
,
P.
,
Chen
,
C.-M.
,
O'Connell
,
M. N.
,
Mills
,
A.
, &
Schroeder
,
C. E.
(
2007
).
Neuronal oscillations and multisensory interaction in primary auditory cortex
.
Neuron
,
53
,
279
292
.
Lambon Ralph
,
M. A.
,
Jefferies
,
E.
,
Patterson
,
K.
, &
Rogers
,
T. T.
(
2017
).
The neural and computational bases of semantic cognition
.
Nature Reviews Neuroscience
,
18
,
42
55
.
Lambon Ralph
,
M. A.
,
Sage
,
K.
,
Jones
,
R. W.
, &
Mayberry
,
E. J.
(
2010
).
Coherent concepts are computed in the anterior temporal lobes
.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
2717
2722
.
Lau
,
E. F.
,
Gramfort
,
A.
,
Hämäläinen
,
M. S.
, &
Kuperberg
,
G. R.
(
2013
).
Automatic semantic facilitation in anterior temporal cortex revealed through multimodal neuroimaging
.
Journal of Neuroscience
,
33
,
17174
17181
.
Lau
,
E. F.
,
Phillips
,
C.
, &
Poeppel
,
D.
(
2008
).
A cortical network for semantics: (De)constructing the N400
.
Nature Reviews Neuroscience
,
9
,
920
933
.
Mahon
,
B. Z.
, &
Caramazza
,
A.
(
2008
).
A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content
.
Journal of Physiology, Paris
,
102
,
59
70
.
Makeig
,
S.
,
Debener
,
S.
,
Onton
,
J.
, &
Delorme
,
A.
(
2004
).
Mining event-related brain dynamics
.
Trends in Cognitive Sceinces
,
8
,
204
210
.
Makeig
,
S.
,
Westerfield
,
M.
,
Jung
,
T.-P.
,
Enghoff
,
S.
,
Tonwnsend
,
J.
,
Courchesne
,
E.
, et al
(
2002
).
Dynamic brain sources of visual evoked responses
.
Science
,
295
,
690
694
.
Marinkovic
,
K.
,
Dhond
,
R. P.
,
Dale
,
A. M.
,
Glessner
,
M.
,
Carr
,
V.
, &
Halgren
,
E.
(
2003
).
Spatiotemporal dynamics of modality-specific and supramodal word processing
.
Neuron
,
38
,
487
497
.
Maris
,
E.
, &
Oostenveld
,
R.
(
2007
).
Nonparametric statistical testing of EEG- and MEG-data
.
Journal of Neuroscience Methods
,
164
,
177
190
.
Martín-Loeches
,
M.
,
Hinojosa
,
J. A.
,
Gómez-Jarabo
,
G.
, &
Rubia
,
F. J.
(
2001
).
An early electrophysiological sign of semantic processing in basal extrastriate areas
.
Psychophysiology
,
38
,
114
124
.
Miller
,
J.
,
Patterson
,
T.
, &
Ulrich
,
R.
(
1998
).
Jackknife-based method for measuring LRP onset latency differences
.
Psychophysiology
,
35
,
99
115
.
Miller
,
R. G.
(
1974
).
The jackknife—A review
.
Biometrika
,
61
,
1
15
.
Mitra
,
P. P.
, &
Pesaran
,
B.
(
1999
).
Analysis of dynamic brain imaging data
.
Biophysical Journal
,
76
,
691
708
.
Mollo
,
G.
,
Cornelissen
,
P. L.
,
Millman
,
R. E.
,
Ellis
,
A. W.
, &
Jefferies
,
E.
(
2017
).
Oscillatory dynamics supporting semantic cognition: MEG evidence for the contribution of the anterior temporal lobe hub and modality-specific spokes
.
PLoS One
,
12
,
e0169269
.
Mormann
,
F.
,
Fell
,
J.
,
Axmacher
,
N.
,
Weber
,
B.
,
Lehnertz
,
K.
,
Elger
,
C. E.
, et al
(
2005
).
Phase/amplitude reset and theta–gamma interaction in the human medial temporal lobe during a continuous word recognition memory task
.
Hippocampus
,
15
,
890
900
.
Oostenveld
,
R.
,
Fries
,
P.
,
Maris
,
E.
, &
Schoffelen
,
J. M.
(
2011
).
FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data
.
Computational Intelligence and Neuroscience
,
2011
,
156869
.
Pobric
,
G.
,
Jefferies
,
E.
, &
Lambon Ralph
,
M. A.
(
2010
).
Category-specific versus category-general semantic impairment induced by transcranial magnetic stimulation
.
Current Biology
,
20
,
964
968
.
Pulvermüller
,
F.
(
2013
).
How neurons make meaning: Brain mechanisms for embodied and abstract-symbolic semantics
.
Trends in Cognitive Sciences
,
17
,
458
470
.
Pulvermüller
,
F.
(
2018
).
Neural reuse of action perception circuits for language, concepts and communication
.
Progress in Neurobiology
,
160
,
1
44
.
Pulvermüller
,
F.
,
Harle
,
M.
, &
Hummel
,
F.
(
2000
).
Neurophysiological distinction of verb categories
.
NeuroReport
,
11
,
2789
2793
.
Pulvermüller
,
F.
,
Shtyrov
,
Y.
, &
Ilmoniemi
,
R.
(
2005
).
Brain signatures of meaning access in action word recognition
.
Journal of Cognitive Neuroscience
,
17
,
884
892
.
Pylkkänen
,
L.
, &
Marantz
,
A.
(
2003
).
Tracking the time course of word recognition with MEG
.
Trends in Cognitive Sciences
,
7
,
187
189
.
Rogers
,
T. T.
, &
Patterson
,
K.
(
2007
).
Object categorization: Reversals and explanations of the basic-level advantage
.
Journal of Experimental Psychology: General
,
136
,
451
.
Rogers
,
T. T.
,
Ralph
,
L.
,
Matthew
,
A.
,
Garrard
,
P.
,
Bozeat
,
S.
,
McClelland
,
J. L.
, et al
(
2004
).
Structure and deterioration of semantic memory: A neuropsychological and computational investigation
.
Psychological Review
,
111
,
205
.
Salisbury
,
D. F.
, &
Taylor
,
G.
(
2012
).
Semantic priming increases left hemisphere theta power and intertrial phase synchrony
.
Psychophysiology
,
49
,
305
311
.
Tadel
,
F.
,
Baillet
,
S.
,
Mosher
,
J. C.
,
Pantazis
,
D.
, &
Leahy
,
R. M.
(
2011
).
Brainstorm: A user-friendly application for MEG/EEG analysis
.
Computational Intelligence and Neuroscience
,
2011
,
879716
.
Tallon-Baudry
,
C.
,
Bertrand
,
O.
,
Delpuech
,
C.
, &
Pernier
,
J.
(
1996
).
Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human
.
Journal of Neuroscience
,
16
,
4240
4249
.
Teige
,
C.
,
Mollo
,
G.
,
Millman
,
R.
,
Savill
,
N.
,
Smallwood
,
J.
,
Cornelissen
,
P. L.
, et al
(
2018
).
Dynamic semantic cognition: Characterising coherent and controlled conceptual retrieval through time using magnetoencephalography and chronometric transcranial magnetic stimulation
.
Cortex
,
103
,
329
349
.
Ulrich
,
R.
, &
Miller
,
J.
(
2001
).
Using the jackknife-based scoring method for measuring LRP onset effects in factorial designs
.
Psychophysiology
,
38
,
816
827
.
Westerlund
,
M.
, &
Pylkkänen
,
L.
(
2014
).
The role of the left anterior temporal lobe in semantic composition vs. semantic memory
.
Neuropsychologia
,
57
,
59
70
.
Zhang
,
L.
, &
Pylkkänen
,
L.
(
2015
).
The interplay of composition and concept specificity in the left anterior temporal lobe: An MEG study
.
Neuroimage
,
111
,
228
240
.