Abstract

Previous studies have suggested that the posterior parietal cortices and premotor areas are involved in mental image transformation. However, it remains unknown whether these regions really cooperate to realize mental image transformation. In this study, simultaneous EEG and fMRI were performed to clarify the spatio-temporal properties of neural networks engaged in mental image transformation. We adopted a modified version of the mental clock task used by Sack et al. [Sack, A. T., Camprodon, J. A., Pascual-Leone, A., & Goebel, R. The dynamics of interhemispheric compensatory processes in mental imagery. Science, 308, 702–704, 2005; Sack, A. T., Sperling, J. M., Prvulovic, D., Formisano, E., Goebel, R., Di Salle, F., et al. Tracking the mind's image in the brain II: Transcranial magnetic stimulation reveals parietal asymmetry in visuospatial imagery. Neuron, 35, 195–204, 2002]. In the modified mental clock task, participants mentally rotated clock hands from the position initially presented at a learned speed for various durations. Subsequently, they matched the position to the visually presented clock hands. During mental rotation of the clock hands, we observed significant beta EEG suppression with respect to the amount of mental rotation at the right parietal electrode. The beta EEG suppression accompanied activity in the bilateral parietal cortices and left premotor cortex, representing a dynamic cortical network for mental image transformation. These results suggest that motor signals from the premotor area were utilized for mental image transformation in the parietal areas and for updating the imagined clock hands represented in the right posterior parietal cortex.

INTRODUCTION

We can visualize objects in our mind even in the absence of them. This is called mental imagery. We can also transform mental images. For example, we can imagine the view of an object from another angle. In the field of cognitive psychology, however, there has been much debate about whether mental imagery is pictorial or conceptual and propositional (Pylyshyn, 1973). The phenomenon, termed as mental rotation, was first described by Shepard and Metzler (1971) and is one source of support for the argument that mental imagery is pictorial. In their study, participants were presented with a pair of 3-D block patterns that were related by rotation. Participants were asked to indicate whether these were identical or mirror-reversed pairs regardless of rotation. They demonstrated that the participants' RTs increased linearly with increased angular disparity between the two block patterns. This result suggested the following processes occurred during the mental rotation task. First, mental images isomorphic to the 3-D block patterns were generated. Second, one image was “physically” rotated to align it with the other. Third, participants matched the block patterns and judged if they were the same or a mirror-reversed pair. Since their work, a number of studies have confirmed these results (e.g., Tarr & Pinker, 1989; Cooper, 1975; Cooper & Shepard, 1973).

As noninvasive functional brain imaging techniques have developed, the neural basis of mental rotation has been investigated in a number of studies. Most studies have shown activation in the posterior parietal cortex during mental rotation (e.g., Lamm, Windischberger, Moser, & Bauer, 2007; Schendan & Stern, 2007; Wraga, Thompson, Alpert, & Kosslyn, 2003; Kosslyn, Thompson, Wraga, & Alpert, 2001; Richter et al., 2000; Carpenter, Just, Keller, Eddy, & Thulborn, 1999; Kosslyn, DiGirolamo, Thompson, & Alpert, 1998; Richter, Ugurbil, Georgopoulos, & Kim, 1997; Cohen et al., 1996). The posterior parietal cortex is thought to be involved in spatial processing and spatial attention (Cohen et al., 1996). Consistent with this interpretation, it has been reported that activation in the posterior parietal cortex correlated with angular disparity and RTs in mental rotation tasks. For instance, Carpenter et al. (1999) found that bilateral activation in the intraparietal sulci increased with the angular disparity between a pair of Shepard and Metzler type block patterns. Richter et al. (2000) showed that the width of the activation peaks in the bilateral superior parietal lobules and premotor areas was significantly correlated with RTs in a mental rotation task, suggesting that these areas are involved in the execution of mental rotation.

Activation in motor-related areas, such as the primary motor area, premotor area, and SMA, has also been observed in a number of imaging studies of mental rotation (e.g., Lamm et al., 2007; Wraga et al., 2003; Kosslyn et al., 2001; Richter et al., 2000; Cohen et al., 1996). Previous behavioral studies have shown an interaction between mental rotation and hand rotation, suggesting that mental rotation and planning of hand movements share the same mechanism (Wexler, Kosslyn, & Berthoz, 1998; Wohlschläger & Wohlschläger, 1998). However, the exact role of the motor-related areas in mental rotation remains a subject of debate. For instance, it has been suggested that the activation of the primary motor cortex reflects the use of motor imagery (Kosslyn et al., 2001), implying that participants “virtually” rotate objects by using their hands to rotate objects in their mind. Kosslyn et al. (2001) reported activation in the left primary motor area in participants who were familiarized with Shepard and Metzler type block patterns by rotating them with their right (dominant) hand in advance of the fMRI session. The activation in the primary motor area was therefore contralateral to the hand used to rotate the block patterns. However, it should be noted that participants' manual response for judgments might have resulted in the activation of motor-related areas. Richter et al. (2000) reported activations in both primary motor and premotor areas during a mental rotation task but suggested that the primary motor area activation was associated with manual responses rather than mental rotation. Vingerhoets, de Lange, Vandemaele, Deblaere, and Achten (2002) hypothesized that activation in premotor areas is more prominent than activation in the primary motor area during mental rotation because of the input from the posterior parietal cortex that is necessary to spatially guide a potential movement. Zacks (2008) suggested that activity in premotor areas during mental rotation tasks reflects the updating of not only an object-based reference frame but also a coupled object- and effector-based frame. Spatially guided movements, such as tool use, require this coupling. A recent study suggested that the left inferior parietal cortex is involved in the mental imagery of tool use (Imazu, Sugio, Tanaka, & Inui, 2007). Thus, activation in motor-related and parietal cortices during mental rotations might be based on a premotor-parietal network that is involved in spatially guided movements. If the motor-related cortices, in particular the premotor cortex, are involved in mental image transformation, the motor-related and parietal cortices must form the dynamical network when the mental image transformation is carried out.

To test this hypothesis, we performed simultaneous EEG and fMRI. Although EEG has good temporal resolution, its spatial resolution is relatively poor and vice versa for fMRI. By simultaneous measurement with both EEG and fMRI, these deficits can compensate for one another. For example, the combined analysis of EEG and fMRI can reveal cortical regions where BOLD signals correlate to a specific EEG band power modulation. These regions can be considered to form a cortical network within the time window of EEG temporal precision. In this study, we focused on beta-band EEG activity. A previous EEG study demonstrated that beta-band EEG power in the left frontal and bilateral parietal sites was suppressed during mental rotation (Williams, Rippon, Stone, & Annett, 1995). Beta-band power suppression was also observed in the local field potential (LFP) measured in parietal areas while monkeys were performing a prehension task (Asher, Stark, Abeles, & Prut, 2007). In addition, Engel and Fries (2010) reported that the beta oscillation was tightly coupled with the maintenance of motor status. This suggests that beta suppression represents the instantaneous motor execution timing. On the basis of these studies, we hypothesized that if the motor execution signal is crucial for mental image transformation, a parieto-motor network should be detected (1) during mental image transformation and (2) when beta EEG suppression occurs in the parietal cortices.

Mental rotation comprises several subprocesses including generation of a mental image, transformation (e.g., rotation) of a mental image, and matching between a mental image and an input stimulus. In most previous neuroimaging studies using classical mental rotation tasks, however, it was impossible to identify the neural substrates engaged in mental image transformation because mental rotation speed was not controlled, which makes it difficult to dissociate the transformation process from the matching process. In this study, we employed a modified version of the mental clock task (Sack, Camprodon, Pascual-Leone, & Goebel, 2005; Sack et al., 2002; Trojano et al., 2000). The mental clock task is an imagery task in which participants generate visual images of two clock faces (e.g., “2:00” and “5:00”), with auditory presentation of the times on the clock faces, and compare them. We added an element of transformation to the task. In advance of the task, participants learned the rotation speed by manually controlling the movement of clock hands. In the beginning of each trial of the task, participants were visually presented with a clock face, and they memorized the positions of the clock hands. Participants then mentally rotated the clock hands at the learned speed during presentation of a dot. Subsequently, participants matched the clock hands to the visually presented stimulus. Therefore, this task can be considered a sort of mental rotation task in which the speed and amount of rotation can be controlled across participants. In this procedure, matching is carried out after the mental rotation is finished, which permits us to dissociate the mental transformation process.

METHODS

Participants

Twenty-three healthy men participated in the experiment. All participants were right-handed according to the H. N. Handedness Inventory (Hatta & Nakatsuka, 1975). Written informed consent was obtained from each participant in accordance with the Declaration of Helsinki. The ethics committee of the Advanced Telecommunications Research Institute International, Japan, approved all experimental procedures.

Task

The experiment consisted of practice and fMRI sessions. During the practice session, participants actively learned the rotation speed of clock hands (learning phase) and subsequently performed the practice trials (practice phase) outside the MRI scanner.

Learning Phase

In the learning phase, a clock face was presented on a cathode ray tube monitor for 5 min. Participants could rotate the clock hands by manipulating a 3-D mouse (SpaceExplorer, 3DConnexion, Boston, MA) using their right hand. They were asked to learn the rotation speed of the clock hands. The 3-D mouse had a cap that could be twisted in the counterclockwise or clockwise direction. When participants twisted the cap, the clock hands rotated in the direction they twisted. The clock hands rotated at a constant speed (60°/sec for the large clock hand) irrespective of the torque applied to twist the cap. Therefore, this 3-D mouse worked like a switch to control the movement and direction of rotation of the clock hands.

Practice Phase

In the practice phase, participants performed a modified mental clock task. Figure 1 shows the procedure for a single trial. At the beginning of each trial, one clock face was presented for 1.5 sec (first stimulus). Participants were asked to memorize the position of the clock hands. A fixation cross, colored white or red, was presented for 2 sec. The color of the fixation cross was used as a cue to indicate the direction of rotation: White denoted clockwise, and red denoted counterclockwise rotation. Following the presentation of the fixation cross, a dot of the same color was presented. Participants were asked to imagine rotating the clock hands from the initial position in the cued direction at the learned speed until the dot disappeared. The duration of the dot presentation was 1, 3, or 5 sec; that is, the amount of rotation of the large clock hand was 60°, 180°, or 300°, respectively. During the presentation of the fixation cross or the dot, participants were asked to fixate it. After the presentation of a blank screen for 2 sec, a single clock face (second stimulus) was presented. Participants were asked to indicate whether the second clock face was identical to the position of the imagined clock hands rotated correctly at the learned speed while the dot was presented. Responses were made by using the left forefinger or middle finger to press the corresponding key. The second stimulus was presented until the response was made. Following the study by Kosslyn et al. (2001), we used only right-handed participants, so we predicted activation in the motor-related areas in the left hemisphere contralateral to the right hand used to control the 3-D mouse in the learning phase. If responses had been made with the fingers of the right hand, activation in the motor-related areas could not have been dissociated from activation caused by the manual responses. Participants were asked to respond as quickly and accurately as possible. In half of the trials in a single session, the position of the clock hands of the second stimulus was incorrect. The incorrect position of the clock hands was 15 min ahead or behind the correct position, meaning that the participants could perform the task with less attention on the position of the small clock hand than on the large clock hand, although participants needed to rotate both the small and the large clock hands. A beep sound was presented for 150 msec immediately after participants made an incorrect response. The intertrial interval was chosen randomly from 1, 2, or 3 sec on each trial. During the entire intertrial interval, a blank screen was presented. Participants performed 24 trials in a single practice session. Participants completed the learning and practice phases five times.

Figure 1. 

Diagram of the task. Participants were instructed to memorize the locations of the clock hands on the clock face (first stimulus) and rotate them mentally at the learned speed (60°/sec for the large clock hand) during the presentation of a dot for visual fixation. The duration of dot presentation was 1, 3, or 5 sec. The color of the dot indicated the direction of rotation. When the second clock face (second stimulus) was presented, participants indicated whether the locations of the clock hands on the second clock face were identical to those rotated correctly from the initial locations at the learned speed during the presentation of a dot. Participants used their left fingers for responses. The second clock face was presented until the response was made. The bold lines along the time axis represent the period of each event (encoding, transformation, and matching). See Methods for details.

Figure 1. 

Diagram of the task. Participants were instructed to memorize the locations of the clock hands on the clock face (first stimulus) and rotate them mentally at the learned speed (60°/sec for the large clock hand) during the presentation of a dot for visual fixation. The duration of dot presentation was 1, 3, or 5 sec. The color of the dot indicated the direction of rotation. When the second clock face (second stimulus) was presented, participants indicated whether the locations of the clock hands on the second clock face were identical to those rotated correctly from the initial locations at the learned speed during the presentation of a dot. Participants used their left fingers for responses. The second clock face was presented until the response was made. The bold lines along the time axis represent the period of each event (encoding, transformation, and matching). See Methods for details.

Simultaneous EEG and fMRI Recording Session

In the simultaneous EEG and fMRI recording session, participants performed the same task as in the practice sessions, except that no feedback on performance was given. Participants completed two sessions. Each session consisted of 72 trials. In order to dissociate subprocesses in mental rotation, we adopted an event-related design. We created regressors for each task event: (1) the period of presentation of the first stimulus (encoding), (2) the period of presentation of the dot (transformation), (3) the 0.5 sec from the onset of the second stimulus (matching), and (4) from 1 sec after the key response until the onset of the next first stimulus (baseline).1 We decided the duration of the regressors for matching based on the time (524 msec) obtained by subtraction of a standard deviation (352 msec) from the shortest mean RT over the 16 participants used for analysis among all conditions in the amount of rotation (876 msec). The fMRI data for each event were compared with the baseline.

Simultaneous EEG and fMRI Measurement

The EEG data acquisition was performed by a 64-channel MRI-compatible EEG system (BrainProducts, Gilching, Germany). Two MR-compatible amplifiers (Brain Vision MR, BrainProducts) were used to acquire the EEG during fMRI measurements. We used a 10% standard system electrode cap with sintered Ag/AgCl ring electrodes (Brain Cap, Falk Minow Services, Herrsching, Germany). The electrodes had 62 scalp EEG channels, one EOG channel, and one electrocardiogram channel. An FCz electrode, located between Fz and Cz, was used as the measurement reference, and the inion was used as the measurement ground. The electrocardiogram electrode was placed on the participant's back, and the EOG electrode was set on the face, below the left eye. Using the Brain Vision Recorder (Brain Products), the raw EEG was sampled at 5 kHz with a 1-Hz high-pass and a 250-Hz low-pass filter. In simultaneous measurement of EEG and fMRI, a slow-frequency drift of EEG data can sometimes cause MR artifacts to exceed measurement range. In such a case, it is impossible to eliminate MR artifacts. To avoid this, we applied a 1-Hz high-pass filter.

The fMRI data acquisition was performed using a 1.5-T MRI scanner (Shimadzu-Marconi, Kyoto, Japan). In a single experimental run, 320 scans were acquired using an echo-planar T2*-weighted gradient-echo sequence with the following scan parameters: repetition time = 3 sec, echo time = 49 msec, flip angle = 90°, field of view = 192 mm, in-plane resolution = 64 × 64, and slice thickness = 5 mm without gap. T1- and T2-weighted anatomical images were also acquired for each participant.

EEG Data Analysis

Artifacts that originate from MRI and cardio ballistic artifacts contaminate EEG data that are measured simultaneously with fMRI. In order to eliminate such artifacts, the averaged waveforms of the MR and cardio ballistic artifacts were subtracted from the contaminated periods using Brain Vision Analyzer software (BrainProducts; Allen, Josephs, & Turner, 2000; Allen, Polizzi, Krakow, Fish, & Lemieux, 1998). We also removed the ocular artifacts by subtracting the voltages of the vertical and horizontal EOG time series from the EEG data, multiplied by a channel-dependent correction factor calculated on the basis of linear regression of this correction. The EEG data were referred to the average of the EEG electrodes. Vertical and horizontal EOG were computed by subtracting the EOG data from EEG at Fp1 and Fp2 electrodes. The EEG data were then down-sampled to 500 Hz and exported to Matlab (Mathworks, Sherborn, MA, USA) software for further analysis.

We assessed how eye artifacts may have contaminated our data. First, we counted the number of trials that contained artifacts for each participant. If the absolute value of the EEG data during transformation in a trial at any EEG channel and the EOG channel exceeded 50 μV, it was classified as a trial that contained eye artifacts. This analysis revealed that 18.1% of the total 144 trials (mean = 26.2 trials, SD = 33.7 trials) averaged over 16 participants contained artifacts. We classified the artifacts into three categories, “blinks,” “horizontal EOG,” and “vertical EOG,” according to the subtraction of EOG from Fp1 and Fp2, by visual inspection. This revealed that 17.4% of the 144 trials contained eye artifacts by eye blinks, 0.5% by horizontal EOG, 0.1% by vertical EOG, and 0.04% by other causes, including muscular activity. This illustrates that most eye artifacts were caused by eye blinks, but there were few trials contaminated by eye artifacts caused by eye movements. We applied the eye artifact correction algorithm to the EEG data, and 72.3% of the artifacts were successfully corrected. However, the electrode setting for detecting eye movements is not optimal for EEG measurement alone. This is one of the limitations of simultaneous measurement of EEG and fMRI. Thus, we have to note the possibility that saccade artifacts may have remained in the data, because of this limitation in the electrode setting. We did not reject any trial from the analysis, because we needed continuous EEG data to apply a regression analysis based on regressors corresponding to each event. Therefore, the signal-to-noise ratio in our data was not as high as that obtained by EEG measurement alone.

The EEG data were transformed into time–frequency representations of EEG power using the Morlet wavelet, which is defined by the following equation (Tallon-Baudry, Bertrand, Delpuech, & Pernier, 1997), from 2 to 50 Hz in 30 logarithmically spaced steps,
formula
where t is time, f is frequency, and σf is the variance at the frequency f. σf is expressed by σt, which is the variance at time t, as σf = 1/(2πσt). The wavelet is characterized by the ratio of the frequency to the variance (ff) and was set to be ff = 7 in the current study. By convolving the mother wavelet expressed in Equation 1 to the EEG time series sn(t), the EEG power en(t, f) at electrode n was computed by the following equation:
formula
The time–frequency representation of EEG power computed by Equation 2 was used for further analyses.

fMRI Data Analysis

Image processing and statistical analyses were performed using SPM5 software (Wellcome Department of Cognitive Neurology, London, UK, URL: www.fil.ion.ucl.ac.uk/spm). The first three scans were discarded to allow for T1 equilibration. The remaining 317 volumes were corrected for slice timing. The images were spatially realigned to the first of the remaining 317 volumes and re-realigned to the mean of all images to correct for head movements. T2-weighted anatomical images were then coregistered to the mean of the echo-planar images. Coregistered T2-weighted anatomical images were spatially normalized to a Montreal Neurological Institute (MNI) template and spatially smoothed (8-mm FWHM kernel).

The voxel-based statistical analysis on the preprocessed echo-planar images was performed using a general linear model. Each event (encoding, transformation, and matching) was modeled as the onset of the box-car function. The box-car functions were convolved with the canonical hemodynamic response function and then used as the regressors for the regression analysis. The six head motion parameters, derived from the realignment processing, were also used as regressors to reduce the motion-related artifacts. Before the regression analysis, low-frequency confounding effects were removed using a high-pass filter with a 120-sec cutoff period, and serial correlations among scans were estimated with an autoregressive model (AR (1)) to remove the high-frequency noise that contaminated the echo-planar image time series. The regression coefficients for each event were computed for each participant by using the fixed-effects model and then taken into the group analysis using a random-effects model of one sample t test. Activation was considered statistically significant when the height threshold probability value was less than .005 (uncorrected for multiple comparisons at the voxel level) with an extent threshold of 20 voxels relative to the baseline (see also the section “Simultaneous EEG and fMRI recording session”). This combination of height and extent thresholds leads to a threshold at p < .05 (corrected at the cluster level).

Combined Analysis of fMRI and EEG

In order to identify transformation-related components from EEG data, we carried out a regression analysis between the EEG energy and the amount of rotation at each EEG channel. We only used data from the period during the presentation of the dot (transformation) for the analysis. The EEG frequency range was divided into 30 logarithmically spaced steps from 2 to 50 Hz. We first computed regression coefficients for each of 30 EEG frequency steps on an intraindividual basis (fixed effects model) and then carried out a t test of the regression coefficients on an interindividual basis (random effects model). The EEG activity of interest, which was used as a regressor for the combined analysis of fMRI and EEG, was decided by the results of t statistics.

To bridge fMRI and EEG data, it is necessary to manage the difference in the timescales of the fMRI BOLD and EEG signals. BOLD responses are known to appear after EEG activities with some time delay. This time delay is caused by the hemodynamic response (Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001). To analyze the fMRI BOLD signals in combination with the EEG signals, we constructed the expected BOLD responses. First, the time series of EEG power within each frequency band was element-wise multiplied with a vector containing 0 or 1 values for the event of interest (i.e., transformation). In the vector, time steps corresponding to a period when an event occurred were represented by 1, and time steps corresponding to a period when an event did not occur were represented by 0. The time series of EEG data was element-wise multiplied with the vector, enabling EEG data specific to transformation to be extracted. Second, the resulting time series of data for transformation was convolved with the hemodynamic response function. The six realignment parameters were also included in the regressor to reduce the motion artifact. Before the regression analysis, the low-frequency confounding effect and serial correlation were filtered out in the same manner as in the conventional fMRI analysis. The regression coefficient for the EEG activity regressor was computed on an intraindividual basis (fixed effects model). The EEG activity-related regions were then identified by a one-sample t test of the regression coefficient on an interindividual basis (random effects model). Activation was considered statistically significant when the probability value of the height threshold was less than .001 (uncorrected for multiple comparisons at the voxel level)2 with an extent threshold of 20 voxels. Anatomical names and Brodmann's area (BA) labels were determined using an atlas of the human brain (Mai, Assheuer, & Paxinos, 2004). The extension of each activated region was taken into account when determining the anatomical names.

RESULTS

Behavioral Data

The data from seven participants were excluded because the mean correct rate in the final practice session was below chance. Figure 2 shows the mean correct rates for 16 participants. Three-way ANOVA with factors of the Amount of rotation, the Direction of rotation, and the Number of sessions revealed significant main effects of the Amount of rotation, F(2, 30) = 24.39, p < .001, and the Direction of rotation, F(2, 30) = 6.99, p < .05. There were no significant interactions. Tukey's HSD post hoc comparisons revealed significant differences in the Amount of rotation between all conditions (p < .05). We confirmed linear trends for hit rates in the negative and positive directions (all |t(46)| > 4.10; all p < .0002). We also fitted a quadratic function to hit rates. The results showed no significant quadraticity in any direction or session (for all of the coefficients of the second-order terms, all |t(45)| < 0.95; all p > .346). This result is consistent with the results of previous mental rotation studies.

Figure 2. 

Mean hit rate. The dashed line represents results from the first session. The solid line represents results from the second session. The horizontal axis represents the amount of rotation of the large clock hand. The positive number in the amount of rotation corresponds to the counterclockwise direction. The error bars represent standard error.

Figure 2. 

Mean hit rate. The dashed line represents results from the first session. The solid line represents results from the second session. The horizontal axis represents the amount of rotation of the large clock hand. The positive number in the amount of rotation corresponds to the counterclockwise direction. The error bars represent standard error.

We also assessed whether the participants' performances for the plus and minus 300° rotation conditions were above chance. Although hit rates for these rotation conditions were not significantly above chance, d′s for these angles were significantly above zero (all t(15) > 3.92; all p < .001). To make successive transformations of mental imagery at a constant speed, it is necessary to apply a recursive transformation. This means that large amounts of noise will be generated in large transformations; therefore, participants' performance will be poor in the large rotation condition.

The main effect of Direction indicates the facilitation of mental rotation in the minus direction, which corresponds to the clockwise rotation, suggesting an advantage of clockwise rotation. All participants reported that they used visual imagery to perform the task and did not adopt any alternative strategy that did not require visual images.

fMRI Data

Figure 3 shows the brain activation during transformation (n = 16, uncorrected p < .005, cluster threshold k > 20 voxels). We found significant activation in the left motor-related areas and the left posterior parietal cortex, including the left dorsal premotor cortex (BA 6), the left SMA (BA 6), the left ventral premotor cortex (BA 6) extended medially to the opercular part of the inferior frontal gyrus (BA 44), and the left inferior parietal lobule (IPL) extended from the supramarginal gyrus (BA 40) to a part of the anterior intraparietal sulcus (aIPS; BA 7/40). A regression analysis of BOLD signals with the amount of rotation during transformation revealed activation in the bilateral visual areas (BA 18), the right fusiform gyrus (BA 19), the right parieto-occipital junction (BA 7/19), and the left precuneus (Figure 4; Table 1).

Figure 3. 

Activated regions observed during transformation (uncorrected p < .005, cluster threshold k > 20 voxels). PMd = dorsal premotor cortex; PMv = ventral premotor cortex.

Figure 3. 

Activated regions observed during transformation (uncorrected p < .005, cluster threshold k > 20 voxels). PMd = dorsal premotor cortex; PMv = ventral premotor cortex.

Figure 4. 

Rotation-dependent activation during transformation (uncorrected p < .005, cluster threshold k > 20 voxels). The medial view of the brain in the inset shows the activation observed in the precuneus. POJ = parieto-occipital junction; VIS = visual cortex.

Figure 4. 

Rotation-dependent activation during transformation (uncorrected p < .005, cluster threshold k > 20 voxels). The medial view of the brain in the inset shows the activation observed in the precuneus. POJ = parieto-occipital junction; VIS = visual cortex.

Table 1. 

The fMRI Results in Anatomical Regions, MNI Coordinates, and t Values of Significant Activations during Transformation

Region
Cluster Size
t
Coordinates (MNI)
Transformation 
Left precentral gyrus (BA 6) 85 6.28 −30 −15 60 
Left precentral gyrus (BA 6) 4.02 −21 −18 55 
Left precentral gyrus (BA 6) 3.70 −27 −24 45 
Left hippocampus 47 5.74 −27 −48 
Left superior frontal gyrus (BA 6) 28 5.41 −9 −6 65 
Left superior frontal gyrus (BA 6) 4.53 −6 −12 70 
Left precentral gyrus (BA 6) 49 4.57 −63 −3 20 
Left opercular part of inferior frontal gyrus (BA 44) 3.72 −42 15 
Left precentral gyrus (BA 6) 3.29 −54 15 
Right hippocampus 25 4.51 39 −33 −10 
Left supramarginal gyrus (BA 40) 20 3.76 −51 −45 45 
Left aIPS (BA 7/40) 3.29 −39 −48 45 
 
Transformation (Rotation Dependent) 
Right fusiform gyrus (BA 19) 1078 6.13 27 −69 −10 
Right occipital gyrus (BA 18) 5.44 27 −96 −5 
Left occipital gyrus (BA 18) 5.23 −21 −90 15 
Right parieto-occipital junction (BA 7/19) 31 4.31 21 −81 40 
Right parieto-occipital junction (BA 19) 2.99 30 −87 35 
Left precuneus (BA 31) 71 4.25 −12 −51 35 
Left cingulate gyrus (BA 31) 3.51 −6 −33 40 
Left paracentral lobule (BA 5) 3.25 −3 −42 55 
Region
Cluster Size
t
Coordinates (MNI)
Transformation 
Left precentral gyrus (BA 6) 85 6.28 −30 −15 60 
Left precentral gyrus (BA 6) 4.02 −21 −18 55 
Left precentral gyrus (BA 6) 3.70 −27 −24 45 
Left hippocampus 47 5.74 −27 −48 
Left superior frontal gyrus (BA 6) 28 5.41 −9 −6 65 
Left superior frontal gyrus (BA 6) 4.53 −6 −12 70 
Left precentral gyrus (BA 6) 49 4.57 −63 −3 20 
Left opercular part of inferior frontal gyrus (BA 44) 3.72 −42 15 
Left precentral gyrus (BA 6) 3.29 −54 15 
Right hippocampus 25 4.51 39 −33 −10 
Left supramarginal gyrus (BA 40) 20 3.76 −51 −45 45 
Left aIPS (BA 7/40) 3.29 −39 −48 45 
 
Transformation (Rotation Dependent) 
Right fusiform gyrus (BA 19) 1078 6.13 27 −69 −10 
Right occipital gyrus (BA 18) 5.44 27 −96 −5 
Left occipital gyrus (BA 18) 5.23 −21 −90 15 
Right parieto-occipital junction (BA 7/19) 31 4.31 21 −81 40 
Right parieto-occipital junction (BA 19) 2.99 30 −87 35 
Left precuneus (BA 31) 71 4.25 −12 −51 35 
Left cingulate gyrus (BA 31) 3.51 −6 −33 40 
Left paracentral lobule (BA 5) 3.25 −3 −42 55 

Dynamic Cortical Network Revealed by Simultaneous EEG and fMRI

If the motor-related areas contributed to transformation of mental images, the motor and parietal cortices must have formed the dynamic network when the parietal cortex was activated. We first attempted to identify the EEG band and electrode site that could be used as an indicator of parietal cortical activation based on the result of the regression analysis of EEG power only during the transformation period with the amount of rotation.

Figure 5 (bottom) shows histograms of the number of the electrodes that showed a significant correlation between EEG power and the amount of rotation. Consistent with previous studies (Engel & Fries, 2010; Asher et al., 2007; Williams et al., 1995), the beta EEG power (14.7 Hz) was suppressed when the amount of rotation was increased. Figure 5 (top) shows the t map at this beta EEG band. This indicates that the right parietal electrode site exhibited the largest t value. This might indicate that the more the parietal cortex is activated, the more beta EEG power is suppressed at the right parietal scalp electrode site. We also observed electrode sites that exhibited significant correlations in the delta and gamma bands. We observed significant positive correlations between delta band EEG power and the amount of rotation broadly at the left central to parietal and right frontal EEG channels. We observed negative correlations between gamma band EEG power and the amount of rotation at most of the EEG channels. However, we did not use data in these bands. We focused on the beta frequency band because (1) we aimed to identify the network related to motor imagery and (2) the frequency range boundary (2 and 50 Hz) of wavelet analysis could introduce an artifactual correlation.

Figure 5. 

(Top) t Map obtained by the regression analysis of EEG power and the amount of rotation in the 14.7-Hz beta band. The gray circles correspond to individual EEG channels. The shade of the circles indicates the t value obtained by the regression analysis. See Results for details. (Bottom) The number of electrode sites that showed a significant correlation between EEG power and the amount of rotation for each EEG band. The EEG frequency range was divided into 30 equal frequency bands on a log scale.

Figure 5. 

(Top) t Map obtained by the regression analysis of EEG power and the amount of rotation in the 14.7-Hz beta band. The gray circles correspond to individual EEG channels. The shade of the circles indicates the t value obtained by the regression analysis. See Results for details. (Bottom) The number of electrode sites that showed a significant correlation between EEG power and the amount of rotation for each EEG band. The EEG frequency range was divided into 30 equal frequency bands on a log scale.

We identified brain regions where the BOLD signal during transformation was correlated with the temporal fluctuations of beta-band EEG power of the right parietal electrode. The expected BOLD signal was calculated by convolving the 14.7-Hz EEG power obtained from the electrode to the hemodynamic response function. We conducted a regression analysis using the expected BOLD as a regressor. This analysis revealed significant negative correlations between the expected BOLD signal and the measured BOLD signal in the right aIPS and caudal intraparietal sulcus (cIPS), the left intraparietal sulcus (BA 7/40), and the left premotor area (BA 9/6) extended to the opercular part of the left inferior frontal gyrus (BA 44; uncorrected p < .001, cluster threshold k > 20 voxels; Table 2). The activated clusters reported as the aIPS and cIPS in this study partly overlapped with the areas within the standard deviation of the mean peak coordinates of these areas reported in several previous neuroimaging studies (Shikata et al., 2008). Figure 6 shows the activated regions revealed by the combined analysis in red, superimposed by the regions activated during transformation (green) and those activated in a rotation-dependent manner (blue) in the fMRI analysis. The left IPL activation in this analysis overlapped with the activated region in the left IPL observed during transformation in the fMRI analysis (Figure 7, top). Although there was no exact overlap in the activation in the right cIPS with the right parieto-occipital junction found in the regression analysis between the BOLD signal and the amount of rotation, they were adjacent to each other (Figure 7, bottom).

Table 2. 

Results of the Combined Analysis of EEG and fMRI

Region
Cluster Size
t
Coordinates (MNI)
Right superior parietal lobule (BA 7) 75 4.25 36 −72 50 
Right intraparietal sulcus (BA 7/40) 4.06 27 −78 45 
Left intraparietal sulcus (BA 7/40) 28 4.06 −36 −51 45 
Left intraparietal sulcus (BA 7/40) 3.73 −36 −69 55 
Left intraparietal sulcus (BA 7/40) 3.73 −33 −57 50 
Right intraparietal sulcus (BA 7/40) 32 4.04 51 −45 50 
Right supramarginal gyrus (BA 40) 3.46 42 −45 50 
Right intraparietal sulcus (BA 7/40) 3.43 36 −51 50 
Left middle frontal gyrus (BA 9/6) 30 3.78 −54 12 40 
Left opercular part of inferior frontal gyrus (BA 44) 3.53 −54 21 25 
Left middle frontal gyrus (BA 6) 3.45 −42 50 
Region
Cluster Size
t
Coordinates (MNI)
Right superior parietal lobule (BA 7) 75 4.25 36 −72 50 
Right intraparietal sulcus (BA 7/40) 4.06 27 −78 45 
Left intraparietal sulcus (BA 7/40) 28 4.06 −36 −51 45 
Left intraparietal sulcus (BA 7/40) 3.73 −36 −69 55 
Left intraparietal sulcus (BA 7/40) 3.73 −33 −57 50 
Right intraparietal sulcus (BA 7/40) 32 4.04 51 −45 50 
Right supramarginal gyrus (BA 40) 3.46 42 −45 50 
Right intraparietal sulcus (BA 7/40) 3.43 36 −51 50 
Left middle frontal gyrus (BA 9/6) 30 3.78 −54 12 40 
Left opercular part of inferior frontal gyrus (BA 44) 3.53 −54 21 25 
Left middle frontal gyrus (BA 6) 3.45 −42 50 

The anatomical regions, MNI coordinates, and t values for significant activations during transformation.

Figure 6. 

Brain regions activated during transformation observed in the fMRI analysis and the combined analyses of EEG and fMRI. Red: Brain regions where the BOLD signal was correlated with the beta-band EEG power of the right parietal electrode (see Results for details) using the fMRI data during transformation (uncorrected p < .001, cluster threshold k > 20 voxels) revealed by the combined analysis. Green: Activated regions observed in the fMRI analysis (uncorrected p < .005, cluster threshold k > 20 voxels). Blue: Rotation-dependent activations observed in the regression analysis using fMRI data (uncorrected p < .005, cluster threshold k > 20 voxels). Anatomical labels are indicated for the activated regions observed in the combined analysis of EEG and fMRI. PM = premotor cortex.

Figure 6. 

Brain regions activated during transformation observed in the fMRI analysis and the combined analyses of EEG and fMRI. Red: Brain regions where the BOLD signal was correlated with the beta-band EEG power of the right parietal electrode (see Results for details) using the fMRI data during transformation (uncorrected p < .001, cluster threshold k > 20 voxels) revealed by the combined analysis. Green: Activated regions observed in the fMRI analysis (uncorrected p < .005, cluster threshold k > 20 voxels). Blue: Rotation-dependent activations observed in the regression analysis using fMRI data (uncorrected p < .005, cluster threshold k > 20 voxels). Anatomical labels are indicated for the activated regions observed in the combined analysis of EEG and fMRI. PM = premotor cortex.

Figure 7. 

(Top) Left IPL activated during transformation observed in the fMRI (left) and combined analyses of EEG and fMRI (right). (Bottom) Right cIPS regions activated during transformation. (Left) cIPS cluster activated in a rotation-dependent manner observed in the conventional fMRI analysis. (Right) cIPS cluster observed in the combined analysis of EEG and fMRI.

Figure 7. 

(Top) Left IPL activated during transformation observed in the fMRI (left) and combined analyses of EEG and fMRI (right). (Bottom) Right cIPS regions activated during transformation. (Left) cIPS cluster activated in a rotation-dependent manner observed in the conventional fMRI analysis. (Right) cIPS cluster observed in the combined analysis of EEG and fMRI.

DISCUSSION

The current study used simultaneous fMRI and EEG measurements during the mental clock task. fMRI analysis allowed us to identify cortical regions with spatial precision. EEG analysis revealed that beta-band oscillations may be involved in mental spatial transformation. Further analysis of the combined fMRI and EEG revealed that the dynamic cortical network emerges for the mental transformation with the spatial precision of fMRI and the temporal precision of EEG.

Cortical Candidates for Transformation

The fMRI analysis revealed left-dominant activation in motor-related areas and the IPL during transformation. The activation in the left motor-related area is contralateral to the right hand used to manipulate a 3-D mouse in the learning phase. This result is consistent with the findings of Kosslyn et al. (2001). The familiarization method used by Kosslyn et al. (2001) was different from ours in that their participants were allowed to actually manipulate the stimuli. However, previous behavioral studies have suggested that even indirect control over the rotation of objects can result in the use of motor imagery and impact the recognition process. For instance, some studies have reported that seeing various views of 3-D objects rotated by manipulating a trackball or turntable facilitated subsequent object recognition (Sasaoka, Asakura, & Kawahara, 2010; Harman, Humphrey, & Goodale, 1999). Previous studies showed a somatotopic map in the premotor cortex (Schubotz & von Cramon, 2003). We compared the peak coordinate of the left dorsal premotor area observed in this study with the coordinates reported in previous studies. Our peak coordinate (MNI coordinates: −30, −15, 60) was located near the coordinates reported in studies in which participants performed imagery tasks of hand and arm movements (Schubotz & von Cramon, 2003; Figure 2). Although it is possible that the button press by the left hand resulted in activation of ipsilateral motor-related areas, it has been shown that ipsilateral activation reflects the execution of the left hand movement itself rather than movement preparation and that ipsilateral activation is more prominent during a complex left hand movement than a simple one such as tapping (Verstynen, Diedrichsen, Albert, Aparicio, & Ivry, 2005). Thus, the left premotor activation can be interpreted as participants' use of a motor strategy, that is, utilizing information from the motor system for mental image transformation.

We also found left-dominant activation in the IPL. Sack et al. (2005) suggested that the left parietal cortex is predominant in generating mental images and that the right parietal cortex is involved in the spatial comparison process (but can compensate for the function of the left parietal cortex) by applying TMS to the left and right parietal cortices at various times while participants were performing the mental clock task. A recent study that employed a tracking task with a computer mouse suggested that the left IPL played an important role in prediction of movement of an object (i.e., mouse cursor) moved by participants (Ogawa & Inui, 2007). Another fMRI study showed that the left IPL was more active when pantomiming and imagining the use of chopsticks than when actually physically using them (Imazu et al., 2007). Particularly, the peak voxel (MNI coordinates: −51, −45, 45) in the left IPL cluster observed in the current study was located less than 3 mm from that reported by Imazu et al. (2007) (pantomiming > execution, Talairach coordinates: −52, −44, 44). This suggests that the left IPL is also involved in generating mental images of tool use. To solve the modified mental clock task, participants had to update the position of the imagined clock hands. Participants manually controlled a 3-D mouse to learn the rotation speed of the clock hands; therefore, the 3-D mouse might be regarded as a “tool” to rotate mental images. Although the left IPL activation included the aIPS, the left IPL activation peak was more lateral than activation reported as the IPL in previous mental rotation studies (Gauthier et al., 2002; Alivisatos & Petrides, 1997). Zacks (2008) pointed out that variations in activation in parietal regions across mental rotation paradigms reflect the engagement of representations in the effector-based coordinates. Our task might have led to coupling of the object- and effector-based reference frames, which resulted in left IPL activation associated with generating mental imagery of tool use. Therefore, it is likely that participants used the motor imagery of using the 3-D mouse to rotate the imagined clock hands. Thus, the activation in the left IPL suggests that this area is involved in generating and updating the representation of clock hands by utilizing motor commands.

We also found rotation-dependent activation in the right parieto-occipital junction. Podzebenko, Egan, and Watson (2002) reported that similar regions showed significant linear correlation of BOLD response with rotation angle in a mental rotation task using alphanumeric patterns. In particular, it has been suggested that the right parieto-occipital area is involved in processing the spatial relation of visual features of objects (Schendan & Stern, 2007). Schendan and Stern (2008) also showed that this area exhibited activation that increased in a rotation-dependent manner during an object recognition task. This suggests that mental images of objects are represented and transformed in the right parieto-occipital area during mental rotation. We also observed rotation-dependent activation in the bilateral occipital areas. It is possible that the simplicity of the stimulus resulted in the rotation-dependent activation in the primary visual areas. We should note that the modified mental clock task differs from the typical mental rotation tasks, because mental rotation was performed at the learned speed and the stimuli were 2-D lines, which are much simpler than those used in typical mental rotation tasks. These differences might result in activation in regions that are not typically active in mental rotation studies. Nevertheless, we confirmed the linearity of the participants' hit rates with respect to the rotation angle. This result is consistent with previous mental rotation studies. Moreover, the transformation process of simple 2-D lines is essential for 2-D mental rotation, because it requires rotation of a main axis of an object, which is a cue to determine the orientation of an object.

Index of Motor Execution

As we expected based on previous studies (Engel & Fries, 2010; Asher et al., 2007; Williams et al., 1995), we found a beta band (14.7 Hz) of EEG suppression that was associated with the increase in the amount of rotation during transformation (Figure 5). Asher et al. (2007) reported that the beta power of the LFP in the macaque posterior parietal cortex decreased during a reach-to-grasp task. In particular, the beta power of the LFP in Area 5 and the aIPS tended to decrease when monkeys were preparing reach-to-grasp movements. Engel and Fries (2010) proposed that the beta oscillation was for maintaining the motor state (i.e., steady-state force output for achieving movements) and was suppressed when the motor state changed. Furthermore, de Lange, Jensen, Bauer, and Toni (2008) reported that beta suppression was observed during motor imagery of hands. On the basis of these findings, the beta suppression observed in the current study can be presumed to have occurred when the fixation cross was replaced by the dot in the modified mental clock task. The motor imagery was utilized for rotating the imagined clock hands, that is, for simulation of the result of the manual rotation of clock hands without actual execution of manual rotation, which resulted in a change of the motor state.

Dynamic Cortical Network for Transformation

We hypothesized that parietal and motor-related cortices would emerge in the dynamic network for mental image transformation with the beta EEG suppression. The fMRI analysis without temporal information revealed by EEG showed that left motor-related areas and the posterior parietal cortex were activated during transformation. However, based on the spatial information from the fMRI analysis, it is still an open question as to whether the motor cortex really contributes to mental image transformation by forming a network with the parietal cortex. The combined analysis of EEG and fMRI allowed us to more thoroughly evaluate our hypothesis.

The combined analysis of EEG and fMRI revealed that the bilateral parietal cortices and the left premotor area were activated when beta-band EEG power was suppressed. In particular, the beta power of the LFP in the anterior intraparietal area tended to decrease in preparing reach-to-grasp movements (Asher et al., 2007). Asher et al. (2007) also conducted a single-unit recording study of the same area and showed a negative correlation between the beta power in the LFP and spike rate. This observation is consistent with the negative correlation between the BOLD signal and beta EEG power observed in this study. Accordingly, our results indicate that these areas dynamically formed a cortical network with the beta suppression during transformation. In particular, we observed two activated clusters in the right aIPS and cIPS. The aIPS and cIPS are suggested to be the homologues of macaque aIPS and cIPS, respectively (Culham & Valyear, 2006; Culham & Kanwisher, 2001). Previous electrophysiological studies have shown that the aIPS and cIPS play important roles in grasping. For instance, Murata, Gallese, Kaseda, and Sakata (1996) reported that the shape of the target to be grasped modulated the activation of neurons in the aIPS. To achieve successful grasping, it is necessary that the motor commands to control effectors are appropriately integrated with the visual information of the target shape. The properties of the aIPS neurons reported by Murata et al. (1996) suggest that integrated visuomotor representations exist in the aIPS and are involved in planning of grasping depending on the shape of targets. Moreover, activation of aIPS neurons can be modulated by motor signals from the motor areas (Snyder, Grieve, Brotchie, & Andersen, 1998). It has also been shown that the aIPS has strong connections with the ventral premotor area (e.g., Luppino, Murata, Govoni, & Matelli, 1999). This connection is a part of the AIP-F5-F1 circuit, suggesting that it is responsible for planning and execution of grasping (Fagg & Arbib, 1998). Through this strong connection between parietal and premotor cortices, the motor signals that determine the angular velocity of mental rotation from the ventral premotor area could be transferred to the aIPS and could realize integration of visuomotor information at the aIPS. cIPS neurons show selectivity for the orientation of the longitudinal axis of objects (Sakata et al., 1998; Sakata & Taira, 1994). Although there was no exact overlap, the cIPS was adjacent to the site of the rotation-dependent activation in the parieto-occipital junction observed in fMRI correlation analysis (Figure 7), suggesting that these include the same anatomical region. The left IPL activation was observed again in the combined analysis of EEG and fMRI. The fMRI analysis suggested that this area was involved in updating the mental image of clock hands by utilizing motor commands. The combined analysis showed that this area was activated during the same time window as premotor, right aIPS and cIPS activation. This result provides strong support for our interpretation of the activation in the left IPL. On the other hand, there was no overlap between the regions of the left premotor areas observed in the fMRI and combined analyses. However, the activated region observed in the combined analysis was adjacent to both the dorsal and ventral clusters in the premotor cortex observed in the fMRI analysis. We classified this as the premotor area because the activated region mainly included BA 6, which corresponds to the caudal part of the middle frontal gyrus. As this cluster included BA 9 and BA 44, we cannot entirely reject the possibility that this reflected a cognitive function (e.g., working memory) rather than motor processing only. However, the BOLD signal in this region was negatively correlated with the beta-band EEG power, which was related to motor processing. From an electrophysiological standpoint, therefore, it is appropriate to interpret this activation as the use of motor imagery.

Seurinck, de Lange, Achten, and Vingerhoets (2011) also reported the activations in the middle intraparietal sulcus and dorsal premotor area during mental rotation and interpreted these activations as the reflection of attention and eye movement control implemented by the lateral intraparietal sulcus and FEF. Although the cluster in the dorsal premotor area observed in our conventional fMRI analysis extended to more caudal areas, we cannot entirely exclude the possibility that this cluster includes the FEF. In the case of the combined analysis of EEG and fMRI, however, the activated regions observed in the current study were obviously located more ventrally than those reported in the study by Seurinck et al. Furthermore, in the combined analysis, we found two clearly distinct clusters in the aIPS and cIPS, which are relatively close to the coordinates of aIPS and posterior IPS reported by Seurinck et al. (2011), respectively. However, in a previous study of mental rotation, activations were reported in the FEF and the intraparietal sulcus, such as the aIPS, posterior IPS, and a part of the cIPS in the saccade task employed as a control (Schendan & Stern, 2007). In this study, participants were instructed to fixate a dot during mental imagery transformation. The FEF was not involved in the cortical network revealed by the combined analysis. Nevertheless, as we did not directly monitor eye movements other than the EOG channel in this study, we cannot entirely exclude the possibility that the intraparietal sulcus activation reflected eye movements. Also, because of limitations in the electrode settings, saccade artifacts might not be entirely detected by the EOG channel alone. We need to examine this issue in a future study.

Taken together, the results of the combined analysis of EEG and fMRI suggest that motor processing was involved in mental transformation with the cooperation of the parietal areas and served to update the imagined clock hands represented in the right cIPS. Although there was overlap between the results from our fMRI and combined analyses of EEG and fMRI, there also existed some differences. There are three possible explanations for this discrepancy. The first is that activation of these regions was relatively independent of the amount of rotation. The second is that these regions formed a cortical network correlated to non-beta EEG power. In this study, we focused on the beta-band EEG power change based on an a priori hypothesis from previous studies. The activated areas observed in the fMRI analysis could include BOLD signal changes caused by the neural activity related to the non-beta EEG bands. Nevertheless, in this study, we found the cortical network that emerged with the beta EEG suppression. This was the result of using the combined analysis of EEG and fMRI, illustrating the advantage of combined analysis over analyses based only on BOLD signals. The third is the difference in the signal-to-noise ratio between the fMRI and combined analyses. In the fMRI analysis, we conducted a conventional correlation analysis between the BOLD signal and task events. In the combined analysis, we examined correlations between the BOLD signal and EEG power, that is, between two physiological signals. This could yield a higher signal-to-noise ratio than the analysis based only on fMRI data. For instance, we observed activation in the right aIPS only in the combined analysis. In the fMRI analysis, we observed activation in the right aIPS at a low threshold (Z = 1.77, p = .0384, uncorrected). Also, we could not find rotation-dependent activation in the motor-related areas in the fMRI analysis. At a low threshold, however, we found rotation-dependent activation in the premotor cortex (Z = 1.88, p = .0303, uncorrected), located near to the activation observed in the combined analysis.

Although we carefully created regressors in the fMRI analysis to dissociate the transformation process from encoding and matching processes in mental rotation, we have to acknowledge the possibility that these processes might not be perfectly separable because of the slow temporal resolution of fMRI. However, such deficits in the fMRI analysis would be compensated by the combined analysis of EEG and fMRI. Our finding that the parieto-premotor network emerged when beta oscillation was suppressed could be obtained by simultaneous measurement with both EEG and fMRI, but not by EEG or fMRI measurement alone.

Conclusion

In this study, we used simultaneous measurement of EEG and fMRI to reveal the parieto-premotor network involved in mental image transformation. This network was negatively correlated with beta-band EEG power, indicating that this network emerged when beta-band EEG power was suppressed. This network included the right aIPS, right cIPS, left IPL, and left premotor cortex, suggesting that the premotor area was dynamically utilized for coordinate transformation in the parietal areas and for updating the imagined clock hands represented in the right cIPS. Our results provide the first evidence that these areas are linked for mental image transformation.

Acknowledgments

We would like to thank Nobuhiko Asakura, Naoyuki Sato, and Yoko Yamaguchi for helpful discussions. This work was supported by a Grant-in-Aid for Scientific Research (S) (20220003) from JSPS and a Grant-in-Aid for Scientific Research on Innovative Areas “Constructive Developmental Science” (25119503) from MEXT, Japan.

Reprint requests should be sent to Takafumi Sasaoka, Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan, or via e-mail: sasaoka@i.kyoto-u.ac.jp.

Notes

1. 

We did not model the period between the dot and onset of the second stimulus. To model this period would allow us to exclude the activation of the default mode network. In this study, however, the default mode network was not our focus. Moreover, it may have resulted in multicollinearity in the regressors because of the redundancy of regressors. For these reasons, we did not model the period between the dot and onset of the second stimulus in this study.

2. 

In reporting the results of fMRI analysis, we used a threshold of p < .005 uncorrected but also an extent threshold so as to be a threshold at p < .05 corrected at the cluster level. Without an extent threshold, even at the .001 threshold level, we confirmed significant activations in the same areas reported at the .005 threshold.

REFERENCES

REFERENCES
Alivisatos
,
B.
, &
Petrides
,
M.
(
1997
).
Functional activation of the human brain during mental rotation.
Neuropsychologia
,
35
,
111
118
.
Allen
,
P. J.
,
Josephs
,
O.
, &
Turner
,
R.
(
2000
).
A method for removing imaging artifact from continuous EEG and recorded during functional MRI.
Neuroimage
,
12
,
230
239
.
Allen
,
P. J.
,
Polizzi
,
G.
,
Krakow
,
K.
,
Fish
,
D. R.
, &
Lemieux
,
L.
(
1998
).
Identification of EEG events in the MR scanner: The problem of pulse artifact and a method for its subtraction.
Neuroimage
,
8
,
229
239
.
Asher
,
I.
,
Stark
,
E.
,
Abeles
,
M.
, &
Prut
,
Y.
(
2007
).
Comparison of direction and object selectivity of local field potentials and single units in macaque posterior parietal cortex during prehension.
Journal of Neurophysiology
,
97
,
3684
3695
.
Carpenter
,
P. A.
,
Just
,
M. A.
,
Keller
,
T. A.
,
Eddy
,
W.
, &
Thulborn
,
K.
(
1999
).
Graded functional activation in the visuospatial system with the amount of task demand.
Journal of Cognitive Neuroscience
,
11
,
9
24
.
Cohen
,
M. S.
,
Kosslyn
,
S. M.
,
Breiter
,
H. C.
,
Digirolamo
,
G. J.
,
Thompson
,
W. L.
,
Anderson
,
A. K.
,
et al
(
1996
).
Changes in cortical activity during mental rotation: A mapping study using functional magnetic resonance imaging.
Brain
,
119
,
89
100
.
Cooper
,
L. A.
(
1975
).
Mental rotation of random two-dimensional shapes.
Cognitive Psychology
,
7
,
20
43
.
Cooper
,
L. A.
, &
Shepard
,
R. N.
(
1973
).
Chronometric studies of the rotation of mental images.
In W. G. Chase (Ed.)
,
Visual information processing
(pp.
75
176
).
New York
:
Academic Press
.
Culham
,
J. C.
, &
Kanwisher
,
N. G.
(
2001
).
Neuroimaging of cognitive functions in human parietal cortex.
Current Opinion in Neurobiology
,
11
,
157
163
.
Culham
,
J. C.
, &
Valyear
,
K. F.
(
2006
).
Human parietal cortex in action.
Current Opinion in Neurobiology
,
16
,
205
212
.
de Lange
,
F. P.
,
Jensen
,
O.
,
Bauer
,
M.
, &
Toni
,
I.
(
2008
).
Interactions between posterior gamma and frontal alpha/beta oscillations during imagined actions.
Frontiers in Human Neuroscience
,
2
,
7
.
Engel
,
A. K.
, &
Fries
,
P.
(
2010
).
Beta-band oscillations—Signalling the status quo?
Current Opinion in Neurobiology
,
20
,
156
165
.
Fagg
,
A. H.
, &
Arbib
,
M. A.
(
1998
).
Modeling parietal-premotor interactions in primate control of grasping.
Neural Networks
,
11
,
1277
1303
.
Gauthier
,
I.
,
Hayward
,
W. G.
,
Tarr
,
M. J.
,
Anderson
,
A. W.
,
Skudlarski
,
P.
, &
Gore
,
J. C.
(
2002
).
BOLD activity during mental rotation and viewpoint-dependent object recognition.
Neuron
,
34
,
161
171
.
Harman
,
K. L.
,
Humphrey
,
G. K.
, &
Goodale
,
M. A.
(
1999
).
Active manual control of object views facilitates visual recognition.
Current Biology
,
9
,
1315
1318
.
Hatta
,
T.
, &
Nakatsuka
,
Z.
(
1975
).
Handedness inventory.
In D. Ohno (Ed.)
,
Papers on celebrating 63rd birthday of Prof. Ohnishi
(pp.
224
245
).
Osaka, Japan
:
Osaka City University
.
Imazu
,
S.
,
Sugio
,
T.
,
Tanaka
,
S.
, &
Inui
,
T.
(
2007
).
Differences between actual and imagined usage of chopsticks: An fMRI study.
Cortex
,
43
,
301
307
.
Kosslyn
,
S. M.
,
DiGirolamo
,
G. J.
,
Thompson
,
W. L.
, &
Alpert
,
N. M.
(
1998
).
Mental rotation of objects versus hands: Neural mechanisms revealed by positron emission tomography.
Psychophysiology
,
35
,
151
161
.
Kosslyn
,
S. M.
,
Thompson
,
W. L.
,
Wraga
,
M.
, &
Alpert
,
N. M.
(
2001
).
Imagining rotation by endogenous versus exogenous forces: Distinct neural mechanisms.
NeuroReport
,
12
,
2519
2525
.
Lamm
,
C.
,
Windischberger
,
C.
,
Moser
,
E.
, &
Bauer
,
H.
(
2007
).
The functional role of dorso-lateral premotor cortex during mental rotation: An event-related fMRI study separating cognitive processing steps using a novel task paradigm.
Neuroimage
,
36
,
1374
1386
.
Logothetis
,
N. K.
,
Pauls
,
J.
,
Augath
,
M.
,
Trinath
,
T.
, &
Oeltermann
,
A.
(
2001
).
Neurophysiological investigation of the basis of the fMRI signal.
Nature
,
412
,
150
157
.
Luppino
,
G.
,
Murata
,
A.
,
Govoni
,
P.
, &
Matelli
,
M.
(
1999
).
Largely segregated parietofrontal connections linking rostral intraparietal cortex (areas AIP and VIP) and the ventral premotor cortex (areas F5 and F4).
Experimental Brain Research
,
128
,
181
187
.
Mai
,
J. K.
,
Assheuer
,
J.
, &
Paxinos
,
G.
(
2004
).
Atlas of the human brain
(2nd ed.).
San Diego, CA
:
Elsevier Academic Press
.
Murata
,
A.
,
Gallese
,
V.
,
Kaseda
,
M.
, &
Sakata
,
H.
(
1996
).
Parietal neurons related to memory-guided hand manipulation.
Journal of Neurophysiology
,
75
,
2180
2186
.
Ogawa
,
K.
, &
Inui
,
T.
(
2007
).
Lateralization of the posterior parietal cortex for internal monitoring of self- versus externally generated movements.
Journal of Cognitive Neuroscience
,
19
,
1827
1835
.
Podzebenko
,
K.
,
Egan
,
G. F.
, &
Watson
,
J. D. G.
(
2002
).
Widespread dorsal stream activation during a parametric mental rotation task, revealed with functional magnetic resonance imaging.
Neuroimage
,
15
,
547
558
.
Pylyshyn
,
Z. W.
(
1973
).
What the mind's eye tells the mind's brain: A critique of mental imagery.
Psychological Bulletin
,
80
,
1
24
.
Richter
,
W.
,
Somorjai
,
R.
,
Summers
,
R.
,
Jarmasz
,
M.
,
Menon
,
R. S.
,
Gati
,
J. S.
,
et al
(
2000
).
Motor area activity during mental rotation studied by time-resolved single-trial fMRI.
Journal of Cognitive Neuroscience
,
12
,
310
320
.
Richter
,
W.
,
Ugurbil
,
K.
,
Georgopoulos
,
A.
, &
Kim
,
S. G.
(
1997
).
Time-resolved fMRI of mental rotation.
NeuroReport
,
8
,
3697
3702
.
Sack
,
A. T.
,
Camprodon
,
J. A.
,
Pascual-Leone
,
A.
, &
Goebel
,
R.
(
2005
).
The dynamics of interhemispheric compensatory processes in mental imagery.
Science
,
308
,
702
704
.
Sack
,
A. T.
,
Sperling
,
J. M.
,
Prvulovic
,
D.
,
Formisano
,
E.
,
Goebel
,
R.
,
Di Salle
,
F.
,
et al
(
2002
).
Tracking the mind's image in the brain II: Transcranial magnetic stimulation reveals parietal asymmetry in visuospatial imagery.
Neuron
,
35
,
195
204
.
Sakata
,
H.
, &
Taira
,
M.
(
1994
).
Parietal control of hand action.
Current Opinion in Neurobiology
,
4
,
847
856
.
Sakata
,
H.
,
Taira
,
M.
,
Kusunoki
,
M.
,
Murata
,
A.
,
Tanaka
,
Y.
, &
Tsutsui
,
K.
(
1998
).
Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
353
,
1363
1373
.
Sasaoka
,
T.
,
Asakura
,
N.
, &
Kawahara
,
T.
(
2010
).
Effect of active exploration of 3-D object views on the view-matching process in object recognition.
Perception
,
39
,
289
308
.
Schendan
,
H. E.
, &
Stern
,
C. E.
(
2007
).
Mental rotation and object categorization share a common network of prefrontal and dorsal and ventral regions of posterior cortex.
Neuroimage
,
35
,
1264
1277
.
Schendan
,
H. E.
, &
Stern
,
C. E.
(
2008
).
Where vision meets memory: Prefrontal-posterior networks for visual object constancy during categorization and recognition.
Cerebral Cortex
,
18
,
1695
1711
.
Schubotz
,
R. I.
, &
von Cramon
,
D. Y.
(
2003
).
Functional-anatomical concepts of human premotor cortex: Evidence from fMRI and PET studies.
Neuroimage
,
20
,
S120
S131
.
Seurinck
,
R.
,
de Lange
,
F. P.
,
Achten
,
E.
, &
Vingerhoets
,
G.
(
2011
).
Mental rotation meets the motion aftereffect: The role of hV5/MT+ in visual mental imagery.
Journal of Cognitive Neuroscience
,
23
,
1395
1404
.
Shepard
,
R. N.
, &
Metzler
,
J.
(
1971
).
Mental rotation of three-dimensional objects.
Science
,
171
,
701
703
.
Shikata
,
E.
,
McNamara
,
A.
,
Sprenger
,
A.
,
Hamzei
,
F.
,
Glauche
,
V.
,
Büchel
,
C.
,
et al
(
2008
).
Localization of human intraparietal areas AIP, CIP, and LIP using surface orientation and saccadic eye movement tasks.
Human Brain Mapping
,
29
,
411
421
.
Snyder
,
L. H.
,
Grieve
,
K. L.
,
Brotchie
,
P.
, &
Andersen
,
R. A.
(
1998
).
Separate body- and world-referenced representations of visual space in parietal cortex.
Nature
,
394
,
887
891
.
Tallon-Baudry
,
C.
,
Bertrand
,
O.
,
Delpuech
,
C.
, &
Pernier
,
J.
(
1997
).
Oscillatory γ-band (30–70 Hz) activity induced by a visual search task in humans.
Journal of Neuroscience
,
17
,
722
734
.
Tarr
,
M. J.
, &
Pinker
,
S.
(
1989
).
Mental rotation and orientation-dependence in shape recognition.
Cognitive Psychology
,
21
,
233
282
.
Trojano
,
L.
,
Grossi
,
D.
,
Linden
,
D. E. J.
,
Formisano
,
E.
,
Hacker
,
H.
,
Zanella
,
F. E.
,
et al
(
2000
).
Matching two imagined clocks: The functional anatomy of spatial analysis in the absence of visual stimulation.
Cerebral Cortex
,
10
,
473
481
.
Verstynen
,
T.
,
Diedrichsen
,
J.
,
Albert
,
N.
,
Aparicio
,
P.
, &
Ivry
,
R. B.
(
2005
).
Ipsilateral motor cortex activity during unimanual hand movements relates to task complexity.
Journal of Neurophysiology
,
93
,
1209
1222
.
Vingerhoets
,
G.
,
de Lange
,
F.
,
Vandemaele
,
P.
,
Deblaere
,
K.
, &
Achten
,
E.
(
2002
).
Motor imagery in mental rotation: An fMRI study.
Neuroimage
,
17
,
1623
1633
.
Wexler
,
M.
,
Kosslyn
,
S. M.
, &
Berthoz
,
A.
(
1998
).
Motor processes in mental rotation.
Cognition
,
68
,
77
94
.
Williams
,
J. D.
,
Rippon
,
G.
,
Stone
,
B. M.
, &
Annett
,
J.
(
1995
).
Psychophysiological correlates of dynamic imagery.
British Journal of Psychology
,
86
,
283
300
.
Wohlschläger
,
A.
, &
Wohlschläger
,
A.
(
1998
).
Mental and manual rotation.
Journal of Experimental Psychology: Human Perception and Performance
,
24
,
397
412
.
Wraga
,
M.
,
Thompson
,
W. L.
,
Alpert
,
N. M.
, &
Kosslyn
,
S. M.
(
2003
).
Implicit transfer of motor strategies in mental rotation.
Brain and Cognition
,
52
,
135
143
.
Zacks
,
J. M.
(
2008
).
Neuroimaging studies of mental rotation: A meta-analysis and review.
Journal of Cognitive Neuroscience
,
20
,
1
19
.