Abstract

From which regions of the brain do conscious representations of visual stimuli emerge? This is an important but controversial issue in neuroscience because some studies have reported a major role of the higher visual regions of the ventral pathway in conscious perception, whereas others have found neural correlates of consciousness as early as in the primary visual areas and in the thalamus. One reason for this controversy has been the difficulty in focusing on neural activity at the moment when conscious percepts are generated in the brain, excluding any bottom–up responses (not directly related to consciousness) that are induced by stimuli. In this study, we address this issue with a new approach that can induce a rapid change in conscious perception with little influence from bottom–up responses. Our results reveal that the first consciousness-related activity emerges from the higher visual region of the ventral pathway. However, this activity is rapidly diffused to the entire brain, including the early visual cortex. These results thus integrate previous “higher” and “lower” views on the emergence of neural correlates of consciousness, providing a new perspective for the temporal dynamics of consciousness.

INTRODUCTION

There is considerable evidence for the “unconscious” processing of sensory stimuli in the brain (Sterzer, Jalkanen, & Rees, 2009; Fang & He, 2005). How such unconscious neural responses are converted into conscious representations involving awareness is an important problem in neuroscience (Dehaene, Changeux, Naccache, Sackur, & Sergent, 2006; Block, 2005; Crick & Koch, 2003; Tononi & Edelman, 1998). One solution is to identify brain regions where this conversion from unconscious to conscious representations takes place. Previous studies have examined these regions using psychological paradigms that produce spontaneous alternations of perceptual states, such as binocular rivalry and multistable stimuli (Sterzer, Kleinschmidt, & Rees, 2009). In these methods, one can induce fluctuations in conscious percepts with input stimuli kept constant, thereby dissociating brain responses directly associated with changes in conscious perception (neural correlates of consciousness) from those that are simply reacting to sensory stimulation. Combining these methods with neurophysiological recordings (Maier et al., 2008; Sheinberg & Logothetis, 1997; Leopold & Logothetis, 1996) and neuroimaging techniques (Haynes, Deichmann, & Rees, 2005; Lee, Blake, & Heeger, 2005; Wunderlich, Schneider, & Kastner, 2005; Tong & Engel, 2001), many studies have successfully identified brain regions, especially those in the ventral visual pathway for object recognition (Haxby et al., 1991; Mishkin & Ungerleider, 1982), that show neural responses consistent with the concept of neural correlates of consciousness. However, the question of “from which stage of the visual pathway do the conscious representations first emerge” remains controversial.

The ventral visual pathway for object recognition can be roughly classified into the following two stages: the lower and higher regions. Neurons in the lower visual regions such as V1 and V2 have smaller receptive fields and encode simple features of stimuli (e.g., orientation of a luminance grating; Hubel & Wiesel, 1979), whereas those in the higher regions such as V4 and the interior temporal cortex have larger receptive fields and show selective responses to specific categories of shapes or objects (Fujita, Tanaka, Ito, & Cheng, 1992). Studies using unit cell recordings from monkeys have argued that neurons in the higher visual region rather than the lower visual region play a major role in creating conscious representations (Sheinberg & Logothetis, 1997; Leopold & Logothetis, 1996). These studies have found that, during binocular rivalry, activities of neurons increase when their preferred stimuli (e.g., a grating with a specific orientation or complex shapes such as faces) come into consciousness compared with when they do not, indicating that neural processing in these higher regions is directly associated with conscious percepts. In contrast, V1 and V2 neurons have shown few or no changes in response to perceptual alternations during binocular rivalry (Maier et al., 2008; Leopold & Logothetis, 1996). Thus, these results suggest that the conscious representations of stimuli emerge from the higher visual region rather than the lower visual region.

However, recent studies using fMRI have provided data that are somewhat inconsistent with this view. These studies have revealed that the lower region in the visual pathway, such as V1 and the LGN, shows responses that are coupled with perceptual changes in binocular rivalry (Haynes et al., 2005; Lee et al., 2005; Wunderlich et al., 2005; Tong & Engel, 2001). Some researchers have indicated that this activity in the early visual pathway may reflect top–down signals from the higher region (Tononi & Koch, 2008), although the properties of these top–down signals have not been completely clarified, partly because of the limited temporal resolution of fMRI. Consequently, the stages of the ventral pathway from which conscious representations emerge remain to be elucidated.

A direct approach to clarify this issue would be to isolate neural activities at the moment when conscious percepts are generated in the brain. However, it has been difficult to use these earlier psychophysical methods (e.g., binocular rivalry) to identify a precise moment at which a new conscious percept emerges because perceptual changes with these methods usually occur at a low speed (Kim & Blake, 2005), gradually switching from one percept to another (not transiently). To solve this problem, we have used a new approach that can induce a rapid change in conscious perception with little influence from bottom–up responses. In particular, we focused on neural activity in the disinhibition phase of continuous flash suppression (CFS; Tsuchiya & Koch, 2005). In CFS, a rapid sequence of colored flashes presented to one eye of the participant renders a target image (e.g., grating) invisible to the other eye (Figure 1). When the continuous flashes disappear, however, the grating is released from dichoptic suppression, changing into consciousness as if it had been presented after the continuous flashes disappeared (subjective-onset trials, Figure 1C). Taking advantage of the fine temporal resolution of magneto-encephalography (MEG), we measured the neural activity at the moment of this transition from unconscious to conscious states of perception, thereby characterizing the temporal dynamics when conscious percepts are generated in the brain.

Figure 1. 

Stimulus sequences in three types of trials in Experiment 1a. (A) No-stimulus trials. Continuous flashes (20 Hz) were presented to the participant's dominant eye, with no stimulus given to the nondominant eye. (B) Physical-onset trials. A low-contrast grating was presented to the nondominant eye after the continuous flashes disappeared at one of four quadrants in the peripheral visual field. (C) Subjective-onset trials. Although the grating during the flash period was kept invisible by the continuous flashes to the dominant eye (continuous flash suppression), it came into the participant's consciousness after the flash disappeared (post-flash period), which resulted in subjective onset of the grating with no physical presentation of the stimulus at that time.

Figure 1. 

Stimulus sequences in three types of trials in Experiment 1a. (A) No-stimulus trials. Continuous flashes (20 Hz) were presented to the participant's dominant eye, with no stimulus given to the nondominant eye. (B) Physical-onset trials. A low-contrast grating was presented to the nondominant eye after the continuous flashes disappeared at one of four quadrants in the peripheral visual field. (C) Subjective-onset trials. Although the grating during the flash period was kept invisible by the continuous flashes to the dominant eye (continuous flash suppression), it came into the participant's consciousness after the flash disappeared (post-flash period), which resulted in subjective onset of the grating with no physical presentation of the stimulus at that time.

As control trials for subjective-onset trials, we employed no-stimulus trials (Figure 1A) and physical-onset trials (Figure 1B). In physical-onset trials, the grating is presented at the offset (not onset) of continuous flashes. Participants should perceive the grating after the flashes disappear (as in subjective-onset trials), although this perception is caused by bottom–up signals from the retina. Comparison of neuromagnetic responses between physical- and subjective-onset trials helps in distinguishing differences in neural activity between conscious representations triggered by bottom–up signals (physical-onset trials) and those created through internal disinhibition of unconscious signals in the brain (subjective-onset trials). In particular, our purpose was to discern whether the first neural responses related to conscious perception emerge from the lower or higher visual region of the ventral pathway. If the consciousness-related activity emerges from the higher visual regions as suggested by previous neurophysiological studies using monkeys (Sheinberg & Logothetis, 1997; Leopold & Logothetis, 1996), we should observe the transmission of neural signals from the higher to lower visual regions in subjective-onset trials, whereas normal transmission in a feedforward (from lower to higher) direction should be observed in physical-onset trials.

This study comprised four experiments. We present our main results in the first experiment (Experiment 1a, Figures 1,23456), in which MEG signals in physical- and subjective-onset trials are compared. Supplementary results of a separate behavioral experiment (Experiment 1b, Figure 7) are provided to ensure that the subjective-onset grating in Experiment 1a remained invisible during the continuous flashes. Finally, we measured neuromagnetic responses again in Experiments 2a and 2b (Figures 8 and 9, respectively), with physical- and subjective-onset trials completely indistinguishable to observers.

Figure 2. 

VEFs of MEG measured in a representative subject in Experiment 1b. (A) VEFs in the no-stimulus trials at 204 gradiometers of MEG. The waveforms recorded from anterior regions are plotted in the upper positions. Groups of gradiometers for the areal mean approach (see Methods) are encompassed by dotted lines. The two posterior groups correspond to Area I, the waveforms of which indicated neural signals from the early visual cortex. In contrast, VEFs within Area II in lateral (ventral) regions of both hemispheres primarily represented neural activity in the higher visual region in the ventral pathway. (B) Enlarged VEFs at three gradiometers from 1 to 3 in A. Because an average epoch of VEFs ranged from 1100 msec before to 1000 msec after the offset of the continuous flashes, the left half of each waveform (−1050 to 0 msec) indicates neuromagnetic responses during the flash period, whereas the right half (0–1000 msec) indicates those in the post-flash period. Note that strong neural activity measured through the gradiometers is represented as either a positive or negative deflection of the waveforms. Activity related to the perception of the grating (arrows) is observed as neuromagnetic components selective for the physical- and subjective-onset trials (red, green, blue, and cyan) compared with the no-stimulus trials (black). All waveforms were filtered (1–30 Hz) for display purposes only. Scale bar, 100 msec for horizontal and 100 fT/cm for vertical directions.

Figure 2. 

VEFs of MEG measured in a representative subject in Experiment 1b. (A) VEFs in the no-stimulus trials at 204 gradiometers of MEG. The waveforms recorded from anterior regions are plotted in the upper positions. Groups of gradiometers for the areal mean approach (see Methods) are encompassed by dotted lines. The two posterior groups correspond to Area I, the waveforms of which indicated neural signals from the early visual cortex. In contrast, VEFs within Area II in lateral (ventral) regions of both hemispheres primarily represented neural activity in the higher visual region in the ventral pathway. (B) Enlarged VEFs at three gradiometers from 1 to 3 in A. Because an average epoch of VEFs ranged from 1100 msec before to 1000 msec after the offset of the continuous flashes, the left half of each waveform (−1050 to 0 msec) indicates neuromagnetic responses during the flash period, whereas the right half (0–1000 msec) indicates those in the post-flash period. Note that strong neural activity measured through the gradiometers is represented as either a positive or negative deflection of the waveforms. Activity related to the perception of the grating (arrows) is observed as neuromagnetic components selective for the physical- and subjective-onset trials (red, green, blue, and cyan) compared with the no-stimulus trials (black). All waveforms were filtered (1–30 Hz) for display purposes only. Scale bar, 100 msec for horizontal and 100 fT/cm for vertical directions.

Figure 3. 

Across-sensor VEFs in Areas I and II. The data from the two hemispheres were averaged and classified as either a contralateral (red) or ipsilateral (blue) condition. In the contralateral condition, the physical- and subjective-onset gratings were presented at the visual field contralateral to the sensor position. VEFs in the no-stimulus trials are shown as baselines (black). Note that in these across-sensor VEFs, a strong MEG response is shown as an upward deflection of the waveforms. (A) Physical-onset trials. (B) Subjective-onset trials. In all conditions, presentation of the continuous flashes (at −1050 msec) provoked strong neuromagnetic responses of two peaks, one at approximately 80–100 msec (M1) and the other at 150–200 msec (M2) after the flash onset. In Area I (around the occipital pole), the M1 component is observed more distinctly than the M2 component, whereas the M2 component is dominant in Area II (the ventrolateral regions of both hemispheres). VEFs from 0 to 400 msec after the flashes disappeared are enlarged in the right panels. In the physical-onset trials (A), M1 and M2 components similar to those in the flash periods are observed. In contrast, M1 is missing from Area I in the subjective-onset trials (B, top right), and the first distinct neuromagnetic responses with a clear laterality are instead observed in Area II at approximately 120 msec (B, bottom right). (C) Source locations of the M1 and M2 components. The three left panels show contour maps of magnetic fields (red lines, outflux; blue lines, influx) from the viewpoints of the left, posterior, and right sides of the brain. The green arrows indicate directions of estimated current flows. Locations of the current dipoles are shown on anatomical magnetic resonance images from one participant (right).

Figure 3. 

Across-sensor VEFs in Areas I and II. The data from the two hemispheres were averaged and classified as either a contralateral (red) or ipsilateral (blue) condition. In the contralateral condition, the physical- and subjective-onset gratings were presented at the visual field contralateral to the sensor position. VEFs in the no-stimulus trials are shown as baselines (black). Note that in these across-sensor VEFs, a strong MEG response is shown as an upward deflection of the waveforms. (A) Physical-onset trials. (B) Subjective-onset trials. In all conditions, presentation of the continuous flashes (at −1050 msec) provoked strong neuromagnetic responses of two peaks, one at approximately 80–100 msec (M1) and the other at 150–200 msec (M2) after the flash onset. In Area I (around the occipital pole), the M1 component is observed more distinctly than the M2 component, whereas the M2 component is dominant in Area II (the ventrolateral regions of both hemispheres). VEFs from 0 to 400 msec after the flashes disappeared are enlarged in the right panels. In the physical-onset trials (A), M1 and M2 components similar to those in the flash periods are observed. In contrast, M1 is missing from Area I in the subjective-onset trials (B, top right), and the first distinct neuromagnetic responses with a clear laterality are instead observed in Area II at approximately 120 msec (B, bottom right). (C) Source locations of the M1 and M2 components. The three left panels show contour maps of magnetic fields (red lines, outflux; blue lines, influx) from the viewpoints of the left, posterior, and right sides of the brain. The green arrows indicate directions of estimated current flows. Locations of the current dipoles are shown on anatomical magnetic resonance images from one participant (right).

Figure 4. 

Temporal dynamics of the neural activity to the subjective-onset grating. All the waveforms were differential VEFs in which the across-sensor VEFs in the no-stimulus trials were subtracted from those in the physical- or subjective-onset trials. (A) Comparison of VEFs in Areas I between the physical- (red) and subjective-onset (black) gratings. In this and subsequent figures, error bars denote SEs across the subjects. Significant (p < .05) differences between the two waveforms are observed at 84 consecutive time points from 41 to 124 msec (denoted by *), indicating that the subjective-onset grating induced no M1 responses in Area I. (B) Comparison of the subjective-onset waveforms between Areas I and II. VEF in Area II shows significant (p < .05 at five consecutive time points) responses to the subjective-onset grating at 126 msec (green arrow), which is earlier by approximately 100 msec than VEFs in Area I (black arrow at 225 msec). (C) Contour maps of MEG responses to the subjective-onset grating. The differential VEFs (Δ amplitude) at each sensor position are color-coded over a topographical layout of all MEG sensors. The first neuromagnetic responses to the subjective-onset grating emerged from anterolateral regions (140 msec), were fed back to the occipital pole (230 msec), and then diffused into broad regions over the brain (350 msec).

Figure 4. 

Temporal dynamics of the neural activity to the subjective-onset grating. All the waveforms were differential VEFs in which the across-sensor VEFs in the no-stimulus trials were subtracted from those in the physical- or subjective-onset trials. (A) Comparison of VEFs in Areas I between the physical- (red) and subjective-onset (black) gratings. In this and subsequent figures, error bars denote SEs across the subjects. Significant (p < .05) differences between the two waveforms are observed at 84 consecutive time points from 41 to 124 msec (denoted by *), indicating that the subjective-onset grating induced no M1 responses in Area I. (B) Comparison of the subjective-onset waveforms between Areas I and II. VEF in Area II shows significant (p < .05 at five consecutive time points) responses to the subjective-onset grating at 126 msec (green arrow), which is earlier by approximately 100 msec than VEFs in Area I (black arrow at 225 msec). (C) Contour maps of MEG responses to the subjective-onset grating. The differential VEFs (Δ amplitude) at each sensor position are color-coded over a topographical layout of all MEG sensors. The first neuromagnetic responses to the subjective-onset grating emerged from anterolateral regions (140 msec), were fed back to the occipital pole (230 msec), and then diffused into broad regions over the brain (350 msec).

Figure 5. 

The areal mean approach covering six areas along the ventrolateral regions. (A) Locations of the six areas. Each area comprised 12 gradiometers along the ventrolateral region of each hemisphere, with the overlaps of six gradiometers between contiguous areas. Note that Areas I and II in Figure 2A correspond to Areas 1 (black) and 4 (green) in this figure, respectively. (B) Across-sensor VEFs in the six areas (Areas 1–6) in the ventrolateral pathway. As in Figure 4, all the waveforms are differential VEFs from the no-stimulus trials. In the contralateral subjective-onset trials (left), the first MEG component at approximately 100 msec is selectively observed in anterior regions (Areas 3–6). This is followed by a later component (delayed activity, 200–400 msec) diffusing into all six areas.

Figure 5. 

The areal mean approach covering six areas along the ventrolateral regions. (A) Locations of the six areas. Each area comprised 12 gradiometers along the ventrolateral region of each hemisphere, with the overlaps of six gradiometers between contiguous areas. Note that Areas I and II in Figure 2A correspond to Areas 1 (black) and 4 (green) in this figure, respectively. (B) Across-sensor VEFs in the six areas (Areas 1–6) in the ventrolateral pathway. As in Figure 4, all the waveforms are differential VEFs from the no-stimulus trials. In the contralateral subjective-onset trials (left), the first MEG component at approximately 100 msec is selectively observed in anterior regions (Areas 3–6). This is followed by a later component (delayed activity, 200–400 msec) diffusing into all six areas.

Figure 6. 

Distributed source estimations of neuromagnetic data. (A) Comparison of source estimations of MEG signals between the physical-onset and no-stimulus trials (time window, 0–200 msec). Upper and lower horizontal lines in the middle (sagittal) slice (x = 10) indicate positions of axial slices in the left (z = 0) and right (z = −10) panels, respectively. Neural responses selective for the physical-onset trials are observed in the early and higher visual regions as well as the frontal regions. (B) Comparison of source estimations between the subjective-onset and no-stimulus trials in the time window of 0–200 msec. In comparison with the physical-onset trials in A, neuromagnetic responses to the subjective-onset grating are diminished in the lower visual region, whereas the higher visual region in the lateral and ventral regions remains active, which suggests a dominant role for the higher region in perception of the subjective-onset grating.

Figure 6. 

Distributed source estimations of neuromagnetic data. (A) Comparison of source estimations of MEG signals between the physical-onset and no-stimulus trials (time window, 0–200 msec). Upper and lower horizontal lines in the middle (sagittal) slice (x = 10) indicate positions of axial slices in the left (z = 0) and right (z = −10) panels, respectively. Neural responses selective for the physical-onset trials are observed in the early and higher visual regions as well as the frontal regions. (B) Comparison of source estimations between the subjective-onset and no-stimulus trials in the time window of 0–200 msec. In comparison with the physical-onset trials in A, neuromagnetic responses to the subjective-onset grating are diminished in the lower visual region, whereas the higher visual region in the lateral and ventral regions remains active, which suggests a dominant role for the higher region in perception of the subjective-onset grating.

Figure 7. 

Experiment 1b to examine for residual visibility of the grating during the flash period. (A) Structure of one trial. After the continuous flashes, participants performed three tasks: (i) detection of fixation change occurring in catch trials, (ii) a yes/no judgment of the grating in the flash period, and (iii) direction discrimination of the grating (vertical or horizontal). Three types of trials (no-stimulus, low-grating, and high-grating) were randomly intermixed with catch trials. Grating in the low-grating trials were identical to those for the MEG measurements in Experiment 1a. (B) Whereas the detection rate of the fixation change (Task 1) was nearly perfect, the detection rate for the low-acuity grating in Task 2 was less than 2%, not significantly different from the false alarm rate in the no-stimulus trials. The accuracy of the direction discrimination (Task 3) was also close to the chance level (50%) in the low-grating trials, indicating that perception of the low-acuity grating was totally suppressed during the flash period.

Figure 7. 

Experiment 1b to examine for residual visibility of the grating during the flash period. (A) Structure of one trial. After the continuous flashes, participants performed three tasks: (i) detection of fixation change occurring in catch trials, (ii) a yes/no judgment of the grating in the flash period, and (iii) direction discrimination of the grating (vertical or horizontal). Three types of trials (no-stimulus, low-grating, and high-grating) were randomly intermixed with catch trials. Grating in the low-grating trials were identical to those for the MEG measurements in Experiment 1a. (B) Whereas the detection rate of the fixation change (Task 1) was nearly perfect, the detection rate for the low-acuity grating in Task 2 was less than 2%, not significantly different from the false alarm rate in the no-stimulus trials. The accuracy of the direction discrimination (Task 3) was also close to the chance level (50%) in the low-grating trials, indicating that perception of the low-acuity grating was totally suppressed during the flash period.

Figure 8. 

Experiment 2a (temporal order judgment task). (A) Structure of the trials. Participants were presented with the subjective-onset grating at the left or right visual field, followed by the physical-onset grating at the other field. The perceptual latency (interval from the flash offset to the conscious detection of the grating) of the physical-onset grating was manipulated by inserting a lag screen after the flash offset. Participants answered which of the two gratings appeared first (left or right) while monitoring a possible fixation change during the flash period. The mean (±SE across five subjects) detection rate of the fixation change was 95.8 ± 2.6%. (B) The percentage that participants reported the subjective-onset grating as the first one (% S-first) as a function of the duration of the lag period. A point of subjective equality was 208 ± 24 msec (mean ± SE), which indicated that the perceptual latency of the subjective-onset grating was comparable to that of the physical-onset grating presented at 200 msec after the flash offset.

Figure 8. 

Experiment 2a (temporal order judgment task). (A) Structure of the trials. Participants were presented with the subjective-onset grating at the left or right visual field, followed by the physical-onset grating at the other field. The perceptual latency (interval from the flash offset to the conscious detection of the grating) of the physical-onset grating was manipulated by inserting a lag screen after the flash offset. Participants answered which of the two gratings appeared first (left or right) while monitoring a possible fixation change during the flash period. The mean (±SE across five subjects) detection rate of the fixation change was 95.8 ± 2.6%. (B) The percentage that participants reported the subjective-onset grating as the first one (% S-first) as a function of the duration of the lag period. A point of subjective equality was 208 ± 24 msec (mean ± SE), which indicated that the perceptual latency of the subjective-onset grating was comparable to that of the physical-onset grating presented at 200 msec after the flash offset.

Figure 9. 

Results of Experiment 2b (MEG experiment). Across-sensor VEFs in the physical-onset (A) and subjective-onset (B) trials. Color conventions are identical to those in Figure 3A and B. Note that in the physical-onset trials, a blank screen of 200 msec was introduced between the disappearance of the continuous flash (at 0 msec) and the appearance of the physical-onset grating (at 200 msec). (C) Differential VEFs of the physical-onset minus no-stimulus trials. (D) Differential VEFs of subjective-onset minus no-stimulus trials. Color conventions are identical to those in Figure 4B. The first significant response related to a perception of the grating is observed in Area I in the physical-onset trials (black arrow) but Area II in the subjective-onset trials (green arrow).

Figure 9. 

Results of Experiment 2b (MEG experiment). Across-sensor VEFs in the physical-onset (A) and subjective-onset (B) trials. Color conventions are identical to those in Figure 3A and B. Note that in the physical-onset trials, a blank screen of 200 msec was introduced between the disappearance of the continuous flash (at 0 msec) and the appearance of the physical-onset grating (at 200 msec). (C) Differential VEFs of the physical-onset minus no-stimulus trials. (D) Differential VEFs of subjective-onset minus no-stimulus trials. Color conventions are identical to those in Figure 4B. The first significant response related to a perception of the grating is observed in Area I in the physical-onset trials (black arrow) but Area II in the subjective-onset trials (green arrow).

METHODS

Participants

We conducted four experiments (Experiments 1a, 1b, 2a, and 2b). In Experiments 1a and 2b, we measured neural responses of participants using MEG, whereas Experiments 1b and 2a were behavioral studies to supplement the MEG data. The number of participants were 12 (Experiment 1a), 10 (Experiment 1b), 5 (Experiment 2a), and 11 (Experiment 2b). Four participants participated in both MEG experiments (Experiments 1a and 2b). Informed consent was obtained from each participant after the study had been explained. Approval for these experiments was obtained from the ethics committee of the National Institute for Physiological Sciences, Okazaki, Japan.

Experiment 1a (MEG Study)

All visual stimuli were generated using the Matlab Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and presented on a screen at a refresh rate of 60 Hz. For dichoptic viewing, we presented stimuli at two different locations on the screen. A square area (delineated by white lines, 6.1° × 6.1°) on the left half of the screen comprised stimuli for the left eye of the participant, whereas another area on the right half comprised stimuli for the right eye. Stimuli at these two locations were fused using a mirror stereoscope placed in front of the participant's eyes.

Every trial began with a fixation of 1 sec (pre-flash period), followed by a rapid sequence of colored Mondrian patterns (continuous flashes) to the dominant eye. We determined each participant's eye dominance using a variation of the Porta test (Roth, Lora, & Heilman, 2002). In this test, participants extended one arm and aligned a forefinger vertically with a target object in the room, with both eyes open. Next, they alternately closed each eye, and the dominant eye was defined as the eye that caused a larger alignment change between the forefinger and target object when closed. Eleven of the 12 participants had dominant right eyes. The duration of each Mondrian pattern in the continuous flashes was 50 msec, and 21 different patterns were presented in one trial (flash period, 1.05 sec). After the continuous flashes, the screen returned to a fixation lasting for 1.8 sec (post-flash period). In contrast, stimuli to the nondominant eye differed in the three types of trials (Figure 1AC). In no-stimulus trials, only a fixation point was shown throughout the trial. In physical-onset trials, the white fixation was shown during the flash period but was followed by a grayscale grating (vertical or horizontal; size, 2.54° × 2.54°; spatial frequency, 2.54°/cycle) presented at the beginning of the post-flash period. The location of the grating was randomly selected from the four corners of the square field. In subjective-onset trials, the grating appeared synchronously with the onset of the continuous flashes, remaining on the screen until the end of the trial. The visibility of the grating during the flash period was examined later in a separate experiment (see Experiment 1b below).

Over the course of an experimental session, three types of trials above were randomly intermixed with two additional types of trials. The first type was “high-acuity subjective-onset” trials in which a grating with a higher luminance contrast and higher spatial frequency (0.38°/cycle) than that in subjective-onset trials was presented in the flash and post-flash periods. Because of a high visual acuity for the grating, the participants sometimes perceived it consciously even during the flash period, yielding trials with incomplete binocular suppression. This condition was necessary in a later experiment (see Experiment 1b) and was thus introduced here to equalize protocols and data quality across experiments. The second type was “catch” trials involving a brief change in the fixation point during the flash period. Although the fixation point over the continuous flashes was normally small (0.15°) and white, it turned into a black point of larger size (0.46°) in catch trials. This change in the fixation point lasted 300 msec, with its timing randomly selected within the flash period of 1.05 sec. No grating was presented in the flash or post-flash periods of catch trials.

The participants were asked to perform two tasks in every trial. During the flash period, their task was to detect a possible fixation change (in catch trials). After the flash disappeared, their second task was to judge the orientation of the grating. For 6 of the 12 participants, the vertical grating was presented as a target stimulus and the horizontal grating was shown as a nontarget stimulus. They pressed a button when they perceived the target stimulus at any location in the post-flash period. For the remaining participants, the horizontal grating was presented as the target stimulus, and the participants did not respond to the nontarget stimulus (vertical grating). The ratio of target/nontarget trials was 1:8 in both physical- and subjective-onset trials, and we only analyzed the MEG data from nontarget trials to avoid any influence of motor-related neural activity caused by the manual movement of the button presses. In brief, the participants detected the fixation change during the flash period and responded to the grating with the target orientation during the post-flash period (if neither event occurred, the button was not to be pressed). This dual-task procedure guided the attention of the participants to the central visual field in the flash period and to the peripheral visual fields in the post-flash period, which strongly suppressed conscious perception of the low-acuity grating during the continuous flashes (see Experiment 1b and Figure 7).

One experimental session consisted of 84 trials (12 no-stimulus trials, 24 physical-onset trials, 24 subjective-onset trials, 12 high-acuity subjective-onset trials, 6 target trials, and 6 catch trials). Physical- and subjective-onset trials were further subdivided into two types in which the nontarget grating was presented to the left and right visual fields. In the left visual field trials, the grating appeared at the upper or lower left corner of the square area, whereas in the right visual field trials, it appeared at the upper or lower right corner. A total of eight types of trials were randomly intermixed, and the participants performed six sessions per experiment.

MEG Recordings and Data Analysis

Neural activity during flash and post-flash periods was measured using a whole-head 306-channel MEG system (Vector-view, ELEKTA Neuromag, Helsinki, Finland), which recorded changes in neuromagnetic signals from 102 different positions over the scalp. A sensor element in each recording position consisted of one magnetometer and two orthogonal planar-type gradiometers (one for latitudinal and the other for longitudinal directions of changes in neuromagnetic signals). Thus, we analyzed MEG signals recorded from 204 gradiometers at 102 sensor positions. The signals from the planar-type gradiometers are strongest when the sensors are located just above local cerebral sources (Nishitani & Hari, 2002). The measurements were performed with a band-pass filter of 0.1–200 Hz at a sampling rate of 1000 Hz. Neural responses to the physical- and subjective-onsets of the nontarget gratings were investigated by recording visual-evoked fields (VEFs) time-locked to the beginning of the post-flash period. For each of the eight conditions above, neuromagnetic waveforms in 72 trials at maximum were averaged, with an averaging epoch of −1100 to 1000 msec relative to the offset of the continuous flashes. Trials in which signal variations were larger than 3000 fT/cm were excluded from analysis.

Because the primary purpose of the study was to discern whether the first neural activity related to conscious perception emerged from the lower or higher visual region of the ventral pathway, we compared VEFs from these two regions. This comparison was made by applying the areal mean approach to the MEG data used in previous studies (Altmann et al., 2008; Tarkiainen, Helenius, & Salmelin, 2003). In this approach, VEFs at a subset of 102 positions were defined as a group of interest (see Areas I and II in Figure 2A and Areas 1–6 in Figure 5A for examples of these groups), and a collective VEF (across-sensor VEF) over each group was calculated as
formula
where x and y indicated MEG signals (fT/cm) in the latitudinal (x) and longitudinal (y) gradiometers at the sensor position i and N represented the number of sensor positions belonging to the group of interest. In other words, we first combined the signal from the two orthogonal gradiometers at each of the 102 sensor positions and then averaged across N sensor positions within the same group.

We selected 12 gradiometers (at six sensor positions) around the occipital pole as a group for the early visual cortex (Area I, Figure 2A), whereas another group representing neural signals from the higher visual region in the ventral pathway consisted of the same number of sensors over lateral (ventral) regions of the scalp (Area II). Locations of these two areas were based on our previous MEG studies (Noguchi, Inui, & Kakigi, 2004). Areas I and II were defined for each hemisphere, resulting in a total of four areas (Areas I and II × left and right hemispheres) over the entire brain.

After obtaining the across-sensor VEF for each condition of each area, we averaged the waveforms between the two hemispheres (Figure 3A and B). This between-hemisphere average was based on the stimulus–hemisphere correspondence (contralateral or ipsilateral). Thus, the across-sensor VEF of the right visual field trials in the left hemisphere (either physical- or subjective-onset) was averaged with that of the left visual field trials in the right hemisphere (contralateral condition). We then corrected for differences in baseline signals of VEFs across conditions. First, a baseline signal in each condition was calculated using VEF amplitudes from −10 to 0 msec, relative to the flash offset. On the basis of the differences in baseline signals across conditions, each VEF in the physical- and subjective-onset trials was corrected such that it shared the same baseline (−10 to 0 msec) as that in the no-stimulus trial. Grand-averaged data across the 12 participants were calculated after all procedures above were applied to the individual data.

Finally, we obtained neuromagnetic responses related to the perception of the physical- and subjective-onset gratings (a grating-related activity) by subtracting across-hemisphere VEFs among trials (Figure 4). For example, the activity related to the perception of the physical-onset grating was acquired as the differential VEF between physical-onset and no-stimulus trials. The significance of the grating-related activity was evaluated by a one-group t test applied for each time point of the differential VEF. The difference was considered significant only when p values from those one-group t tests were less than 0.05 at five consecutive time points.

Distributed Source Estimations of Neuromagnetic Signals

In addition to the areal mean and current dipole approaches (Nishitani & Hari, 2002), we estimated source locations of MEG signals using Statistical Parametric Mapping 5 (SPM5, available on-line at www.fil.ion.ucl.ac.uk/spm/). After VEFs of two gradiometers at each sensor position were combined as described above, the waveforms were low-pass-filtered at 30 Hz and down-sampled to 200 Hz. Three-dimensional source reconstructions of MEG signals were performed by the multiple sparse priors (MSP) approach (Friston et al., 2008) of SPM5. First, a tessellated cortical mesh was created for each participant using a template brain from the Montreal Neurological Institute. This mesh contained 7200 vertices and served as a brain model to estimate current source distributions. Positions of current dipoles were restricted to a cortical surface, and they were evenly placed at each node of the mesh. Second, this dipole mesh was spatially coregistered with a sensor space of MEG, based on the digitized locations of the nasion and bilateral peri-auricular points of each participant(Henson et al., 2007). We then constructed a single-shell, spherical head model for forward solutions, as implemented in BrainStorm (available on-line at neuroimage.usc.edu/brainstorm). Finally, the forward model was inverted with the MSP approach, mapping the contribution (source strength) of each dipole on the mesh to MEG sensor waveforms. We performed these MSP estimations for each time point, trial, and subject.

For second-level (group-level) analysis across the 12 participants, the MSP solutions for each trial and each participant were averaged over all time points belonging to a time window of interest (0–200 msec in Figure 6A and B). The resultant MSP averages were then analyzed by SPM random-effect analysis (Henson et al., 2007). Differences between trials (no-stimulus vs. physical-onset trials in Figure 6A) were evaluated by voxel-wise t tests of SPM5. The threshold for a significant difference was set at a p value of less than .005, false discovery rate-corrected for multiple comparisons.

Experiment 1b (Behavioral Experiment)

In Experiment 1b, we examined whether the low-acuity grating (in Experiment 1a) remained invisible during the flash period of subjective-onset trials. As shown in Figure 7A, every trial in Experiment 1b consisted of pre-flash and flash periods. The fixation point without a grating was presented to the nondominant eye of the participants in no-stimulus trials, whereas the low-acuity grating was presented in low-grating trials, as in Experiment 1a. The trials with high-acuity grating (high-grating trials, corresponding to high-acuity subjective-onset trials in Experiment 1a) were also employed to ascertain and maintain the participants' motivation for the tasks (Sterzer, Jalkanen, et al., 2009). After the flashes disappeared, the participants completed three tasks sequentially. In the first task, they answered whether the fixation in the flash period underwent changes (forced choice), as in a previous experiment. In the second task (detection task), they were asked to make yes/no judgments of the grating during the flash periods as an indicator for conscious perception of the grating. They answered “yes” when they perceived any type of grating at any location. In the last task (discrimination task), they described the orientation of the grating (vertical or horizontal) during the continuous flashes. This question was asked even if they had answered “no” in the second task to examine the possibility of unconscious (subliminal) processing of “invisible” stimuli. One experimental session included 96 trials (32 no-stimulus trials, 32 low-grating trials, 16 high-grating trials, and 16 catch trials). The participants performed one or two sessions per experiment.

Experiment 2a (Behavioral Experiment)

Although we measured VEFs to the physical- and subjective-onset gratings in Experiment 1, our preliminary experiment, in which behavioral detection times for those two types of gratings were compared (data not shown), indicated that the participants generally detected the physical-onset grating faster than the subjective-onset grating. This suggested that perceptual latency of the grating (intervals from the flash offset to the time when participants perceived the onset of the grating) were not comparable between physical- and subjective-onset trials in Experiment 1, which may affect interpretations of our MEG data. Therefore, we measured VEFs in physical- and subjective-onset trials again, with perceptual sequences between the two types of trials closely matched (Experiment 2). For this purpose, we first estimated the difference in the perceptual latency between the physical- and subjective-onset gratings (Experiment 2a) by the temporal order judgment task (McDonald, Teder-Salejarvi, Di Russo, & Hillyard, 2005; Stelmach & Herdman, 1991).

In every trial of Experiment 2a, we presented two (physical- and subjective-onset) gratings, one at the left and another at the right visual fields (Figure 8A). When the flashes disappeared, the subjective-onset grating remained on the screen for a variable duration (the lag period, variable from 100 to 400 msec in steps of 50 msec), followed by the physical-onset grating presented at the other visual field. These gratings appeared at the upper half of the square field for three participants and at the lower field for the remaining two participants. As in previous experiments, the participants monitored a possible fixation change (in catch trials) during the flash period. If no fixation change was detected, they moved to the second (temporal order judgment) task after the flash disappeared, reporting which of the two gratings appeared first (left or right, forced choice). For each lag period, we measured the percentage that participants reported the subjective-onset grating as the first one (% subjective-first). These changes in % subjective-first as a function of lag duration were then fitted with a psychometric sigmoid function. The difference in the perceptual latency between the physical- and subjective-onset gratings was estimated on the basis of a point of subjective equality, a crossing point of the psychometric curve with a 50% threshold. One experimental session consisted of 74 trials (70 temporal order judgment trials and 4 catch trials, randomly intermixed). In half of the temporal order judgment trials, the subjective-onset grating was presented at the left visual field, whereas in the other half, they were presented at the right visual field. Each participant performed three sessions.

Experiment 2b (MEG Experiment)

On the basis of the results of Experiment 2a, we again recorded VEFs in physical-onset, subjective-onset, and no-stimulus trials. All procedures were identical to those in Experiment 1a (MEG experiment), except that a blank period of 200 msec was introduced between the flash offset and presentation of the physical-onset grating, such that perceptual sequences of physical-onset trials were equalized with those of subjective-onset trials.

RESULTS

Experiment 1a (MEG Experiment)

Mean (±SE across subjects) detection rates of fixation change and the target grating were 89.1% (±3.3%) and 93.9% (±1.9%), respectively. Figure 2A showed VEFs in no-stimulus trials recorded by 204 gradiometers of a whole-head MEG, with time 0 corresponding to the flash offset. Clear MEG responses were observed primarily in the occipital and lateral (ventral) regions in both hemispheres, which was consistent with our previous study (Noguchi et al., 2004). Note that strong neural responses measured through the MEG gradiometers are represented either as positive or negative deflections of neuromagnetic waveforms. Figure 2B shows enlarged VEFs at three gradiometers (two in the posterior and one in the lateral regions) in which the waveforms in no-stimulus, physical-onset, and subjective-onset trials are superimposed. Physical- and subjective-onset trials were further subdivided into trials with the grating presented at the left and right visual fields. Because we averaged the waveforms from −1100 to 1000 msec after the flash offset, the MEG responses in the first half (−1000 to 0 msec) were caused by onset of the continuous flashes, whereas those in the second half (0–1000 msec) primarily indicated neural activity in response to the onset of the grating. At all three gradiometers in Figure 2B, the MEG responses related to the perception of the grating can be observed as neuromagnetic components selective for physical- and subjective-onset trials compared with no-stimulus (baseline) trials. These responses to the grating are clearly seen specifically when the grating was presented at the visual fields contralateral to the sensor positions (arrows in Figure 2B).

Collective Neuromagnetic Signals in Lower and Higher Visual Regions

To discern whether neural activity related to conscious perception emerged from the lower or higher visual region in the ventral pathway, we compared VEFs in these two regions. This comparison was made using the areal mean approach (Altmann et al., 2008; Tarkiainen et al., 2003) in which a collective (across-sensor) VEF over a subset (group) of gradiometers was calculated (see Methods). We selected 12 gradiometers around the occipital pole as a group for the early visual cortex (Area I, Figure 2A) and another group representing the signals from the higher visual region consisted of the same number of gradiometers over lateral (ventral) regions of the scalp (Area II). Figure 3A and B show these across-sensor VEFs in physical- and subjective-onset trials, with the data from the two hemispheres averaged. The red lines represent the waveforms in response to the contralateral condition, whereas blue lines represent the waveforms in response to the ipsilateral grating. The data in no-stimulus trials are also plotted (black) as baselines. Note that in these across-sensor VEFs, strong MEG responses are shown as upward deflections of the waveforms.

In all three (physical-onset, subjective-onset, and no-stimulus) trials, the onset of the continuous flashes (at −1050 msec) induced strong neuromagnetic responses approximately −1000 to 700 msec (the flash responses; Figure 3A and B, left). These responses primarily consisted of two peaks, one at approximately 80–100 msec and another at 150–200 msec after the flash onset, which was consistent with previous studies of MEG and electroencephalography (EEG; Liu, Harris, & Kanwisher, 2002; Noesselt et al., 2002; Hillyard & Anllo-Vento, 1998). Analysis using current dipoles indicated that the initial MEG component approximately 80–100 msec (M1) emerged from the early visual cortex (Figure 3C), whereas anatomical sources for the 150–200 msec component (M2) were located in the ventrolateral regions in both hemispheres, presumably corresponding to the higher visual regions in the ventral pathway. In fact, amplitudes of the M1 component were larger than those of the M2 component in Area I (Figure 3A and B, top left), indicating that the waveforms in this area primarily represented signals from the early visual cortex. In contrast, this relationship was reversed in Area II, in which the amplitudes of M2 exceeded those of M1 (Figure 3A and B, bottom left).

Grating-related Activity of Physical- and Subjective-onset Trials

Neuromagnetic responses to the grating are enlarged in Figure 3A and B (right). The waveforms in physical-onset trials (Figure 3A, right) showed basically the same patterns as the responses to the continuous flashes described above. The M1 and M2 components were observed to be dominant in Area I (Figure 3A, top right) and Area II (Figure 3A, bottom right), respectively. In contrast, the waveforms in subjective-onset trials (Figure 3B, right) were totally different from those in physical-onset trials. First, the subjective-onset waveforms showed no significant M1 component in Area I (Figure 3B, top right). In the contralateral and ipsilateral gratings, no distinct MEG responses to the grating were induced until after 200 msec, which presumably indicated the lack of a physical presentation of the grating after the flashes in subjective-onset trials. Second, despite the lack of an early MEG component in Area I, substantial neural activity in Area II was observed in the contralateral subjective-onset trial, which began as early as 110–130 msec after the flashes disappeared (Figure 3B, bottom right).

These characteristics of the subjective-onset trials are summarized in Figure 4. In this figure, we show differential waveforms in which the across-sensor VEFs in no-stimulus trials were subtracted from those in physical- or subjective-onset trials. In Area I, initial neuromagnetic signals from the early visual cortex (M1) were not observed in subjective-onset trials, although both the physical- and subjective-onset gratings produced delayed activity at a later period of 250–400 msec (Figure 4A). Figure 4B compares the differential waveforms of subjective-onset trials between Areas I and II. Point-by-point one-group t tests indicated that significant (p < .05 at five consecutive points, relative to the no-stimulus trials) activity related to perception of the subjective-onset grating was initially observed in Area II at 126 msec (green arrow), followed by delayed response in Area I at 225 msec after the flash offset (black arrow). The temporal dynamics of the neural activity in the subjective-onset trials are shown topographically in the contour maps depicted over layouts of all MEG sensors (Figure 4C). These maps indicated that the first neural response to the subjective-onset grating was detected by MEG sensors in anterior-lateral regions (140 msec). This activity was fed back to the posterior regions around the early visual cortex (230 msec) and then propagated to the entire brain, especially to lateral regions in both hemispheres (350 msec).

Comparisons of VEFs between Anterior and Posterior Halves of the Ventral Pathway

To investigate VEFs at a broader region in the ventral pathway, we defined six areas (Areas 1–6) along the ventrolateral regions of both hemispheres, obtaining the across-sensor VEF for each of these six areas (see Figure 5A for locations of these areas). The resultant waveforms in the subjective-onset trials are shown in Figure 5B. When the subjective-onset grating was perceived, the first MEG responses (100–150 msec) were observed in regions covering the anterior half (Areas 3–6) of the contralateral hemisphere. These responses were followed by delayed activity propagating into all six areas. The waveforms in the ipsilateral condition (Figure 5B, right) indicated that the initial MEG components in Areas 3–6 (100–150 msec) were selective for the contralateral condition, which eliminated the possibility that those components in the anterior regions indicated signals caused by eye blinking and microsaccades.

Source Estimation Analysis of Neuromagnetic Data

We finally performed distributed source estimation of MEG data using SPM5 (see Methods). The second-level (group-level) comparison of MSP estimations between physical-onset and no-stimulus trials (time window, 0–200 msec) revealed significant neuromagnetic responses in early and ventral visual cortices as well as several areas in the frontal lobe (Figure 6A), indicating neural activity elicited by presentation of the physical-onset grating. In contrast, no significant responses were observed in the early visual cortex (around the occipital pole) in comparison of the subjective-onset and no-stimulus trials, although this comparison showed significant activities in the ventral pathway and frontal areas (Figure 6B). These results indicated a major role of the higher visual region rather than the early visual region in creating conscious representations of the subjective-onset grating.

Another difference in the MSP results between the physical- and subjective-onset trials was the lack of activity in the lateral frontal regions in the subjective-onset trials. This may be related to downward deflections in VEF waveforms in the initial 100 msec after the flash offset (see differential VEFs at Areas 5 and 6 in Figure 5B, left). Although reasons for these negative responses in the anterior regions were unclear (and we confirmed that these downward deflections were not statistically significant, p > .05, one-group t test), such negative VEFs specific to subjective-onset trials may result in the lack of neural responses in the lateral frontal regions in Figure 6B.

Experiment 1b (Behavioral Experiment)

In Experiment 1b, we examined whether the low-acuity grating (in Experiment 1a) had remained invisible during the flash period of the subjective-onset trials. The results of the three tasks in Experiment 1b are shown in Figure 7. In Task 1, the detection rate of the fixation change was 98.4 ± 0.8%, with a false alarm rate of 0.2 ± 0.2% (mean ± SE across subjects). In contrast, detection (hit) rates of the grating in Task 2 were 1.3 ± 0.7% in low-grating and 38.8 ± 15.4% in high-grating trials, with a false alarm rate of 0.2 ± 0.2% in no-stimulus trials. No difference was observed between the false alarm rate and hit rate in the low-grating trials [t(9) = 1.48, p = .17], indicating that the low-acuity grating during the flashes was subjectively invisible. Finally, accuracies in Task 3 (discrimination of the direction of the grating) were 49.1 ± 1.7% in low-grating and 65.9 ± 8.4% in high-grating trials. The accuracy in the low-grating trials was not significantly different from the chance level [t(9) = 0.54, p = .60], which negated the possibility of unconscious processing of the grating in the flash period. Overall, the results in Experiment 1b showed that the low-acuity grating during the continuous flashes remained invisible both subjectively and objectively.

Experiment 2a (Temporal Order Judgment Task)

One of the purposes of Experiment 2 was to record VEFs again, with the perceptual sequences in the physical- and subjective-onset trials closely matched. For this purpose, we first estimated the difference in the perceptual latency of the physical- and subjective-onset gratings using the temporal order judgment task. As shown in Figure 8B, the point of subjective equality of the psychometric function was 208 ± 24 msec (mean ± SE across subjects), indicating that the perceptual latency of the subjective-onset grating was equal to that of the physical-onset grating presented at 208 msec after the flash offset. Thus, in the next experiment (Experiment 2b), we introduced a blank period of 200 msec between the flash offset and the presentation of the physical-onset grating, thereby equalizing the perceptual sequences between the two types of trials.

Experiment 2b (MEG Experiment)

In Experiment 2b, we measured VEFs again with perceptual latency of the physical- and subjective-onset gratings matched. Mean (± SE across subjects) detection rates of the fixation change and the target grating were 89.6% (±3.4%) and 97.0% (±0.9%), respectively. The across-sensor VEFs in Areas I and II are shown in Figure 9. In physical-onset trials (Figure 9A), neuromagnetic responses to the grating were clearly observed from 200 to 700 msec after the flash offset. These grating-related responses were more distinct than those in Experiment 1a (Figure 3A), probably because the introduction of the 200-msec blank after the flash period restored an excitability of visual neurons (that had been lowered by the continuous flashes of 1.05 sec). Although these augmented responses to the physical-onset grating in Experiment 2b made the distinction of M1 and M2 less clear, the temporal order of neural responses between Areas I and II was the same as that in Experiment 1a. As shown in differential waveforms of the across-sensor VEFs (Figure 9C, the physical-onset minus the no-stimulus trials), the first significant activity elicited by the grating was observed in Area I (at 307 msec, p < .05, at five consecutive time points, one-group t test), followed by that in Area II at 323 msec (note that the physical-onset grating was presented at 200 msec in this experiment). The across-sensor VEFs in the subjective-onset trials (Figure 9B) also resembled those in Experiment 1a (Figure 3B). Significant responses related to conscious perception of the grating were first seen in Area II (97 msec, green arrow in Figure 9D), not in Area I. These results thus confirmed a major role of the higher visual region rather than the lower visual region in conscious perception of the subjective-onset grating.

Similarity of Neural Waveforms between Physical- and Subjective-onset Trials

As shown in Figure 9, neuromagnetic responses in the physical-onset trials were substantially different from those in the subjective-onset trials. However, given that we equalized perceptual sequences for these two types of trials in Experiment 2b, one could expect some similarities in MEG waveforms between them (because the conscious percept was the same, the neural correlates should also be the same). Thus, we examined this similarity by performing correlation analysis of MEG waveforms between the physical- and subjective-onset trials. Because our preliminary experiment indicated that the detection time for the subjective-onset grating was about 680 msec, we performed correlation analysis using differential waveforms from 0 to 700 msec after the offset of the continuous flashes. Mean (±SE across 11 subjects) correlation coefficients between the waveforms in the physical- and subjective-onset trials (black and green lines in Figure 9C and D) were 0.33 (±0.09) in Area I and 0.32 (±0.09) in Area II. No significant difference in correlation coefficients was observed between the two areas (t = 0.16, p = .88). We then narrowed the time window of correlation analysis from 0–700 msec to 200–500 msec because our data indicated a close relationship between delayed activity observed in this time period and consciousness (Figures 4B and 9B). The resultant correlations of waveforms between the physical- and subjective-onset trials were higher in Area II (0.29 ± 0.12) than in Area I (0.16 ± 0.16), although this difference did not reach significance (t = 1.15, p = .28). Overall, our data showed that there were some correlations in MEG waveforms between the two types of trials and that these correlations were comparable or somewhat higher in the later visual regions (Area II) than the early visual regions (Area I).

Neural Responses to an Invisible Grating during the Flash Period

In Experiments 1a and 2b, we found two MEG components related to the perception of the subjective-onset grating: the initial response in the higher visual regions (around 100 msec after the flash offset) and delayed activity after 200 msec (Figures 5 and 9). An important and unanswered question was “which component of these MEG signals (initial or delayed one) was directly associated with consciousness?” One possibility was that a conscious percept of the grating was generated as a result of the initial activity in the higher visual areas. Another possibility, however, was that the conscious percept did not emerge until delayed activity was diffused into the entire brain. In the latter case, the first neural activity in the higher regions would be necessary but not sufficient for the emergence of consciousness.

To investigate this, we performed additional analysis of MEG signals during the flash period (rather than the post-flash period) using the data from 19 subjects who participated in either Experiment 1a or 2b. In the flash period of the subjective-onset trials, the grating was continuously presented to the nondominant eye of the participants, although no grating was given in the physical-onset and no-stimulus trials. Comparing the neuromagnetic signals in the subjective-onset trials with those in the other two types of trials, we should observe MEG responses related to the processing of “unseen grating” during the flash period. The results (differential VEFs in Areas I and II) are shown in Figure 10. We found significant (p < .05 for five consecutive time points) responses to the unseen grating in Area I (lower regions) and Area II (higher regions) in the first 200 msec. In contrast, no significant activity was observed after 200–400 msec, indicating that delayed activity in response to the grating was inhibited during the flash period. These results showed that the initial response in the higher visual regions (Area II) was evoked even when stimuli remained invisible to the participants. VEFs in Figure 10 thus suggested a close relationship between delayed activity, rather than the initial activity, and consciousness.

Figure 10. 

Neuromagnetic responses to an invisible grating during the flash period. (A) Differential VEFs. VEFs averaged across the no-stimulus and physical-onset trials were subtracted from those in the subjective-onset trials. The 0 msec in the horizontal axis denotes an onset (not offset) of the continuous flashes. Although the subjective-onset grating during the flash period remained invisible, they induced substantial responses in Areas I and II in the initial 200 msec. In contrast, delayed activity, which should be observed in 200–400 msec (blue-shaded interval) in both areas, was not observed. These data indicate that delayed activity plays a critical role in emergence of a conscious percept of the grating. (B) Changes in p values of one-group t tests. For each time point, a one-group t test was applied to judge whether an amplitude of the differential VEFs in A was significantly larger (or smaller) than 0 fT/cm in the vertical axis. Although deflections of differential VEFs were significant (p < .05 for five consecutive points) in the first 200 msec, delayed activity (200–400 msec) did not reach significance.

Figure 10. 

Neuromagnetic responses to an invisible grating during the flash period. (A) Differential VEFs. VEFs averaged across the no-stimulus and physical-onset trials were subtracted from those in the subjective-onset trials. The 0 msec in the horizontal axis denotes an onset (not offset) of the continuous flashes. Although the subjective-onset grating during the flash period remained invisible, they induced substantial responses in Areas I and II in the initial 200 msec. In contrast, delayed activity, which should be observed in 200–400 msec (blue-shaded interval) in both areas, was not observed. These data indicate that delayed activity plays a critical role in emergence of a conscious percept of the grating. (B) Changes in p values of one-group t tests. For each time point, a one-group t test was applied to judge whether an amplitude of the differential VEFs in A was significantly larger (or smaller) than 0 fT/cm in the vertical axis. Although deflections of differential VEFs were significant (p < .05 for five consecutive points) in the first 200 msec, delayed activity (200–400 msec) did not reach significance.

DISCUSSION

We investigated the temporal dynamics of neural activity when visual stimuli come into consciousness. Focus on the disinhibition phase of CFS enabled us to measure an activity at the moment of emergence of a conscious percept in the human brain. The areal mean VEFs (Figure 3A) in the physical-onset trials indicated that the first neuromagnetic responses (M1) were observed in the posterior regions (Area I), followed by the M2 response in the ventrolateral regions (Area II). Because our analysis using dipoles (Figure 3C) suggested that source locations for the M1 and M2 components were the lower and higher (ventral) visual regions, respectively, these results showed transmission of neural signals in a feedforward direction (from the lower to higher regions). In contrast, the areal mean VEFs in the subjective-onset trials (Figure 3B) revealed that the first neural responses to the grating were observed in Area II (not Area I), although this activity was diffused into both areas after 200 msec. These patterns of neural responses in the subjective-onset trials were further confirmed by contour map analysis (Figure 4C) and the expanded areal mean approach covering the six areas (Areas 1–6) along the ventral pathway (Figure 5). Overall, these results indicated transmission of neural signals from the higher to lower regions in the subjective-onset trials. Consistent with this view, source estimation analysis using the MSP approach (Figure 6) showed lack of activity in the lower visual areas at an initial 200 msec of the subjective-onset trials, suggesting a dominant role of the higher visual areas in conscious perception of these gratings. Finally, we replicated our results with the perceptual sequences in the physical-onset trials matched with those in the subjective-onset trials (Experiment 2, Figures 8 and 9).

Latency of the Consciousness-related Activity in the Ventral Pathway

Although several recent EEG and electrocorticography studies have investigated neural responses underlying visual awareness, there has been controversy about the latency of the first neural activity related to conscious perception. For example, a previous EEG study used a paradigm of attentional blink and compared EEG waveforms when participants reported the conscious perception of the target stimuli (“seen” trials) with those when participants did not report a conscious perception (“unseen” trials), concluding that neural responses more than 250 msec were critical for the target to access consciousness (Sergent, Baillet, & Dehaene, 2005). In contrast, an electrocorticography study used backward masking and found a significant difference in amplitudes of the N170 component between the “recognized” and “unrecognized” trials of target stimuli (Fisch et al., 2009). Finally, a study combining binocular rivalry with EEG measurements reported a neural correlate of visual awareness at a latency as early as 100 msec (P1 component; Roeber et al., 2008). One reason for this inconsistency may have been the use of nontarget visual stimuli to manipulate consciousness of the target stimuli. In attentional blink and backward masking, the mask stimuli are presented just before or after the target stimuli to render the target invisible (Fisch et al., 2009; Sergent et al., 2005). For measurements of the EEG waveforms time-locked to the perceptual alternations in the binocular rivalry, some changes in input stimuli (visual transient) were required to induce rapid alternations of conscious representations (Roeber et al., 2008). These stimulations to manipulate consciousness provoked strong bottom–up neural responses over broad regions of the visual cortex, resulting in a mixture of sensory responses with the neural components related to consciousness.

In contrast, in this study, emergence of conscious perception was triggered by the offset (not onset) of the continuous flashes. Compared with an onset of ordinary visual stimuli, the offset of these dynamic visual noises substantially inhibits the neuromagnetic responses in the visual cortex (Noguchi et al., 2004; Okusa, Kaneoke, Koyama, & Kakigi, 1998), thus providing a paradigm suitable for isolating consciousness-related activity. Our results consequently reveal that the first neural activity related to conscious perception of the subjective-onset grating emerges from the higher visual regions. Interestingly, the latency of this consciousness-related response was found to be short (approximately 100 msec after the offset of the continuous flashes; Figures 5 and 9), which may be consistent with a recent study reporting a temporal advantage of invisible stimuli for accessing consciousness (Wu, Busch, Fabre-Thorpe, & VanRullen, 2009).

Temporal Dynamics of the Neural Activity Underlying Conscious Perception

As described in the Introduction, there have been some inconsistencies between previous electrophysiological and fMRI data. Although studies using electrophysiological methods have indicated that conscious representations emerge from neural processing in the higher (not the lower) visual regions in the ventral pathway, fMRI studies have reported neural activity associated with changes in conscious percepts as early as in V1 and LGN. Our study provides data that are highly consistent with both groups of studies.

VEFs in the subjective-onset trials (Figures 4 and 5) show that the first activity related to conscious perception emerges from the anterior half of the lateral (ventral) regions in the contralateral hemisphere. These results support the view from previous electrophysiological data (Sheinberg & Logothetis, 1997; Leopold & Logothetis, 1996) indicating a major role for the ventral pathway (and the frontal cortex) in conscious perception (Crick & Koch, 2003). However, our results can also explain recent fMRI results reporting that activity in the early visual cortex correlated with changes in conscious percepts (Haynes et al., 2005; Lee et al., 2005; Wunderlich et al., 2005; Tong & Engel, 2001). Although the initial responses to the subjective-onset grating were observed in the anterior regions, these were followed by delayed activity diffusing into broad regions of the brain, including the lower visual regions (Figure 5B). Our results thus integrate previous “higher” (electrophysiological) and “lower” (fMRI) views on the emergence of consciousness, providing new perspectives on the temporal dynamics of conscious perception.

Neural Responses Directly Associated with Emergence of Consciousness

Although we identified two MEG components related to the perception of the subjective-onset grating (the initial response in the higher visual regions and delayed activity after 200 msec), our analysis in Figure 10 indicate a closer relationship of delayed rather than initial activity with consciousness. This association between consciousness and delayed activity is consistent with previous studies. A recent theoretical study argued that neural networks over broad regions in the brain play a critical role in emergence of conscious representations (Tononi & Koch, 2008). In line with this view, several neuroimaging studies reported that oscillatory brain rhythms synchronized across distant brain regions, such as those in theta and gamma bands, were strongly coupled with changes in conscious percepts in the binocular rivalry (Doesburg, Green, McDonald, & Ward, 2009) and an apparent motion across visual hemispheres (Rose & Buchel, 2005). Given that delayed activity in this study was observed in a broad region of the brain (Figures 4 and 5), our delayed activity may also indicate neural signals for communications across multiple cortical areas (as the oscillation signals). Although these neural communications were inhibited during the flash period (as seen in a lack of delayed activity to the subjective-onset grating; Figure 10A), the disappearance of continuous flashes enabled an exchange of visual information regarding the grating between lower and higher visual regions, which triggered the emergence of delayed activity and conscious percept of the grating in the post-flash period.

Acknowledgments

This work was supported by a grant from the Japan Society for the Promotion of Science for Young Scientists to Y. N. and a grant from Japan Society for the Promotion of Science for Young Researchers to T. Y. We thank Mr. Y. Takeshima (National Institute for Physiological Sciences, Japan) for his technical support. The authors would like to thank Enago (www.enago.jp) for the English language review.

Reprint requests should be sent to Yasuki Noguchi, Department of Psychology, Kobe University, 1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan, or via e-mail: ynoguchi@lit.kobe-u.ac.jp.

REFERENCES

REFERENCES
Altmann
,
C. F.
,
Nakata
,
H.
,
Noguchi
,
Y.
,
Inui
,
K.
,
Hoshiyama
,
M.
,
Kaneoke
,
Y.
,
et al
(
2008
).
Temporal dynamics of adaptation to natural sounds in the human auditory cortex.
Cerebral Cortex
,
18
,
1350
1360
.
Block
,
N.
(
2005
).
Two neural correlates of consciousness.
Trends in Cognitive Sciences
,
9
,
46
52
.
Brainard
,
D. H.
(
1997
).
The psychophysics toolbox.
Spatial Vision
,
10
,
433
436
.
Crick
,
F.
, &
Koch
,
C.
(
2003
).
A framework for consciousness.
Nature Neuroscience
,
6
,
119
126
.
Dehaene
,
S.
,
Changeux
,
J. P.
,
Naccache
,
L.
,
Sackur
,
J.
, &
Sergent
,
C.
(
2006
).
Conscious, preconscious, and subliminal processing: A testable taxonomy.
Trends in Cognitive Sciences
,
10
,
204
211
.
Doesburg
,
S. M.
,
Green
,
J. J.
,
McDonald
,
J. J.
, &
Ward
,
L. M.
(
2009
).
Rhythms of consciousness: Binocular rivalry reveals large-scale oscillatory network dynamics mediating visual perception.
PLoS One
,
4
,
e6142
.
Fang
,
F.
, &
He
,
S.
(
2005
).
Cortical responses to invisible objects in the human dorsal and ventral pathways.
Nature Neuroscience
,
8
,
1380
1385
.
Fisch
,
L.
,
Privman
,
E.
,
Ramot
,
M.
,
Harel
,
M.
,
Nir
,
Y.
,
Kipervasser
,
S.
,
et al
(
2009
).
Neural “ignition”: Enhanced activation linked to perceptual awareness in human ventral stream visual cortex.
Neuron
,
64
,
562
574
.
Friston
,
K.
,
Harrison
,
L.
,
Daunizeau
,
J.
,
Kiebel
,
S.
,
Phillips
,
C.
,
Trujillo-Barreto
,
N.
,
et al
(
2008
).
Multiple sparse priors for the M/EEG inverse problem.
Neuroimage
,
39
,
1104
1120
.
Fujita
,
I.
,
Tanaka
,
K.
,
Ito
,
M.
, &
Cheng
,
K.
(
1992
).
Columns for visual features of objects in monkey inferotemporal cortex.
Nature
,
360
,
343
346
.
Haxby
,
J. V.
,
Grady
,
C. L.
,
Horwitz
,
B.
,
Ungerleider
,
L. G.
,
Mishkin
,
M.
,
Carson
,
R. E.
,
et al
(
1991
).
Dissociation of object and spatial visual processing pathways in human extrastriate cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
88
,
1621
1625
.
Haynes
,
J. D.
,
Deichmann
,
R.
, &
Rees
,
G.
(
2005
).
Eye-specific effects of binocular rivalry in the human lateral geniculate nucleus.
Nature
,
438
,
496
499
.
Henson
,
R. N.
,
Mattout
,
J.
,
Singh
,
K. D.
,
Barnes
,
G. R.
,
Hillebrand
,
A.
, &
Friston
,
K.
(
2007
).
Population-level inferences for distributed MEG source localization under multiple constraints: Application to face-evoked fields.
Neuroimage
,
38
,
422
438
.
Hillyard
,
S. A.
, &
Anllo-Vento
,
L.
(
1998
).
Event-related brain potentials in the study of visual selective attention.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
781
787
.
Hubel
,
D. H.
, &
Wiesel
,
T. N.
(
1979
).
Brain mechanisms of vision.
Scientific American
,
241
,
150
162
.
Kim
,
C. Y.
, &
Blake
,
R.
(
2005
).
Psychophysical magic: Rendering the visible “invisible.”
Trends in Cognitive Sciences
,
9
,
381
388
.
Lee
,
S. H.
,
Blake
,
R.
, &
Heeger
,
D. J.
(
2005
).
Traveling waves of activity in primary visual cortex during binocular rivalry.
Nature Neuroscience
,
8
,
22
23
.
Leopold
,
D. A.
, &
Logothetis
,
N. K.
(
1996
).
Activity changes in early visual cortex reflect monkeys' percepts during binocular rivalry.
Nature
,
379
,
549
553
.
Liu
,
J.
,
Harris
,
A.
, &
Kanwisher
,
N.
(
2002
).
Stages of processing in face perception: An MEG study.
Nature Neuroscience
,
5
,
910
916
.
Maier
,
A.
,
Wilke
,
M.
,
Aura
,
C.
,
Zhu
,
C.
,
Ye
,
F. Q.
, &
Leopold
,
D. A.
(
2008
).
Divergence of fMRI and neural signals in V1 during perceptual suppression in the awake monkey.
Nature Neuroscience
,
11
,
1193
1200
.
McDonald
,
J. J.
,
Teder-Salejarvi
,
W. A.
,
Di Russo
,
F.
, &
Hillyard
,
S. A.
(
2005
).
Neural basis of auditory-induced shifts in visual time-order perception.
Nature Neuroscience
,
8
,
1197
1202
.
Mishkin
,
M.
, &
Ungerleider
,
L. G.
(
1982
).
Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys.
Behavioural Brain Research
,
6
,
57
77
.
Nishitani
,
N.
, &
Hari
,
R.
(
2002
).
Viewing lip forms: Cortical dynamics.
Neuron
,
36
,
1211
1220
.
Noesselt
,
T.
,
Hillyard
,
S. A.
,
Woldorff
,
M. G.
,
Schoenfeld
,
A.
,
Hagner
,
T.
,
Jancke
,
L.
,
et al
(
2002
).
Delayed striate cortical activation during spatial attention.
Neuron
,
35
,
575
587
.
Noguchi
,
Y.
,
Inui
,
K.
, &
Kakigi
,
R.
(
2004
).
Temporal dynamics of neural adaptation effect in the human visual ventral stream.
Journal of Neuroscience
,
24
,
6283
6290
.
Okusa
,
T.
,
Kaneoke
,
Y.
,
Koyama
,
S.
, &
Kakigi
,
R.
(
1998
).
Random dots blinking: A new approach to elucidate the activities of the extrastriate cortex in humans.
NeuroReport
,
9
,
3961
3965
.
Pelli
,
D. G.
(
1997
).
The VideoToolbox software for visual psychophysics: Transforming numbers into movies.
Spatial Vision
,
10
,
437
442
.
Roeber
,
U.
,
Widmann
,
A.
,
Trujillo-Barreto
,
N. J.
,
Herrmann
,
C. S.
,
O'Shea
,
R. P.
, &
Schroger
,
E.
(
2008
).
Early correlates of visual awareness in the human brain: Time and place from event-related brain potentials.
Journal of Vision
,
8
,
21.21
12
.
Rose
,
M.
, &
Buchel
,
C.
(
2005
).
Neural coupling binds visual tokens to moving stimuli.
Journal of Neuroscience
,
25
,
10101
10104
.
Roth
,
H. L.
,
Lora
,
A. N.
, &
Heilman
,
K. M.
(
2002
).
Effects of monocular viewing and eye dominance on spatial attention.
Brain
,
125
,
2023
2035
.
Sergent
,
C.
,
Baillet
,
S.
, &
Dehaene
,
S.
(
2005
).
Timing of the brain events underlying access to consciousness during the attentional blink.
Nature Neuroscience
,
8
,
1391
1400
.
Sheinberg
,
D. L.
, &
Logothetis
,
N. K.
(
1997
).
The role of temporal cortical areas in perceptual organization.
Proceedings of the National Academy of Sciences, U.S.A.
,
94
,
3408
3413
.
Stelmach
,
L. B.
, &
Herdman
,
C. M.
(
1991
).
Directed attention and perception of temporal order.
Journal of Experimental Psychology: Human Perception and Performance
,
17
,
539
550
.
Sterzer
,
P.
,
Jalkanen
,
L.
, &
Rees
,
G.
(
2009
).
Electromagnetic responses to invisible face stimuli during binocular suppression.
Neuroimage
,
46
,
803
808
.
Sterzer
,
P.
,
Kleinschmidt
,
A.
, &
Rees
,
G.
(
2009
).
The neural bases of multistable perception.
Trends in Cognitive Sciences
,
13
,
310
318
.
Tarkiainen
,
A.
,
Helenius
,
P.
, &
Salmelin
,
R.
(
2003
).
Category-specific occipitotemporal activation during face perception in dyslexic individuals: An MEG study.
Neuroimage
,
19
,
1194
1204
.
Tong
,
F.
, &
Engel
,
S. A.
(
2001
).
Interocular rivalry revealed in the human cortical blind-spot representation.
Nature
,
411
,
195
199
.
Tononi
,
G.
, &
Edelman
,
G. M.
(
1998
).
Consciousness and complexity.
Science
,
282
,
1846
1851
.
Tononi
,
G.
, &
Koch
,
C.
(
2008
).
The neural correlates of consciousness: An update.
Annals of the New York Academy of Sciences
,
1124
,
239
261
.
Tsuchiya
,
N.
, &
Koch
,
C.
(
2005
).
Continuous flash suppression reduces negative afterimages.
Nature Neuroscience
,
8
,
1096
1101
.
Wu
,
C. T.
,
Busch
,
N. A.
,
Fabre-Thorpe
,
M.
, &
VanRullen
,
R.
(
2009
).
The temporal interplay between conscious and unconscious perceptual streams.
Current Biology
,
19
,
2003
2007
.
Wunderlich
,
K.
,
Schneider
,
K. A.
, &
Kastner
,
S.
(
2005
).
Neural correlates of binocular rivalry in the human lateral geniculate nucleus.
Nature Neuroscience
,
8
,
1595
1602
.