Abstract

Perceiving the temporal order of sensory events typically depends on participants' attentional state, thus likely on the endogenous fluctuations of brain activity. Using magnetoencephalography, we sought to determine whether spontaneous brain oscillations could disambiguate the perceived order of auditory and visual events presented in close temporal proximity, that is, at the individual's perceptual order threshold (Point of Subjective Simultaneity [PSS]). Two neural responses were found to index an individual's temporal order perception when contrasting brain activity as a function of perceived order (i.e., perceiving the sound first vs. perceiving the visual event first) given the same physical audiovisual sequence. First, average differences in prestimulus auditory alpha power indicated perceiving the correct ordering of audiovisual events irrespective of which sensory modality came first: a relatively low alpha power indicated perceiving auditory or visual first as a function of the actual sequence order. Additionally, the relative changes in the amplitude of the auditory (but not visual) evoked responses were correlated with participant's correct performance. Crucially, the sign of the magnitude difference in prestimulus alpha power and evoked responses between perceived audiovisual orders correlated with an individual's PSS. Taken together, our results suggest that spontaneous oscillatory activity cannot disambiguate subjective temporal order without prior knowledge of the individual's bias toward perceiving one or the other sensory modality first. Altogether, our results suggest that, under high perceptual uncertainty, the magnitude of prestimulus alpha (de)synchronization indicates the amount of compensation needed to overcome an individual's prior in the serial ordering and temporal sequencing of information.

INTRODUCTION

To build a stable representation of the world, the integration of information within and across sensory modalities is essential. Multisensory information is typically integrated within a few tens of milliseconds (Spence & Parise, 2010; Vroomen & Keetels, 2010; van Wassenhove, Grant, & Poeppel, 2007), consistent with the range of temporal tuning observed in multisensory neuronal populations (Meredith, Nemitz, & Stein, 1987; Benevento, Fallom, Davis, & Rezak, 1977). Within such timescales, the temporal sequencing of sensory information is a priori irrelevant to establish a stable representation of a conscious integrated content (Pöppel, 2009; Pöppel, Schill, & von Steinbüchel, 1990); indeed, temporal resolution would be lost by virtue of integration. In this context, it is surprising that temporal order perception—that is, the conscious representation of primitive temporal sequences—is highly variable across individuals (Grabot & van Wassenhove, 2017; Ipser et al., 2017; Kösem, Gramfort, & van Wassenhove, 2014; Hanson, Heron, & Whitaker, 2008; Fujisaki, Shimojo, Kashino, & Nishida, 2004; Vroomen, Keetels, de Gelder, & Bertelson, 2004; Stone et al., 2001), possibly indicating individual specificity in the duration of information processing (Stone et al., 2001). Consistent with this, temporal order perception displays large interindividual variability but within-individual stability over weeks (Grabot & van Wassenhove, 2017). If the outcome of multisensory integration informs on the content of conscious representations, the perceived timing of sensory information may thus inform on the structuration of conscious representations (Lashley, 1951).

In this magnetoencephalography (MEG) study, we investigated the neural sources of interindividual variability in audiovisual temporal order perception and specifically asked whether ongoing intrinsic neural fluctuations could predict the perception of audiovisual temporal order. Participants were presented with near threshold audiovisual stimuli (individually tailored) while they performed a temporal order judgment (TOJ) task. The temporal asynchrony between the auditory and visual stimuli was chosen on a per individual basis so as to yield ambiguous order perception and to contrast brain responses to two differently perceived sequences given the same physical stimulation—that is, audio perceived first, thereafter referred to as “AV,” or visual perceived first, thereafter “VA.”

One working hypothesis was that an individual bias in temporal order perception may reflect the spontaneous preference in paying more attention toward one or the other sensory modality. Such intrinsic preference would predict, in agreement with the law of prior entry (Spence & Parise, 2010), that a stimulus being attended to would be perceived as occurring earlier than a simultaneous stimulus being less attended to. Stated differently, an attended stimulus would have priority over another one in the conscious temporal sequencing of sensory information, in agreement with the notion that ongoing neural dynamics endogenously determine the temporal structuring of sensory information (Lashley, 1951). Using a TOJ task, which sensory modality an individual attends to at the time of stimulus presentation should thus predict which stimulus will be perceived first: attending to audition should facilitate auditory processing yielding earlier conscious perception of the sound for near synchrony audiovisual stimuli, and conversely, attending to vision should lead to perceiving the visual event as being first in the pair (Vibell, Klinge, Zampini, Spence, & Nobre, 2007; McDonald, Teder-Salejarvi, Di Russo, & Hillyard, 2005; Zampini, Shore, & Spence, 2005). Several neural indices have been reported supporting this hypothesis: In seminal EEG experiments testing TOJ, the attended stimuli that were more often perceived first also elicited evoked responses with earlier latencies (Vibell et al., 2007) and larger amplitudes (McDonald et al., 2005) than when perceived second. The report of latency shifts in early sensory cortices are consistent with temporal prediction (Stekelenburg & Vroomen, 2007) and parametrical predictions of speech content (van Wassenhove, Grant, & Poeppel, 2005). Additionally, an increase of visual attention has been shown to decrease ongoing prestimulus alpha power in visual cortices (Klimesch, Sauseng, & Hanslmayr, 2007; Sauseng et al., 2005), in turn favoring the detection of an incoming near-threshold stimulus (van Dijk, Schoffelen, Oostenveld, & Jensen, 2008; Hanslmayr et al., 2007; Ergenoglu et al., 2004). Alpha oscillations have been proposed to act as selective filters, which inhibit task-irrelevant stimulus processing in a cyclic or pulsed manner (Sadaghiani & Kleinschmidt, 2016; Klimesch, 2012; Jensen & Mazaheri, 2010). During a TOJ task, a decrease of prestimulus alpha power in auditory cortices together with an increase of alpha power in visual cortices was predicted to index an intrinsic preference in paying more attention to audition associated with an increased likelihood of perceiving the sound first. Conversely, participants whose tendency would be to attend to vision would exhibit a decrease in prestimulus alpha power in the visual cortex associated with an increased likelihood of perceiving the flash first (Figure 1A).

Figure 1. 

Working hypotheses for prestimulus alpha oscillations. (A) Prestimulus alpha power fluctuations and attention. A decrease of prestimulus alpha power in one sensory modality predicts increased attention to this sensory modality, hence a higher likelihood to detect and report the stimulus as being first. A decrease of prestimulus alpha power in auditory cortices combined with an increase alpha power in visual cortices would thus be predicted when perceiving the sound before the flash; conversely, an increase of prestimulus alpha power in auditory cortices combined with a decrease of prestimulus alpha power in visual cortices would be predicted when perceiving the flash before the sound. (B) iAPF. Following recent reports of correlations between timing ability and higher alpha peak frequency (Cecere et al., 2015; Samaha & Postle, 2015), the higher the individual peak frequency, the finer the discriminability between two sensory events, and the smaller the individual's PSS could be. (C) Phase of prestimulus alpha. Following the hypothesis that the phase of ongoing oscillations indexes excitability, two distinct phases may index the detectability of the stimuli in each sensory cortices resulting in differences of perceived audiovisual temporal order. Specifically, the sensory cortices showing the highest excitability in the prestimulus period should also predict seeing the stimulus in that sensory modality first.

Figure 1. 

Working hypotheses for prestimulus alpha oscillations. (A) Prestimulus alpha power fluctuations and attention. A decrease of prestimulus alpha power in one sensory modality predicts increased attention to this sensory modality, hence a higher likelihood to detect and report the stimulus as being first. A decrease of prestimulus alpha power in auditory cortices combined with an increase alpha power in visual cortices would thus be predicted when perceiving the sound before the flash; conversely, an increase of prestimulus alpha power in auditory cortices combined with a decrease of prestimulus alpha power in visual cortices would be predicted when perceiving the flash before the sound. (B) iAPF. Following recent reports of correlations between timing ability and higher alpha peak frequency (Cecere et al., 2015; Samaha & Postle, 2015), the higher the individual peak frequency, the finer the discriminability between two sensory events, and the smaller the individual's PSS could be. (C) Phase of prestimulus alpha. Following the hypothesis that the phase of ongoing oscillations indexes excitability, two distinct phases may index the detectability of the stimuli in each sensory cortices resulting in differences of perceived audiovisual temporal order. Specifically, the sensory cortices showing the highest excitability in the prestimulus period should also predict seeing the stimulus in that sensory modality first.

Second, an individual's simultaneity threshold (or Point of Subjective Simultaneity [PSS]) could reflect an individual's bias toward one or the other sensory modality, irrespective of the participant's attentional state (Grabot & van Wassenhove, 2017). For example, one seminal study reported systematic differences between early visual and auditory evoked responses as a function of individuals who may have preferences toward one or the other sensory modality (Giard & Peronnet, 1999). On the basis of the observations that individuals show robust differences in alpha peak frequency (Haegens, Cousijn, Wallis, Harrison, & Nobre, 2014) and that an individual alpha peak frequency (iAPF) may be predictive of an individual's visual temporal resolution (Cecere, Rees, & Romei, 2015; Samaha & Postle, 2015), we tested whether iAPF could also be a good predictor of subsequent temporal order perception across sensory modalities (Figure 1B).

In a third working hypothesis, we focused on the phase of ongoing neuronal oscillations, in particular alpha and theta oscillations, as predictors of participants' temporal order perception. A growing body of evidence has suggested that the phase of ongoing alpha oscillations predicts the detectability of a sensory event (Mathewson, Fabiani, Gratton, Beck, & Lleras, 2010; Busch, Dubois, & VanRullen, 2009; Mathewson, Gratton, Fabiani, Beck, & Ro, 2009): A stimulus presented at the highest excitability phase of neural oscillations is more likely to be detected and missed otherwise (Schroeder & Lakatos, 2009). The impact of oscillatory phase differences in timing and perception has also been reported across sensory modalities (Thorne & Debener, 2014; Kösem & van Wassenhove, 2012; Thorne, De Vos, Viola, & Debener, 2011). When presented with temporally regular stimuli, cortical oscillations entrain to the input rhythm (Rees, Green, & Kay, 1986; Regan, 1966) and the phase of low-frequency neuronal oscillations has been shown to reflect the temporal expectancy or predictability of event timing (Stefanics et al., 2010). As changes in the phase of entrained neuronal oscillations were previously shown to linearly predict an individual's changes in perceived audiovisual temporal order (Kösem et al., 2014), one hypothesis was that the phase diversity in spontaneous neuronal fluctuations and in the absence of sensory entrainment may predict conscious temporal order. On the basis of the cortical temporal framing hypothesis (Gho & Varela, 1988; Varela, Toro, John, & Schwartz, 1981), the duty cycle of alpha and theta neural oscillations have been hypothesized to code for the temporal sequencing of sensory events (Lisman & Jensen, 2013). The alpha phase, under attentional control (Samaha, Bauer, Cimaroli, & Postle, 2015), has specifically been hypothesized to engage in the prioritization of information in time (Bonnefond & Jensen, 2012; Jensen, Bonnefond, & VanRullen, 2012), thus predicting possible differences in prestimulus alpha phase as a function of perceived order (Figure 1C).

METHODS

Participants

Fourteen participants (6 women, mean age = 24.9 −+ 4.1 years) took part in the study. All had normal or corrected-to-normal vision and normal hearing and were naive as to the purpose of the study. Each participant provided written informed consent in accordance with the Declaration of Helsinki (2008) and the ethics committee on human research at NeuroSpin (Gif-sur-Yvette, France). Two participants were discarded from the analysis: One participant showed excessive head movements during MEG recordings, and a second participant did not perform the task properly, preventing the assessment of perceptual thresholds. Hence, a total of 12 participants (four women, mean age = 24.8 ± 4.4 years) were analyzed.

Stimuli

The experiment was written in MATLAB 2014a (The MathWorks, Inc., Natick, MA) with the PsychToolbox (version 3.0.11). Visual stimuli consisted of a photodiode turning on for 30 msec in a square wave manner and placed 90 cm away from the participants seated under the MEG dewar. Auditory stimuli consisted of 30 msec sinusoid signals (including 5 msec fade-in and fade-out, with a fundamental frequency of 2 kHz). Sounds were presented binaurally through Etymotic earphones (Etymotic Research, Inc, Elk Grove Village, IL).

Procedure

The experiment was run in three consecutive steps: 5 min familiarization with the task, followed by 10 min of psychometric assessment, and 40 min of MEG recordings. In all three steps, participants performed a TOJ task on pairs of audiovisual stimuli (Figure 2A): In all TOJ thereafter, a trial consisted of a pair of asynchronous audiovisual stimuli and participants had to indicate by a button press which of the auditory or the visual stimulus was presented first. During familiarization, participants completed 20 TOJ trials to become familiarized with the task and with the motor response mapping. Participants then performed the TOJ task from which we established the psychometric curve and derive the audiovisual SOA used during the MEG recordings in the third part of the experiment (Figure 2B). To establish participants' psychometric curve, 11 SOAs were used: ±300, 240, 180, 120, 60, and 0 msec. A negative (positive) SOA corresponds to auditory leading (lagging) visual stimuli. Each SOA was presented 10 times in random order within and across individuals. Each intertrial interval (ITI) was computed from a standard uniform distribution ranging from 1 to 2 sec. The resulting individuals' psychometric curves were individually fitted with a logistic function, and three parameters were extracted from the fit: the PSS, which corresponds to the SOA when participants report perceiving 50% of “visual stimulus first,” and the two Just-Noticeable Differences (JND1 and JND2) corresponding to 25% and 75% of “visual stimulus perceived first” responses, respectively. These three individually tailored SOAs were used for the MEG recordings.

Figure 2. 

Experimental design and psychophysical results. (A) The experiment consisted of presenting a pair of desynchronized audiovisual stimuli to participants, who performed a TOJ task, that is, judged which of the sound or the visual event came first. A first psychophysics experiment used SOAs selected among 11 values ranging from −300 msec (sound leading) to +300 msec (flash leading) to derive an individual's PSS. ITIs ranged from 1 to 2 sec. During MEG acquisition, only three SOAs were tested tailored to the individual's performance (PSS and JNDs). During MEG, ITIs ranged from 3 to 5 sec. (B) Psychometric curves obtained in the first part of the experiment. The percentage of visual first responses are plotted as a function of audiovisual SOA (msec) for each individual (black) and for the grand average (red). (C) Individual PSS and JND values. A majority of participants showed a negative PSS value indicating that participants required the sound to lead the visual event to perceive them as being simultaneous. PSS and JND values were used as audiovisual SOA in the MEG experiment. (D) Grand-averaged PSS and JND observed during the MEG acquisition in Blocks 1, 2, and 3. No significant changes in PSS (or JND2) were observed, but a significant increase of JND1 was found from Blocks 1 to 3.

Figure 2. 

Experimental design and psychophysical results. (A) The experiment consisted of presenting a pair of desynchronized audiovisual stimuli to participants, who performed a TOJ task, that is, judged which of the sound or the visual event came first. A first psychophysics experiment used SOAs selected among 11 values ranging from −300 msec (sound leading) to +300 msec (flash leading) to derive an individual's PSS. ITIs ranged from 1 to 2 sec. During MEG acquisition, only three SOAs were tested tailored to the individual's performance (PSS and JNDs). During MEG, ITIs ranged from 3 to 5 sec. (B) Psychometric curves obtained in the first part of the experiment. The percentage of visual first responses are plotted as a function of audiovisual SOA (msec) for each individual (black) and for the grand average (red). (C) Individual PSS and JND values. A majority of participants showed a negative PSS value indicating that participants required the sound to lead the visual event to perceive them as being simultaneous. PSS and JND values were used as audiovisual SOA in the MEG experiment. (D) Grand-averaged PSS and JND observed during the MEG acquisition in Blocks 1, 2, and 3. No significant changes in PSS (or JND2) were observed, but a significant increase of JND1 was found from Blocks 1 to 3.

The third and last part of the experiment (∼40 min) consisted of three MEG acquisition blocks. In each block, the three tested AV asynchronies corresponded to the SOAs obtained for the individual's PSS, JND1, and JND2. Each audiovisual asynchrony was presented 30–40 times per block (four and eight participants, respectively). Thus, 90–120 trials per condition were tested during the entire MEG session. The ITIs were selected from a standard uniform distribution ranging from 3 to 5 sec. Long ITIs were chosen for two main reasons: Long delays prevented oscillatory entrainment to the stimuli and ensured sufficient time periods for time–frequency analysis in the low-frequency regions (e.g., a minimum of three cycles was available to analyze 1 Hz oscillatory activity). JND1 and JND2 conditions were added to balance task difficulty and prevent decreasing attention from the participant. Specifically, JND1 and JND2 stimuli could provide clear sensory evidence that could confirm or counteract a participant's intrinsic temporal order bias, whereas the PSS condition, by definition, could not. Considering that behavioral outcomes served as our main criterion to classify and contrast the data, preserving only the PSS data meant that the ambiguity in sensory evidence was most likely preserving the initial bias in participants' decision-making. In other words, this condition was the most likely to provide the most legitimate quantification of intrinsic bias during the prestimulus period. Hence, only one contrast reported in Figure 4 used all combined conditions (i.e., JND1, JND2, and PSS). Otherwise, data collected for JND1 and JND2 were not used for the MEG analysis, which we specifically focused on the PSS condition yielding maximal perceptual ambiguity on a per individual basis.

MEG Data Acquisition

Brain activity was recorded in a magnetically shielded room using the whole-head Elekta Neuromag Vector View 306 MEG system (Elekta Neuromag, Helsinki, Finland) equipped with 102 triple-sensors elements (two orthogonal planar gradiometers, grad1 and grad2 in text, and one magnetometer per sensor location). Participants were seated in upright position, and their head position was measured before each block using four head position coils placed over the frontal and the mastoid areas. The four head position coils and three additional fiducial points (nasion, left and right preauricular areas) were used during digitization to help with coregistration of the individual's anatomical MRI. MEG recordings were sampled at 1 kHz and band-pass filtered between 0.03 and 330 Hz. The EOGs (horizontal and vertical eye movements) and electrocardiograms were recorded simultaneously with MEG. Before each experiment, a ∼1 min empty room recording (no participant in the room) was acquired for the computation of the noise covariance matrix.

MEG Data Analysis

Preprocessing

Signal space separation was applied to decrease the impact of external noise on recorded brain signals (Taulu & Simola, 2006). Gradiometers with amplitudes exceeding 4e−13 T/m were set as “bad sensors” and excluded from further analysis. Signal space separation correction, head movement compensation, and bad sensor rejection was done using MaxFilter Software (Elekta Neuromag). PCA using Graph software (Elekta Neuromag) was performed to remove eye blinks and cardiac artifacts (Uusitalo & Ilmoniemi, 1997). On average, 41 ± 9.8 trials (mean ± 1 SD) were used in the PSS contrasts comparing each perceptual outcome (AV and VA).

MEG–aMRI Coregistration

MRI was used to provide high-resolution structural image of each individual's brain. The anatomical MRI was recorded using a 3-T Siemens Trio MRI scanner. Parameters of the sequence were voxel size = 1.0 × 1.0 × 1.1 mm, acquisition time = 466 sec; repetition time = 2300 msec, and echo time = 2.98 msec. Volumetric segmentation of participants' anatomical MRI and cortical surface reconstruction was performed with the FreeSurfer software (surfer.nmr.mgh.harvard.edu/). These procedures were used for group analysis with the MNE suite software (Gramfort et al., 2014). The coregistration of the MEG data with the individual's structural MRI was carried out by realigning the digitized fiducial points with MRI slices. Using mne_analyze within the MNE suite, digitized fiducial points were aligned manually with the multimodal markers on the automatically extracted scalp of the participant. To ensure reliable coregistration, an iterative refinement procedure was used to realign all digitized points with the individual's scalp.

MEG Source Reconstruction

Individual forward solutions for all source locations located on the cortical sheet were computed using a three-layer boundary element model constrained by the individual's anatomical MRI. Cortical surfaces extracted with FreeSurfer were subsampled to 10,242 equally spaced sources on each hemisphere (3.1 mm between sources). The noise covariance matrix for each individual was estimated from the raw empty room MEG recordings preceding the individual's MEG acquisition. The forward solution, the noise covariance, and source covariance matrices were used to calculate the dynamic statistical parametric mapping estimates (Dale et al., 2000). The inverse computation was done using a loose orientation constraint (loose = 0.4, depth = 0.8) on the radial component of the signal. Individuals' current source estimates were registered on the FreeSurfer average brain for surface-based analysis and visualization.

Time–Domain Source Estimates and Cortical Labels

Both sensor (event-related fields) and source space analyses (source estimates) were performed. Here, for consistency across analyses, we will only report source estimated data distinguishing auditory and visual cortices. Evoked responses obtained after grand averaging across all conditions, all responses, and all participants were used as localizers. Source localizers predictably revealed auditory and visual sensory cortices as main contributors to the MEG signals. Labels encompassing these two regions were selected from the FreeSurfer neuroanatomical parcellation (https://surfer.nmr.mgh.harvard.edu/fswiki/CorticalParcellation). All vertices within these cortical regions were used for all subsequent source space analyses and referred to as “auditory cortices” and “visual cortices” (corresponding to the transverse temporal gyrus and occipital in FreeSurfer, respectively). MEG data were down-sampled to 250 Hz and epoched in 2 sec from 1 sec before the onset of the first stimulus to 1 sec following the first stimulus onset (first stimulus was auditory for participants with a negative PSS and visual for participants with a positive PSS). Epochs were zero-phased (two passes) low-pass filtered at 45 Hz. Baseline correction was applied using the first second of the epoch. Trials were sorted as a function of participants' response, and the number of epochs was equalized across conditions using automatized procedure in MNE-Python, which ensured that the selection of trials across conditions were as close in time during the acquisition as possible (cf. martinos.org/mne/stable/generated/mne.epochs.equalize_epoch_counts.html?highlight=equalize#mne.epochs.equalize_epoch_counts).

Power Spectra and iAPF

All epochs were combined, and the power spectrum density (PSD) was computed for the signal ranging from −1 sec to −0.1 sec before the onset of the first presented stimulus. Welch fast Fourier transform was used to compute the PSD for a frequency range of 1–45 Hz. The alpha peak frequency was manually reported for each individual as the frequency corresponding to the peak maximum within a [8–13] Hz frequency window.

Time–Frequency Analysis

Trials were sorted according to participants' percept. The number of trials was equalized in the same manner as performed in the evoked response analysis. Data were also down-sampled to 250 Hz. Contamination from evoked activity by prior stimuli was prevented by the selection of large ITIs (3–5 sec). Single-trial time–frequency analysis was applied from −3.3 to +2 sec poststimulus onset. As the time–frequency hypotheses focused on prestimulus activity, no baseline was applied for the computation of oscillatory activity (Jensen & Mazaheri, 2010; Busch et al., 2009; Hanslmayr et al., 2007). Power and phase-locking values (PLVs) were calculated using Morlet wavelets for six frequency bands of interest defined by the central oscillatory peak frequency and by their cycles: delta (1.4–2.7 Hz, 3 cycles), theta (3.9–7.1 Hz, 3.4 cycles), alpha (7.4–12.6 Hz, 3.8 cycles), low beta (13–19 Hz, 5.3 cycles), high beta (19.5–30.4 Hz, 4.6 cycles), and gamma (32–48 Hz, 5 cycles). The bifurcation index (BI) was computed on the basis of the PLV (Busch et al., 2009) using
PLVt=1NnNejθtn
where N was the number of trials and θ(t,k) was the instantaneous phase at time t for trial n.
BI=PLVAVPLVtotPLVVAPLVtot
where AV and VA were the two possible audiovisual temporal order percepts, namely perceiving the sound before the visual event (AV) or perceiving the visual event before the sound (VA). As previously described (Busch et al., 2009), a positive BI indicated that the two conditions were more phase-locked than the total trials together and that their phase differed; otherwise, the total PLV would be identical to the PLV of each condition, and the BI would be around zero. A negative BI indicated that the PLV in one condition was larger than the PLV in the other condition. The power and BI quantifications were averaged on the sensory labels previously determined by the independent source localizer.

Statistical Analysis

Differences in prestimulus alpha power fluctuations were computed for each individual in the auditory and visual sensory cortices and correlated with the individual PSS. Pearson correlations were performed on each time point from −500 to 0 msec relative to the onset of the first physically presented stimulus. The false discovery rate (FDR) correction implemented in MNE-Python was used to correct for multiple comparisons across time (Genovese, Lazar, & Nichols, 2002). Only correlation time points associated with corrected p value under .05 are reported. Phase differences were tested using one-sample cluster-based analysis on BI data testing against zero (Strauß, Henry, Scharinger, & Obleser, 2015). This analysis was run separately in the auditory and visual cortices, using MNE-Python (Gramfort et al., 2013). As no significant differences were found between the two hemispheres, statistical analyses were run on the average. The analysis was restricted to the first 500 msec before the onset of the first physically presented stimulus, and multiple comparisons were corrected by cluster permutation. Additionally, we ran a whole-brain spatiotemporal cluster analysis for each frequency band using the same temporal window. The cluster-forming threshold was set to 0.01, which was equivalent to a t threshold of 3.1 in an experimental design using 12 participants. Only clusters associated with a p value inferior to .05 are reported.

RESULTS

Interindividual Variability in Temporal Order Perception

The mean individual PSS was close to veridical simultaneity with a value of 16 ± 35 msec (SD). A large interindividual variability was found consistent with previous reports (Grabot & van Wassenhove, 2017; Ipser et al., 2017; Freeman et al., 2013; Stone et al., 2001). Individual PSSs ranged from −81 msec (sound leading the flash) to +43 msec (light leading the sound). For 9 of 12 participants, the sound needed to be shown before the visual stimulus to be perceived as simultaneous thus yielding to a negative PSS. The mean JND1 and JND2 were −102 ± 20 msec and 70 ± 75 msec, respectively (Figure 2C; Table 1). During the MEG acquisition, the percentage of responses “I perceived the visual stimulus first” resulted in 59% ± 7% (Figure 2D; Table 1), suggesting that changes in audiovisual synchrony perception had taken place in the course of the experiment: In particular, the more negative the initial PSS was (i.e., the more the participant needed the sound to precede the visual stimulus to be perceived as simultaneous), the more deviation toward perceiving the visual event first was found during the MEG session. Seminal recalibration effects in audiovisual synchrony judgments (Fujisaki et al., 2004; Vroomen et al., 2004) suggest that slow perceptual drifts may have occurred during the experiment on a per individual basis. To test this, we looked at the evolution of participants' audiovisual temporal order perception, separately for each MEG block, and we expected that the deviation from the predicted 50% threshold would increase over time. However, there was no significant difference between blocks excepted for a 10.7% increase of JND1 between Blocks 1 and 3 (Figure 2D; PSS: t = 0.805, p = .439, 95% CI = [−0.039, 0.083]; JND1: t = 2.944, p = .015, 95% CI = [0.024, 0.170]; JND2: t = −1.504, p = .163, 95% CI = [−0.122, 0.024]), suggesting that either audiovisual temporal recalibration is a rapid transient process (Van der Burg, Orchard-Mills, & Alais, 2015) or a very slow hysteretic process undetectable over a short time period of time (Martin, Kösem, & van Wassenhove, 2015). Nevertheless, this side observation is consistent with the notion that the PSS is a “snapshot” of a participant's perceptual state, which may be related to the context in which it is measured (Van Eijk, Kohlrausch, Juola, & van de Par, 2008), although intraindividual variability of the PSS in adults is most likely robustly anchored in a stable intrinsic bias (Grabot & van Wassenhove, 2017). The selected SOAs, based on the initial participant's PSS, remained largely ambiguous for the participants, and great care was taken to equalize the trial count in all subsequent MEG contrasts notably in source estimates.

Table 1. 

Individual Behavioral Data before and during MEG

Participants PSS (msec) JND1(msec) JND2(msec) PSS JND1 JND2 
% “V First” Block Blocks 1-2-3 (Mean) 
S01 −39 −118 40 60-80-70 (70) 10-20-27 (19) 83-80-90 (84) 
S02 −23 −101 55 53-67 (60) 17-13 (15) 73-83 (78) 
S03 −27 −115 60 57-50-70 (59) 20-27-32 (27) 90-90-82 (88) 
S04 −49 −77 −21 50-52-55 (53) 35-50-42 (43) 65-62-50 (59) 
S05 −83 −112 −54 77-60-82 (72) 52-72-70 (65) 87-80-90 (86) 
S06 −3 −75 70 37-52-52 (48) 35-40-40 (38) 77-70-75 (74) 
S07 −43 −89 62-50-57 (62) 35-55-45 (40) 72-60-62 (61) 
S08 −12 −97 73 60-62-67 (63) 0-7-7 (5) 70-70-62 (67) 
S09 −12 −116 93 60-57-50 (56) 25-45-50 (40) 80-67-65 (71) 
S10 43 −121 208 63-30-50 (48) 23-40-43 (35) 63-43-40 (49) 
S11 23 −67 112 52-60-52 (55) 17-10-17 (15) 95-87-100 (94) 
S12 33 −132 198 42-42-42 (42) 60-57-45 (54) 37-42-50 (43) 
Participants PSS (msec) JND1(msec) JND2(msec) PSS JND1 JND2 
% “V First” Block Blocks 1-2-3 (Mean) 
S01 −39 −118 40 60-80-70 (70) 10-20-27 (19) 83-80-90 (84) 
S02 −23 −101 55 53-67 (60) 17-13 (15) 73-83 (78) 
S03 −27 −115 60 57-50-70 (59) 20-27-32 (27) 90-90-82 (88) 
S04 −49 −77 −21 50-52-55 (53) 35-50-42 (43) 65-62-50 (59) 
S05 −83 −112 −54 77-60-82 (72) 52-72-70 (65) 87-80-90 (86) 
S06 −3 −75 70 37-52-52 (48) 35-40-40 (38) 77-70-75 (74) 
S07 −43 −89 62-50-57 (62) 35-55-45 (40) 72-60-62 (61) 
S08 −12 −97 73 60-62-67 (63) 0-7-7 (5) 70-70-62 (67) 
S09 −12 −116 93 60-57-50 (56) 25-45-50 (40) 80-67-65 (71) 
S10 43 −121 208 63-30-50 (48) 23-40-43 (35) 63-43-40 (49) 
S11 23 −67 112 52-60-52 (55) 17-10-17 (15) 95-87-100 (94) 
S12 33 −132 198 42-42-42 (42) 60-57-45 (54) 37-42-50 (43) 

The pre-MEG scanning test provide the PSS, JND1, and JND2 (left-most columns). The MEG acquisition consisted of three acquisition blocks (1, 2, and 3). Responses « I perceived the flash first » (“V first”) were quantified for each condition and each block. The mean across the three blocks is provided in parentheses. Participant S02 did not complete the last block because of technical problems during the MEG acquisition. Figure 2D additionally reports the mean PSS, JND1, and JND2 in the MEG blocks.

Prestimulus Alpha Fluctuations and iAPF

One working hypothesis was that the differences in prestimulus alpha power could account for the perception of temporal order. First, we ensured that spontaneous alpha oscillations were readily seen in the prestimulus period. Figure 3 illustrates the grand-averaged PSD topographically and for one sensor: The alpha peak power (∼10 Hz) could readily be seen. The standard deviation of the PSD was also highest at ∼10 Hz, suggesting that the power of spontaneous alpha oscillations showed highest variability across participants and spectral responses in agreement with prior reports (Haegens et al., 2014). Almost all participants displayed a clear alpha peak frequency (Figure 3B), but no significant correlations were found between an individual's alpha peak frequency and an individual's audiovisual PSS (Figure 3C; r = .11, p > .25). To ensure that this result was not due to the low-frequency resolution in the default PSD computations, the PSD was recomputed between −300 and −100 msec with a finer frequency resolution of 0.34 Hz. Despite finer frequency resolution, the correlation between iAPF and PSS remained nonsignificant (r = .30, p = .38). The correlation between iAPF and JND (as defined by (JND2–JND1)) was also tested and did not yield any significant effect (Figure 3D; r = −.24, p = .47).

Figure 3. 

Inter-iAPF variability. (A) For illustration purposes, the group-average PSD is plotted for one representative magnetometer (white disc). The PSD amplitude are in decibels (dB) and calculated for all trials independently of the individual's percept, from −1000 to −100 msec before the onset of the first presented stimulus. The standard deviation (SD) of the PSD across all sensors (shaded gray) showed a maximal peak at ∼10 Hz (alpha peak frequency). This is also depicted in the bottom inset. The topography (top inset) illustrates the spatial distribution of the standard deviation (SD) of the PSD over all sensors at 10 Hz. (B) The PSD averaged across all sensors is displayed for each individual. (C) The alpha peak frequency (red line in B) was extracted for each individual and plotted against the individual's PSS. The correlation between iAPF and PSS was not significant (r = .06, p = .844). Increasing the frequency resolution (0.34 Hz) on a larger prestimulus temporal interval (−3000 to −100 msec) did not improve the correlation (r = −.031, p = .92; plot not shown). (D) The correlation between iAPF and JND (defined as JND2–JND1) was not significant (r = −.24, p = .47).

Figure 3. 

Inter-iAPF variability. (A) For illustration purposes, the group-average PSD is plotted for one representative magnetometer (white disc). The PSD amplitude are in decibels (dB) and calculated for all trials independently of the individual's percept, from −1000 to −100 msec before the onset of the first presented stimulus. The standard deviation (SD) of the PSD across all sensors (shaded gray) showed a maximal peak at ∼10 Hz (alpha peak frequency). This is also depicted in the bottom inset. The topography (top inset) illustrates the spatial distribution of the standard deviation (SD) of the PSD over all sensors at 10 Hz. (B) The PSD averaged across all sensors is displayed for each individual. (C) The alpha peak frequency (red line in B) was extracted for each individual and plotted against the individual's PSS. The correlation between iAPF and PSS was not significant (r = .06, p = .844). Increasing the frequency resolution (0.34 Hz) on a larger prestimulus temporal interval (−3000 to −100 msec) did not improve the correlation (r = −.031, p = .92; plot not shown). (D) The correlation between iAPF and JND (defined as JND2–JND1) was not significant (r = −.24, p = .47).

Prestimulus Alpha Power Fluctuations and Group-average Performance

We first combined all conditions (PSS, JND1, JND2) and computed the differences of prestimulus alpha power between correct and incorrect trials irrespective of temporal order. A significant decrease in prestimulus alpha power was found when participants were correct. A cluster-based permutation t test contrasting 500 msec prestimulus alpha preceding the presentation of the first stimulus was performed (Figure 4). Three significant clusters were found in sensor space: one widespread cluster including 52 magnetometers for the full 500 msec (p = .026) and two local clusters implicating in gradiometers (15 gradiometers, [−456, −12 msec], p = .023, and 14 gradiometers, [−500, 0 msec], p = .048). Figure 4A reports the alpha power time course observed across the mean of significant magnetometers for correct (cyan) and incorrect (pink) performance, irrespective of veridical temporal order. The same analysis was carried out in source-reconstructed data and revealed a significant cluster (p = .003) ranging from −304 to 0 msec. The cluster was mostly located in the right supramarginal gyrus and auditory regions.

Figure 4. 

Increased prestimulus alpha power predicts correct performance irrespective of temporal order. In this analysis, all conditions (JND1, JND2, PSS) were combined to assess the relationship between prestimulus alpha power changes and performance (correct in blue; incorrect in pink). (A) In sensor space, a significant decrease in prestimulus alpha power was found when the subsequent trial was correctly perceived. A cluster-based permutation t test run on each sensor type on the 500 msec prestimulus period (horizontal dashed line) revealed three significant clusters: one widespread cluster including 52 magnetometers ([−500, 0 msec], p = .026; topographic inset with t values) and two local clusters (15 gradiometers, −456 to −12 msec, p = .023, and 14 gradiometers, −500 to 0 msec, p = .048). The plotted time course was obtained by averaging the significant cluster of magnetometers highlighted in the inset topographic map. (B) The same analysis was performed on source-reconstructed data and revealed a significant cluster (−304 to 0 msec, p = .003) located in the right supramarginal gyrus regions (inset). For illustration, alpha power was corrected by subtracting the mean of the two contrasted performances to the whole epoch. The shaded areas of the time course represents two SEMs.

Figure 4. 

Increased prestimulus alpha power predicts correct performance irrespective of temporal order. In this analysis, all conditions (JND1, JND2, PSS) were combined to assess the relationship between prestimulus alpha power changes and performance (correct in blue; incorrect in pink). (A) In sensor space, a significant decrease in prestimulus alpha power was found when the subsequent trial was correctly perceived. A cluster-based permutation t test run on each sensor type on the 500 msec prestimulus period (horizontal dashed line) revealed three significant clusters: one widespread cluster including 52 magnetometers ([−500, 0 msec], p = .026; topographic inset with t values) and two local clusters (15 gradiometers, −456 to −12 msec, p = .023, and 14 gradiometers, −500 to 0 msec, p = .048). The plotted time course was obtained by averaging the significant cluster of magnetometers highlighted in the inset topographic map. (B) The same analysis was performed on source-reconstructed data and revealed a significant cluster (−304 to 0 msec, p = .003) located in the right supramarginal gyrus regions (inset). For illustration, alpha power was corrected by subtracting the mean of the two contrasted performances to the whole epoch. The shaded areas of the time course represents two SEMs.

Using performance (correct/incorrect) in this analysis could not dissociate between trials in which audiovisual order confirmed or refuted the a priori bias of an individual's toward one or the other temporal order. Specifically, only stimuli with the SOA corresponding to that of the individual's PSS could by definition preserve the ambiguity in sensory evidence and hence leave unchanged the bias of prestimulus activity; to the contrary, JND1 and JND2 provided evidence that could either validate or invalidate the prestimulus bias. Hence, in this analysis, both trials with informative evidence and trials with uncertainty were intermixed leaving uncertain to which extent prestimulus alpha power predicts perceptual bias or some form of preparation toward temporal order decision. To disambiguate this issue and specifically focus on the possibility of a temporal order bias in prestimulus alpha, only trials testing the SOA corresponding to individuals' PSS used for subsequent analyses allowing to preserve maximal perceptual ambiguity on a per individual basis.

Interplay between Prestimulus Alpha Power Fluctuations and Individual Biases

We computed the differences of prestimulus alpha power between the two perceptions of temporal order (i.e., perceiving the sound first minus perceiving the flash first [AV – VA]) on a per individual basis and independently in auditory and visual sensory cortices. A strong trend toward a lower power of spontaneous alpha was observed when perceiving AV as compared with perceiving VA in several posterior and parietal cortical regions (Figure 5A). However, and on average, no significant effects of prestimulus alpha power were found as a function of perceived temporal order in the selected sensory cortices or using a whole-brain analysis corrected for multiple comparisons.

Figure 5. 

Temporal order perception and prestimulus alpha power. (A) The grand-averaged prestimulus alpha power was contrasted between perceiving auditory first (AV) and perceiving visual first (VA) across all individuals. The snapshot of source-reconstructed brain activity was taken at −200 msec before stimulus onset and corresponds to the white cross (10 Hz) indicated in B. The left and right hemispheres are on the left and right panels, respectively. The external and medial views of source reconstructed brain activity are on the top and bottom panels, respectively. Differences in alpha power were mostly seen in sensory and parietal cortices but did not lead to significant differences at the group level when corrected for multiple comparisons. (B) Example of time–frequency differences for two individuals in auditory cortices. Red indicates higher power when the sound was perceived first (AV); blue indicates higher power when the flash was perceived first (VA). As can be seen on these two time–frequency plots, the sign of the prestimulus alpha power differed across individuals suggesting intersubject variability of prestimulus alpha power as a function of perceived temporal order. (C) Individual PSS as a function of alpha power differences (AV − VA). Each point in the correlation represents an individual. The differences of alpha power between perceiving the sound first (AV) and perceiving the visual event first (VA) were averaged across the right and left auditory cortices. After FDR corrections, latencies showing significant power differences ranged from −420 to −192 msec in the auditory cortices (left). No significant correlations were found in visual cortices (right). These results show a significant correlation between the changes in prestimulus alpha power in auditory cortices and the perceived audiovisual order. Irrespective of the signed value of the power difference or PSS, the larger the power difference, the larger the individual's PSS. (D) The difference of source estimate amplitudes between perceived order (AV − VA) was computed individually from 50 to 200 msec poststimulus onset (time window covering the period of evoked responses) and averaged across hemispheres. A Pearson's correlation was run on each time point between the obtained amplitude difference and the PSS. Gray cluster are significant uncorrected p values of the correlation (p < .05), and red clusters are significant FDR-corrected p values. A significant correlation was found in the auditory cortices from 110 to 138 msec. The correlation between differences in source estimates amplitudes and PSS is plotted for this time window. Each point corresponds to a participant.

Figure 5. 

Temporal order perception and prestimulus alpha power. (A) The grand-averaged prestimulus alpha power was contrasted between perceiving auditory first (AV) and perceiving visual first (VA) across all individuals. The snapshot of source-reconstructed brain activity was taken at −200 msec before stimulus onset and corresponds to the white cross (10 Hz) indicated in B. The left and right hemispheres are on the left and right panels, respectively. The external and medial views of source reconstructed brain activity are on the top and bottom panels, respectively. Differences in alpha power were mostly seen in sensory and parietal cortices but did not lead to significant differences at the group level when corrected for multiple comparisons. (B) Example of time–frequency differences for two individuals in auditory cortices. Red indicates higher power when the sound was perceived first (AV); blue indicates higher power when the flash was perceived first (VA). As can be seen on these two time–frequency plots, the sign of the prestimulus alpha power differed across individuals suggesting intersubject variability of prestimulus alpha power as a function of perceived temporal order. (C) Individual PSS as a function of alpha power differences (AV − VA). Each point in the correlation represents an individual. The differences of alpha power between perceiving the sound first (AV) and perceiving the visual event first (VA) were averaged across the right and left auditory cortices. After FDR corrections, latencies showing significant power differences ranged from −420 to −192 msec in the auditory cortices (left). No significant correlations were found in visual cortices (right). These results show a significant correlation between the changes in prestimulus alpha power in auditory cortices and the perceived audiovisual order. Irrespective of the signed value of the power difference or PSS, the larger the power difference, the larger the individual's PSS. (D) The difference of source estimate amplitudes between perceived order (AV − VA) was computed individually from 50 to 200 msec poststimulus onset (time window covering the period of evoked responses) and averaged across hemispheres. A Pearson's correlation was run on each time point between the obtained amplitude difference and the PSS. Gray cluster are significant uncorrected p values of the correlation (p < .05), and red clusters are significant FDR-corrected p values. A significant correlation was found in the auditory cortices from 110 to 138 msec. The correlation between differences in source estimates amplitudes and PSS is plotted for this time window. Each point corresponds to a participant.

However, when looking at the prestimulus time–frequency differences as a function of perceived temporal order for the same physical stimulus, systematic differences in the alpha power were not occurring in the same direction across individuals. As illustrated in Figure 5B, when one individual showed less prestimulus alpha power in auditory cortices when reporting perceiving AV as compared with when perceiving VA (top), another participant would show the opposite pattern with a stronger prestimulus alpha power when reporting perceiving AV as compared with VA (bottom). Considering the observed interindividual variability in PSS (Figure 2B and C), we asked whether the interindividual variability in prestimulus alpha power differences was indicative of an individual's bias toward perceiving one or the other temporal order in the audiovisual sequence.

To test this, we first correlated each individual's PSS with the individual's prestimulus alpha power difference separately for visual and auditory cortices (Figure 5C). A positive correlation between the difference of alpha power as a function of perceived order (AV − VA) and the individual's PSS was found in auditory cortices between −420 and −192 msec before the first stimulus onset (Figure 5C, left inset). This result indicated that a decrease of prestimulus alpha power in auditory cortex could predict perceiving the sound first (AV) if and only if the individual was biased toward perceiving visual first (negative PSS, requiring the presentation of the sound first for audiovisual simultaneity perception). Conversely, a decrease in prestimulus alpha power in auditory cortices predicted perceiving the visual event first if and only if the participant was biased toward perceiving auditory first (positive PSS, requiring the presentation of a visual event first for audiovisual simultaneity perception). In other words, a decrease in prestimulus alpha power in the contrast (AV − VA < 0) indicated that the participant would correctly report the sound as occurring first, whereas an increase (AV − VA > 0) indicated that the participant would correctly report the visual event as first. This effect was not observed in visual cortices.

A simpler alternative for the interpretation of fluctuations in prestimulus alpha power would be that such power changes correlate with the individual's performance. However, we contend that this would be an incorrect interpretation of results reported in Figure 5C. If prestimulus alpha power solely reflected an individual's performance, we would not observe a linear relationship between prestimulus alpha power and individual PSS. Rather, we should observe a categorical pattern with the negative PSS participants (blue) showing negative alpha power in the contrast perceiving AV (correct) versus perceiving VA (incorrect) and the positive PSS participants (green) showing positive alpha power for the same contrast perceiving AV (incorrect) versus perceiving VA (correct). Instead, as seen in Figure 5C, we report a significant linear relationship between alpha power and PSS, suggesting that the magnitude of the prestimulus alpha power changes captures the perceived temporal distance between the auditory and the visual event. To further counteract the simpler explanation of prestimulus alpha power solely reflecting performance, the grand-averaged prestimulus alpha power was computed using all PSS trials, in the same prestimulus window used in analysis reported in Figure 5C (−420 to −192 msec) and in the same cortical label. The correlation between individual performance and prestimulus alpha power was not significant (r = .27, p = .403).

To investigate further this linear relationship, we looked at the effect of alpha power changes on subsequent auditory evoked responses. Prestimulus alpha power has typically been shown to correlate with the amplitude of the following evoked responses so that increases in alpha power tend to be followed by larger sensory evoked responses (e.g., van Dijk et al., 2008; Sauseng et al., 2005; Ergenoglu et al., 2004). We thus tested whether the amplitude of the source estimates in sensory cortices followed a pattern similar to the one found for prestimulus alpha with PSS. On a per individual basis, we computed the amplitude differences of the auditory current source estimates and correlated the obtained amplitude difference with the individual PSS for each time points between 50 and 200 msec poststimulus onset (Figure 5D). A significant correlation between the amplitude difference in the early auditory source estimates and the individual PSS was found between 110 and 138 msec (Figure 5D; red: FDR-corrected p values). This correlation indicated that larger auditory source estimates were associated with the correct perception of audiovisual temporal order, consistent with the preceding variation of prestimulus alpha power.

Altogether, these results suggested that changes in prestimulus alpha power and evoked source amplitudes were only interpretable insofar as an individual's PSS is known. The interpretation of these results could lend themselves to a prediction of performance on the task but additional observation of the correlations between alpha power differences (Figure 5C) and auditory source amplitude differences (Figure 5D) with individual PSS also showed that the signed magnitude difference in both indices was predictive of an individual's PSS: for instance, the individual's with the most negative PSS also showed the lowest alpha power (and highest auditory source amplitude increase) when reporting perceiving AV as compared with VA.

No Systematic Differences in the Phase of Ongoing Neuronal Oscillations as a Function of Perceived Order

To test the hypothesis of temporal encoding in the phase of neural oscillations, the BI contrasted trials in which participants reported perceiving AV and trials in which participants reported perceiving VA on a per individual basis. Across all frequencies, no positive cluster in the prestimulus period was found. Significant negative clusters were found coinciding with the onset of the first physically presented stimulus. A cluster analysis ran over the full time period of −500 to 1000 msec relative to the onset of the first stimulus revealed significant negative clusters in all frequency bands under 30 Hz at the onset of the evoked responses in both auditory and visual cortices (Figure 6A). A whole-brain analysis on a per frequency basis revealed significant clusters at the latency of the evoked responses in several brain regions; these are illustrated in Figure 6B for theta, alpha, and beta. The significant negative BI meant that the PLV in one condition was stronger than in another condition. This was observed in both auditory and visual cortices. However, a closer look at the systematicity of the BI difference across individuals, for instance, in the alpha band (Figure 6C), revealed no clear distinction as a function of the individual's PSS. The lack of conclusiveness with BI quantifications converges with the results obtained in the differences of evoked source amplitudes showing that the direction of the effects are individually based and need to take into account the individual's temporal order bias.

Figure 6. 

BI and PLVs. (A) BI in auditory (left) and visual (right) cortices contrasting perceiving the sound first and perceiving the visual event first (AV − VA). A significant negative BI was found at the time of the evoked responses (lower panels reporting corrected t values). Specifically, the PLVs when perceiving the sound first (AV) were smaller than when perceiving the visual event first (VA). (B) Source estimations of the BI observed across frequency and brain regions for theta, alpha, and high beta (the lower the frequency, the less reliable the latency due to the temporal spread). The peak differences in high-frequency region coincides well with the timing of the early evoked responses. (C) Differences of PLVs on a per individual basis sorted as a function of increasing PSS in the auditory and visual cortices (left and right panels, respectively). The changes in PLV poststimulus onset were not correlated with PSS. This is consistent with the fact that the BI does not take into account the directionality of the PLV differences. Blue shading are individuals with negative PSS, and green shading are individuals with negative PSS (cf. Figure 4). The lack of conclusiveness with BI quantifications is consistent with the results obtained in the differences of evoked source amplitudes showing that the interpretability of the effects need to incorporate the individual's temporal order bias or PSS.

Figure 6. 

BI and PLVs. (A) BI in auditory (left) and visual (right) cortices contrasting perceiving the sound first and perceiving the visual event first (AV − VA). A significant negative BI was found at the time of the evoked responses (lower panels reporting corrected t values). Specifically, the PLVs when perceiving the sound first (AV) were smaller than when perceiving the visual event first (VA). (B) Source estimations of the BI observed across frequency and brain regions for theta, alpha, and high beta (the lower the frequency, the less reliable the latency due to the temporal spread). The peak differences in high-frequency region coincides well with the timing of the early evoked responses. (C) Differences of PLVs on a per individual basis sorted as a function of increasing PSS in the auditory and visual cortices (left and right panels, respectively). The changes in PLV poststimulus onset were not correlated with PSS. This is consistent with the fact that the BI does not take into account the directionality of the PLV differences. Blue shading are individuals with negative PSS, and green shading are individuals with negative PSS (cf. Figure 4). The lack of conclusiveness with BI quantifications is consistent with the results obtained in the differences of evoked source amplitudes showing that the interpretability of the effects need to incorporate the individual's temporal order bias or PSS.

DISCUSSION

In this MEG study, we report evidence that the degree of synchronization in prestimulus alpha regulates an individual's performance in a TOJ task: For individuals with a negative PSS, prestimulus auditory alpha power was lower when a sound was perceived first than when a visual event was perceived first; conversely, for individuals with a positive PSS, prestimulus auditory alpha power was higher when a sound was perceived first than when a visual event was perceived first. Hence, an individual's perceived temporal order could only be inferred from prestimulus alpha power if and only if the individual's psychological bias (PSS) toward perceiving one or the other sensory modality was known in advance. This result suggests that the interpretability of prestimulus alpha power fluctuations for temporal order perception is a function of an individual's intrinsic bias. Second, we observed that the signed amplitude differences in prestimulus alpha power was correlated with the individual's temporal order bias, further suggesting that prestimulus alpha power does not solely predict performance but rather the extent to which the intrinsic bias must be counteracted to produce a correct response. Third, we found no evidence for individual alpha peak correlations with audiovisual temporal order perception (as indexed by PSS or JND). Fourth, in the absence of sensory entrainment, we found no evidence for a spontaneous preferential phase of ongoing oscillatory activity that would systematically indicate an individual's perceived temporal order. We discuss below the implications of our findings for existing interpretations on the role of spontaneous brain rhythms on the temporal sequencing of (multi)sensory information.

The perception of temporal order across sensory modalities is traditionally seen as being dominated by endogenous attentional orienting considering that the attended stimulus (or sensory modality) is perceived earlier than the unattended stimulus, following Titchener's law of prior entry (Spence & Parise, 2010). In TOJ tasks, manipulating attention has been systematically shown to shift an individual's PSS so that the attended stimulus or sensory modality can be perceived more often as occurring first than the unattended stimulus or sensory modality (Spence, Shore, & Klein, 2001). In the current experimental setup, participants' attention was not explicitly manipulated and was instead predicted to freely fluctuate from one to the other sensory modality. To further increase uncertainty on a per individual basis, the temporal delays between audiovisual events corresponded to the individual's measured perceptual threshold (PSS).

In a first working hypothesis (Figure 1A), we predicted that when participants' attention was biased toward audition (vision), the auditory (visual) event would be most often reported first. One index of attentional fluctuation is the degree of alpha (de)synchronization so that a local decrease in alpha power indexes a decrease of sensory inhibition (Klimesch, 2012; Jensen & Mazaheri, 2010; Sauseng et al., 2005): Previous reports have shown that a decrease of prestimulus alpha power in sensory cortices was correlated with better performance (Mazaheri et al., 2014; Yamagishi, Callan, Anderson, & Kawato, 2008; Hanslmayr et al., 2007) whereas an increase of prestimulus alpha power could yield higher illusory reports (Lange, Keil, Schnitzler, van Dijk, Weisz, 2014). Our first set of results (Figures 4 and 7) supports the notion that, on average, lower prestimulus alpha power is associated with correct performance irrespective of the actually perceived order.

Figure 7. 

Summary of findings. Using an audiovisual TOJ task whose difficulty was calibrated on each individual's temporal order bias (PSS), we report two main findings. First, the relative changes of prestimulus alpha power in auditory cortices is indicative of an individual's performance (1). Importantly, performance in TOJ specifies the perceived sequence of sensory events. Hence, these results suggest that fluctuations in the power of prestimulus auditory alpha may play an important role in the temporal sequencing of sensory events. Crucially, however, a power increase of alpha was insufficient to predict a participant's perceived temporal order considering that a participant with a negative PSS (blue) showing a relative decrease of alpha power in the contrast (AV − VA) will correctly report perceiving the sound first (i.e., AV percept) whereas a participant with a negative PSS (green) showing the same relative decrease using the same contrast will have perceived the visual event as being first (i.e., VA percept). Additionally, the signed magnitude difference in this contrast was indicative of an individual's PSS so that the largest changes in prestimulus auditory alpha synchronization corresponded to the largest PSS values (2). This strongly suggests that prestimulus alpha power desynchronization does not only reflect general attentional mechanisms but rather endogenous mechanisms compensating for an individual's structural biases.

Figure 7. 

Summary of findings. Using an audiovisual TOJ task whose difficulty was calibrated on each individual's temporal order bias (PSS), we report two main findings. First, the relative changes of prestimulus alpha power in auditory cortices is indicative of an individual's performance (1). Importantly, performance in TOJ specifies the perceived sequence of sensory events. Hence, these results suggest that fluctuations in the power of prestimulus auditory alpha may play an important role in the temporal sequencing of sensory events. Crucially, however, a power increase of alpha was insufficient to predict a participant's perceived temporal order considering that a participant with a negative PSS (blue) showing a relative decrease of alpha power in the contrast (AV − VA) will correctly report perceiving the sound first (i.e., AV percept) whereas a participant with a negative PSS (green) showing the same relative decrease using the same contrast will have perceived the visual event as being first (i.e., VA percept). Additionally, the signed magnitude difference in this contrast was indicative of an individual's PSS so that the largest changes in prestimulus auditory alpha synchronization corresponded to the largest PSS values (2). This strongly suggests that prestimulus alpha power desynchronization does not only reflect general attentional mechanisms but rather endogenous mechanisms compensating for an individual's structural biases.

Additionally, a decrease in prestimulus alpha power was predicted in the sensory cortices corresponding to the first stimulus being correctly reported, namely, for participants with a positive PSS and reporting the visual event as being first, we expected a decrease (increase) of alpha power in visual (auditory) cortices whereas for participants with a negative PSS, reporting the sound as being first would show a decrease (increase) of alpha power in auditory (visual) cortices. In the contrast perceiving (AV − VA), we thus expected consistent decrease of prestimulus alpha power in auditory cortices and increase of alpha power in visual cortices. This is not what we observed.

First, significant changes of alpha power were exclusively observed in auditory cortices. One possibility for the absence of prestimulus alpha power changes in visual cortices in our experiment is the consistency of attentional allocation to the site of visual stimulation (Slagter, Prinssen, Reteig, & Mazaheri, 2016). Another possibility is that sounds were effectively presented first for a majority of participants, thereby biasing the variability in allocation of attention to auditory inputs; future investigations including a larger number of participants with a positive PSS would be needed to see whether the sole implication of auditory cortices is effectively key for event timing (Kösem et al., 2014; Kanai, Lloyd, Bueti, & Walsh, 2011). Alternatively, this observation may suggest that participants did not readily switch or change their attention across sensory modalities and instead adopted a strategy to use auditory information as the main decisional anchor. This would be coherent with the observation that, across individuals, lowest desynchronization in auditory cortices was correlated with being biased toward vision (Figure 7, blue), requiring participants with the largest bias (or negative PSS) to pay more attention to audition and, conversely, those with the largest positive PSS to pay less attention to audition. For instance, splitting participants according to their preferred sensory modalities, that is, according to whether they performed best in audition or in vision during unimodal presentation has previously been shown to dissociate neural signatures of audiovisual integration (Giard & Peronnet, 1999). In this study, the authors showed, in agreement with the inverse effectiveness principle (Stein & Meredith, 1993), that multisensory integration yielded enhanced early latency cortical responses in the nondominant sensory modality. Here, endogenous modulations of spontaneous alpha synchronization could similarly be seen as counteracting an individual's internal bias (PSS) in temporal order so that the largest relative decrease in alpha power (and associated largest relative increase in the amplitude evoked response) would yield correct performance. Overall, our results are consistent with the role assigned to the power fluctuations of spontaneous alpha oscillations in temporal attention (van Diepen, Cohen, Denys, & Mazaheri, 2015; Klimesch, 2012; Hanslmayr, Gross, Klimesch, & Shapiro, 2011), thereby attentional fluctuations regulate the temporal information flow in sensory regions. Our results converge with the notion that sensory alpha reflects feedback modulatory signals during perceptual analysis (Fries, 2015; van Kerkoerle et al., 2014) and, more specifically, that alpha power may modulate neural excitability and influence perceptual biases (Iemi, Chaumon, Crouzet, & Busch, 2016).

Second, the observed coupling between changes in prestimulus alpha power and subsequent changes in phase-locking and amplitude of the auditory evoked responses was consistent with recent discussions on the sensitivity of phase information as measured with PLV (Canavier, 2015; Ding & Simon, 2013) and BI (vanRullen, 2016). We found increased variability in phase locking in both sensory cortices when participants reported perceiving the visual stimulus first. Although this could suggest that the observed prestimulus fluctuations and the increase trial-to-trial variability may result from varying modes of engagement in the task (Baird, Smallwood, Lutz, & Schooler, 2014), the systematic changes in the power of spontaneous alpha fluctuations may not solely reflect task engagement and task performance. The range of spontaneous alpha power fluctuations on a per individual basis was proportional to the individual's temporal order bias so that the largest the difference of power between the two possible order perceptions given the same physical stimulation, the largest the individual's PSS (Figure 7). In other words, the less sensitive an individual was to audiovisual delays, the larger the changes in synchronization of the prestimulus alpha power. Additionally, the decrease in prestimulus alpha power occurred when participants reported their nonpreferred sensory modality as occurring first. One tentative working hypothesis is thus that the degree of prestimulus sensory alpha synchronization reflects compensatory mechanisms for the temporal limitations imposed by structural biases. Specifically, lower alpha power in auditory cortex would yield a stronger influence of structural priors, thereby perceived temporal order would reflect the individual's bias whereas higher alpha power would more effectively weight sensory evidence in the decision process. This interpretation would be compatible with the findings that reduced alpha power may not always predict better visual processing but rather enhanced excitability (e.g., Mazaheri et al., 2014; Lange, Oostenveld, & Fries, 2013) in line with inhibitory alpha frameworks (Jensen et al., 2012; Klimesch et al., 2007). The notion that higher prestimulus alpha power may reflect ongoing inhibition of sensory evidence so that structural priors may have a stronger weight in the perceptual decision is also well in line with recent findings (e.g., Iemi et al., 2016; Chaumon & Busch, 2014; van Dijk et al., 2008).

Changes in the phase response entrained by audiovisual stimuli have been shown to follow an individual's temporal order perception (Kösem et al., 2014), suggesting possible endogenous control of the oscillatory phase response in spite of entrainment—that is, nonstationarity (van Wassenhove, 2016). The endogenous control of oscillatory phase responses have previously been reported in temporal expectation and attention tasks (Samaha et al., 2015; Cravo, Rohenkohl, Wyart, & Nobre, 2011), with a direct link between perceived timing and alpha phase (Chakravarthi & Vanrullen, 2012; Gho & Varela, 1988; Varela et al., 1981). A recent study has also provided evidence supporting the gamma dependency on the theta phase in sequence order (Heusser, Poeppel, Ezzyat, & Davachi, 2016). As the relationship between gamma power and the phase of theta and alpha neuronal oscillations may be crucial to determine the order of events, phase dependency of order perception on the phase of ongoing theta or alpha neuronal oscillations was expected in our task (Figure 1C). However, no systematic evidence supported the role of oscillatory phase in audiovisual temporal order, and we discuss below several possible reasons.

First, one important observation is that previous results tested visual perception and implicit timing (Chakravarthi & Vanrullen, 2012; Gho & Varela, 1988; Varela et al., 1981) whereas our study tested conscious temporal order across sensory modalities. Additionally, we specifically focused on prestimulus activity and reported that phase diversity, at the moment of sensory stimulation, did impact PLVs across the spectrum, but in a manner that seemed individually specific. Second, previous results have tested sequence order in memory (Heusser et al., 2016), whereas our study tested the perceptual encoding stage of sequence order, leaving open the possibility that a different strategy may be used during the encoding of sensory sequences than during memory maintenance of these sequences. Third, a recent EEG study reported differences in the phase of prestimulus alpha oscillations using a very similar design as ours, but perhaps importantly, asking participants to report the simultaneity of audiovisual events (Ikumi, 2016; Ikumi, Torralba, Ruzzoli, & Soto-Faraco, 2015): One possibility for the differences in findings is that alpha phase and power effects in prestimulus oscillatory activity for the same physical stimulation may be strongly driven by task demands. For instance, the resetting of theta cycles have been shown to discriminate between sensory integration and sensory segregation, providing relevant cues on the extent of temporal organization in the brain (Wutz, Muschter, van Koningsbruggen, Weisz, & Melcher, 2016). In simultaneity tasks, the emphasis is put on the integration of multisensory information, whereas in TOJ tasks, the emphasis is put on the segregation of audiovisual events to elicit a conscious representation of temporal sequence. We thus hope that future studies will help provide direct contrasts between these two forms of temporal encoding multisensory information.

Acknowledgments

This work was supported by ERC-YStG-263584 to V. van Wassenhove. We thank members of UNIACT for their help in recruiting and scheduling participants. Preliminary data were presented at SFN Washington DC 2014, OHBM Honolulu 2015, and Rovereto Attention Workshop 2015. We thank two anonymous reviewers for the quality and insightfulness of their comments on earlier versions of the manuscript.

Reprint requests should be sent to Laetitia Grabot, Cognitive Neuroimaging Unit, CEA DRF/I2BM, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France, or via e-mail: laetitia.grabot@gmail.com.

REFERENCES

Baird
,
B.
,
Smallwood
,
J.
,
Lutz
,
A.
, &
Schooler
,
J. W.
(
2014
).
The decoupled mind: Mind-wandering disrupts cortical phase-locking to perceptual events
.
Journal of Cognitive Neuroscience
,
26
,
2596
2607
.
Benevento
,
A.
,
Fallom
,
J.
,
Davis
,
B. J.
, &
Rezak
,
M.
(
1977
).
Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey
.
Experimental Neurology
,
57
,
849
872
.
Bonnefond
,
M.
, &
Jensen
,
O.
(
2012
).
Alpha oscillations serve to protect working memory maintenance against anticipated distracters
.
Current Biology
,
22
,
1969
1974
.
Busch
,
N. A.
,
Dubois
,
J.
, &
VanRullen
,
R.
(
2009
).
The phase of ongoing EEG oscillations predicts visual perception
.
Journal of Neuroscience
,
29
,
7869
7876
.
Canavier
,
C. C.
(
2015
).
Phase-resetting as a tool of information transmission
.
Current Opinion in Neurobiology
,
31
,
206
213
.
Cecere
,
R.
,
Rees
,
G.
, &
Romei
,
V.
(
2015
).
Individual differences in alpha frequency drive crossmodal illusory perception
.
Current Biology
,
25
,
231
235
.
Chakravarthi
,
R.
, &
Vanrullen
,
R.
(
2012
).
Conscious updating is a rhythmic process
.
Proceedings of the National Academy of Sciences, U.S.A.
,
109
,
10599
10604
.
Chaumon
,
M.
, &
Busch
,
N. A.
(
2014
).
Prestimulus neural oscillations inhibit visual perception via modulation of response gain
.
Journal of Cognitive Neuroscience
,
26
,
2514
2529
.
Cravo
,
A. M.
,
Rohenkohl
,
G.
,
Wyart
,
V.
, &
Nobre
,
A. C.
(
2011
).
Endogenous modulation of low frequency oscillations by temporal expectations
.
Journal of Neurophysiology
,
106
,
2964
2972
.
Dale
,
A. M.
,
Liu
,
A. K.
,
Fischl
,
B. R.
,
Buckner
,
R. L.
,
Belliveau
,
J. W.
,
Lewine
,
J. D.
, et al
(
2000
).
Dynamic statistical parametric mapping: Combining fMRI and MEG for high-resolution imaging of cortical activity
.
Neuron
,
26
,
55
67
.
Ding
,
N.
, &
Simon
,
J. Z.
(
2013
).
Power and phase properties of oscillatory neural responses in the presence of background activity
.
Journal of Computational Neuroscience
,
34
,
337
343
.
Ergenoglu
,
T.
,
Demiralp
,
T.
,
Bayraktaroglu
,
Z.
,
Ergen
,
M.
,
Beydagi
,
H.
, &
Uresin
,
Y.
(
2004
).
Alpha rhythm of the EEG modulates visual detection performance in humans
.
Brain Research, Cognitive Brain Research
,
20
,
376
383
.
Freeman
,
E. D.
,
Ipser
,
A.
,
Palmbaha
,
A.
,
Paunoiu
,
D.
,
Brown
,
P.
,
Lambert
,
C.
, et al
(
2013
).
Sight and sound out of synch: Fragmentation and renormalisation of audiovisual integration and subjective timing
.
Cortex
,
49
,
2875
2887
.
Fries
,
P.
(
2015
).
Rhythms for cognition: Communication through coherence
.
Neuron
,
88
,
220
235
.
Fujisaki
,
W.
,
Shimojo
,
S.
,
Kashino
,
M.
, &
Nishida
,
S.
(
2004
).
Recalibration of audiovisual simultaneity
.
Nature Neuroscience
,
7
,
773
778
.
Genovese
,
C. R.
,
Lazar
,
N. A.
, &
Nichols
,
T.
(
2002
).
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
.
Neuroimage
,
15
,
870
878
.
Gho
,
M.
, &
Varela
,
F. J.
(
1988
).
A quantitative assessment of the dependency of the visual temporal frame upon the cortical rhythm
.
Journal of Physiology (Paris)
,
83
,
95
101
.
Giard
,
M. H.
, &
Peronnet
,
F.
(
1999
).
Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study
.
Journal of Cognitive Neuroscience
,
11
,
473
490
.
Grabot
,
L.
, &
van Wassenhove
,
V.
(
2017
).
Time order as psychological bias
.
Psychological Science
,
28
,
670
678
.
Gramfort
,
A.
,
Luessi
,
M.
,
Larson
,
E.
,
Engemann
,
D. A.
,
Strohmeier
,
D.
,
Brodbeck
,
C.
, et al
(
2013
).
MEG and EEG data analysis with MNE-Python
.
Frontiers in Neuroscience
,
7
,
267
.
Gramfort
,
A.
,
Luessi
,
M.
,
Larson
,
E.
,
Engemann
,
D. A.
,
Strohmeier
,
D.
,
Brodbeck
,
C.
, et al
(
2014
).
MNE software for processing MEG and EEG data
.
Neuroimage
,
86
,
446
460
.
Haegens
,
S.
,
Cousijn
,
H.
,
Wallis
,
G.
,
Harrison
,
P. J.
, &
Nobre
,
A. C.
(
2014
).
Inter- and intra-individual variability in alpha peak frequency
.
Neuroimage
,
92
,
46
55
.
Hanslmayr
,
S.
,
Aslan
,
A.
,
Staudigl
,
T.
,
Klimesch
,
W.
,
Herrmann
,
C. S.
, &
Bäuml
,
K. H.
(
2007
).
Prestimulus oscillations predict visual perception performance between and within subjects
.
Neuroimage
,
37
,
1465
1473
.
Hanslmayr
,
S.
,
Gross
,
J.
,
Klimesch
,
W.
, &
Shapiro
,
K. L.
(
2011
).
The role of α oscillations in temporal attention
.
Brain Research Reviews
,
67
,
331
343
.
Hanson
,
J. V. M.
,
Heron
,
J.
, &
Whitaker
,
D.
(
2008
).
Recalibration of perceived time across sensory modalities
.
Experimental Brain Research
,
185
,
347
352
.
Heusser
,
A. C.
,
Poeppel
,
D.
,
Ezzyat
,
Y.
, &
Davachi
,
L.
(
2016
).
Episodic sequence memory is supported by a theta-gamma phase code
.
Nature Neuroscience
,
19
,
1374
1380
.
Iemi
,
L.
,
Chaumon
,
M.
,
Crouzet
,
S. M.
, &
Busch
,
N. A.
(
2016
).
Spontaneous neural oscillations bias perception by modulating baseline excitability
.
Journal of Neuroscience
,
37
,
807
819
.
Ikumi
,
N.
,
Torralba
,
M.
,
Ruzzoli
,
M.
, &
Soto-Faraco
,
S.
(
2015
).
Ongoing alpha phase explains perceptual variability in audio-visual synchrony judgments
.
Paper presented at the Rovereto Attention Worskhop, Rovereto, Italy
.
Ikumi
,
N. M.
(
2016
).
The effects of cognitive factors on cross-modal synchrony perception
.
Barcelona
:
Universitat Pompeu Fabra
.
Ipser
,
A.
,
Agolli
,
V.
,
Bajraktari
,
A.
,
Al-Alawi
,
F.
,
Djaafara
,
N.
, &
Freeman
,
E. D.
(
2017
).
Sight and sound persistently out of synch: Stable individual differences in audiovisual synchronisation revealed by implicit measures of lip-voice integration
.
Scientific Reports
,
7
,
46413
.
Jensen
,
O.
,
Bonnefond
,
M.
, &
VanRullen
,
R.
(
2012
).
An oscillatory mechanism for prioritizing salient unattended stimuli
.
Trends in Cognitive Sciences
,
16
,
200
206
.
Jensen
,
O.
, &
Mazaheri
,
A.
(
2010
).
Shaping functional architecture by oscillatory alpha activity: Gating by inhibition
.
Frontiers in Human Neuroscience
,
4
,
186
.
Kanai
,
R.
,
Lloyd
,
H.
,
Bueti
,
D.
, &
Walsh
,
V.
(
2011
).
Modality-independent role of the primary auditory cortex in time estimation
.
Experimental Brain Research
,
209
,
465
471
.
Klimesch
,
W.
(
2012
).
α-Band oscillations, attention, and controlled access to stored information
.
Trends in Cognitive Sciences
,
16
,
606
617
.
Klimesch
,
W.
,
Sauseng
,
P.
, &
Hanslmayr
,
S.
(
2007
).
EEG alpha oscillations: The inhibition-timing hypothesis
.
Brain Research Reviews
,
53
,
63
88
.
Kösem
,
A.
,
Gramfort
,
A.
, &
van Wassenhove
,
V.
(
2014
).
Encoding of event timing in the phase of neural oscillations
.
Neuroimage
,
92C
,
274
284
.
Kösem
,
A.
, &
van Wassenhove
,
V.
(
2012
).
Temporal structure in audiovisual sensory selection
.
PLoS One
,
7
,
e40936
.
Lange
,
J.
,
Keil
,
J.
,
Schnitzer
,
A.
,
van Dijk
,
H.
, &
Weisz
,
N.
(
2014
).
The role of alpha oscillations for illusory perceptions
.
Behavioural Brain Research
,
271
,
294
301
.
Lange
,
J.
,
Oostenveld
,
R.
, &
Fries
,
P.
(
2013
).
Reduced occipital alpha power indexes enhanced excitability rather than improved visual perception
.
Journal of Neuroscience
,
33
,
3212
3220
.
Lashley
,
K. S.
(
1951
).
The problem of serial order in behavior
. In
L. A.
Jeffress
(Ed.),
Cerebral mechanisms in behavior
(pp.
112
136
).
New York
:
John Wiley & Sons
.
Lisman
,
J. E.
, &
Jensen
,
O.
(
2013
).
The θ-γ neural code
.
Neuron
,
77
,
1002
1016
.
Martin
,
J.-R.
,
Kösem
,
A.
, &
van Wassenhove
,
V.
(
2015
).
Hysteresis in audiovisual synchrony perception
.
PLoS One
,
10
,
e0119365
.
Mathewson
,
K. E.
,
Fabiani
,
M.
,
Gratton
,
G.
,
Beck
,
D. M.
, &
Lleras
,
A.
(
2010
).
Rescuing stimuli from invisibility: Inducing a momentary release from visual masking with pre-target entrainment
.
Cognition
,
115
,
186
191
.
Mathewson
,
K. E.
,
Gratton
,
G.
,
Fabiani
,
M.
,
Beck
,
D. M.
, &
Ro
,
T.
(
2009
).
To see or not to see: Prestimulus alpha phase predicts visual awareness
.
Journal of Neuroscience
,
29
,
2725
2732
.
Mazaheri
,
A.
,
van Schouwenburg
,
M. R.
,
Dimitrijevic
,
A.
,
Denys
,
D.
,
Cools
,
R.
, &
Jensen
,
O.
(
2014
).
Region-specific modulations in oscillatory alpha activity serve to facilitate processing in the visual and auditory modalities
.
Neuroimage
,
87
,
356
362
.
McDonald
,
J. J.
,
Teder-Salejarvi
,
W. A.
,
Di Russo
,
F.
, &
Hillyard
,
S. A.
(
2005
).
Neural basis of auditory-induced shifts in visual time-order perception
.
Nature Neuroscience
,
8
,
1197
1202
.
Meredith
,
M. A.
,
Nemitz
,
J. W.
, &
Stein
,
B. E.
(
1987
).
Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors
.
Journal of Neuroscience
,
7
,
3215
3229
.
Pöppel
,
E.
(
2009
).
Pre-semantically defined temporal windows for cognitive processing
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
364
,
1887
1896
.
Pöppel
,
E.
,
Schill
,
K.
, &
von Steinbüchel
,
N.
(
1990
).
Sensory integration within temporally neutral systems states: A hypothesis
.
Naturwissenschaften
,
77
,
89
91
.
Rees
,
A.
,
Green
,
G. G.
, &
Kay
,
R. H.
(
1986
).
Steady-state evoked responses to sinusoidally amplitude-modulated sounds recorded in man
.
Hear Res
,
23
,
123
133
.
Regan
,
D.
(
1966
).
Some characteristics of average steady-state and transient responses evoked by modulated light
.
Electroencephalography and Clinical Neurophysiology
,
20
,
238
248
.
Sadaghiani
,
S.
, &
Kleinschmidt
,
A.
(
2016
).
Brain networks and α-oscillations: Structural and functional foundations of cognitive control
.
Trends in Cognitive Sciences
,
20
,
805
817
.
Samaha
,
J.
,
Bauer
,
P.
,
Cimaroli
,
S.
, &
Postle
,
B. R.
(
2015
).
Top–down control of the phase of alpha-band oscillations as a mechanism for temporal prediction
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
8439
8444
.
Samaha
,
J.
, &
Postle
,
B. R.
(
2015
).
The speed of alpha-band oscillations predicts the temporal resolution of visual perception
.
Current Biology
,
25
,
2985
2990
.
Sauseng
,
P.
,
Klimesch
,
W.
,
Stadler
,
W.
,
Schabus
,
M.
,
Doppelmayr
,
M.
,
Hanslmayr
,
S.
, et al
(
2005
).
A shift of visual spatial attention is selectively associated with human EEG alpha activity
.
European Journal of Neuroscience
,
22
,
2917
2926
.
Schroeder
,
C.
, &
Lakatos
,
P.
(
2009
).
Low-frequency neuronal oscillations as instruments of sensory selection
.
Trends in Neurosciences
,
32
,
9
18
.
Slagter
,
H. A.
,
Prinssen
,
S.
,
Reteig
,
L. C.
, &
Mazaheri
,
A.
(
2016
).
Facilitation and inhibition in attention: Functional dissociation of pre-stimulus alpha activity, P1, and N1 components
.
Neuroimage
,
125
,
25
35
.
Spence
,
C.
, &
Parise
,
C.
(
2010
).
Prior-entry: A review
.
Consciousness and Cognition
,
19
,
364
379
.
Spence
,
C.
,
Shore
,
D. I.
, &
Klein
,
R. M.
(
2001
).
Multisensory prior entry
.
Journal of Experimental Psychology: General
,
130
,
799
832
.
Stefanics
,
G.
,
Hangya
,
B.
,
Hernádi
,
I.
,
Winkler
,
I.
,
Lakatos
,
P.
, &
Ulbert
,
I.
(
2010
).
Phase entrainment of human delta oscillations can mediate the effects of expectation on reaction speed
.
Journal of Neuroscience
,
30
,
13578
13585
.
Stein
,
B. E.
, &
Meredith
,
M. A.
(
1993
).
The merging of the senses
.
Cambridge, MA
:
MIT Press
.
Stekelenburg
,
J.
, &
Vroomen
,
J.
(
2007
).
Neural correlates of multisensory integration of ecologically valid audiovisual events
.
Journal of Cognitive Neuroscience
,
19
,
1964
1973
.
Stone
,
J. V.
,
Hunkin
,
N. M.
,
Porrill
,
J.
,
Wood
,
R.
,
Keeler
,
V.
,
Beanland
,
M.
, et al
(
2001
).
When is now? Perception of simultaneity
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
268
,
31
.
Strauß
,
A.
,
Henry
,
M. J.
,
Scharinger
,
M.
, &
Obleser
,
J.
(
2015
).
Alpha phase determines successful lexical decision in noise
.
Journal of Neuroscience
,
35
,
3256
3262
.
Taulu
,
S.
, &
Simola
,
J.
(
2006
).
Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements
.
Physics in Medicine and Biology
,
51
,
1759
1768
.
Thorne
,
J. D.
,
De Vos
,
M.
,
Viola
,
F. C.
, &
Debener
,
S.
(
2011
).
Cross-modal phase reset predicts auditory task performance in humans
.
Journal of Neuroscience
,
31
,
3853
3861
.
Thorne
,
J. D.
, &
Debener
,
S.
(
2014
).
Look now and hear what's coming: On the functional role of cross-modal phase reset
.
Hearing Research
,
307
,
144
152
.
Uusitalo
,
M. A.
, &
Ilmoniemi
,
R. J.
(
1997
).
Signal-space projection method for separating MEG or EEG into components
.
Medical & Biological Engineering & Computing
,
35
,
135
140
.
Van der Burg
,
E.
,
Orchard-Mills
,
E.
, &
Alais
,
D.
(
2015
).
Rapid temporal recalibration is unique to audiovisual stimuli
.
Experimental Brain Research
,
233
,
53
59
.
van Diepen
,
R. M.
,
Cohen
,
M. X.
,
Denys
,
D.
, &
Mazaheri
,
A.
(
2015
).
Attention and temporal expectations modulate power, not phase, of ongoing alpha oscillations
.
Journal of Cognitive Neuroscience
,
27
,
1573
1586
.
van Dijk
,
H.
,
Schoffelen
,
J. M.
,
Oostenveld
,
R.
, &
Jensen
,
O.
(
2008
).
Prestimulus oscillatory activity in the alpha band predicts visual discrimination ability
.
Journal of Neuroscience
,
28
,
1816
1823
.
Van Eijk
,
R. L.
,
Kohlrausch
,
A.
,
Juola
,
J. F.
, &
van de Par
,
S.
(
2008
).
Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type
.
Attention, Perception, & Psychophysics
,
70
,
955
968
.
van Kerkoerle
,
T.
,
Self
,
M. W.
,
Dagnino
,
B.
,
Gariel-Mathis
,
M. A.
,
Poort
,
J.
,
van der Togt
,
C.
, et al
(
2014
).
Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex
.
Proceedings of the National Academy of Sciences, U.S.A.
,
111
,
14332
14341
.
VanRullen
,
R.
(
2016
).
How to evaluate phase difference between trial groups in ongoing electrophysiological signals
.
Frontiers in Neuroscience
,
10
.
van Wassenhove
,
V.
(
2016
).
Temporal cognition and neural oscillations
.
Current Opinion in Behavioral Sciences
,
8
,
124
130
.
van Wassenhove
,
V.
,
Grant
,
K. W.
, &
Poeppel
,
D.
(
2005
).
Visual speech speeds up the neural processing of auditory speech
.
Proceedings of the National Academy of Sciences, U.S.A.
,
102
,
1181
1186
.
van Wassenhove
,
V.
,
Grant
,
K. W.
, &
Poeppel
,
D.
(
2007
).
Temporal window of integration in auditory-visual speech perception
.
Neuropsychologia
,
45
,
598
607
.
Varela
,
F. J.
,
Toro
,
A.
,
John
,
E. R.
, &
Schwartz
,
E. L.
(
1981
).
Perceptual framing and cortical alpha rhythm
.
Neuropsychologia
,
19
,
675
686
.
Vibell
,
J.
,
Klinge
,
C.
,
Zampini
,
M.
,
Spence
,
C.
, &
Nobre
,
A. C.
(
2007
).
Temporal order is coded temporally in the brain: Early event-related potential latency shifts underlying prior entry in a cross-modal temporal order judgment task
.
Journal of Cognitive Neuroscience
,
19
,
109
120
.
Vroomen
,
J.
, &
Keetels
,
M.
(
2010
).
Perception of intersensory synchrony: A tutorial review
.
Attention, Perception, & Psychophysics
,
72
,
871
884
.
Vroomen
,
J.
,
Keetels
,
M.
,
de Gelder
,
B.
, &
Bertelson
,
P.
(
2004
).
Recalibration of temporal order perception by exposure to audio-visual asynchrony
.
Brain Research, Cognitive Brain Research
,
22
,
32
35
.
Wutz
,
A.
,
Muschter
,
E.
,
van Koningsbruggen
,
M. G.
,
Weisz
,
N.
, &
Melcher
,
D.
(
2016
).
Temporal integration windows in neural processing and perception aligned to saccadic eye movements
.
Current Biology
,
26
,
1659
1668
.
Yamagishi
,
N.
,
Callan
,
D. E.
,
Anderson
,
S. J.
, &
Kawato
,
M.
(
2008
).
Attentional changes in pre-stimulus oscillatory activity within early visual cortex are predictive of human visual performance
.
Brain Research
,
1197
,
115
122
.
Zampini
,
M.
,
Shore
,
D. I.
, &
Spence
,
C.
(
2005
).
Audiovisual prior entry
.
Neuroscience Letters
,
381
,
217
222
.