Psychophysical experiments show that auditory change detection can be disturbed in situations in which listeners have to monitor complex auditory input. We made use of this change deafness effect to segregate the neural correlates of physical change in auditory input from brain responses related to conscious change perception in an fMRI experiment. Participants listened to two successively presented complex auditory scenes, which consisted of six auditory streams, and had to decide whether scenes were identical or whether the frequency of one stream was changed between presentations. Our results show that physical changes in auditory input, independent of successful change detection, are represented at the level of auditory cortex. Activations related to conscious change perception, independent of physical change, were found in the insula and the ACC. Moreover, our data provide evidence for significant effective connectivity between auditory cortex and the insula in the case of correctly detected auditory changes, but not for missed changes. This underlines the importance of the insula/anterior cingulate network for conscious change detection.
Most of the time in everyday life we are exposed to complex sensory stimulation. Detecting changes within this input is essential to cope with the environment. Remarkably, previous studies on visual change detection have demonstrated that even pronounced changes within a complex visual scene can remain unnoticed if they occur after a short interruption of the visual presentation (Simons & Rensink, 2005; Rensink, 2002). Recently, several psychophysical experiments provided evidence for the existence of a similar effect in the auditory modality (Backer & Alain, 2012; Fenn et al., 2011; Vitevitch & Donoso, 2011; Gregg & Samuel, 2008, 2009; Eramudugolla, McAnally, Martin, Irvine, & Mattingley, 2008; Pavani & Turatto, 2008; Eramudugolla, Irvine, McAnally, Martin, & Mattingley, 2005; Vitevitch, 2003). In a similar manner to visual change blindness, this change deafness emerges when listeners have to monitor demanding auditory scenes consisting of several streams (Gregg & Samuel, 2008, 2009; Eramudugolla et al., 2005, 2008; Pavani & Turatto, 2008) or when concentrating on a specific aspect of the auditory input (Fenn et al., 2011; Vitevitch, 2003). Hence, in change deafness paradigms, the same salient change within a complex auditory scene can either be consciously perceived or remain undetected by the listener. This enables the separation of brain activity related to physical change in auditory stimulation from activity induced by conscious perception of change. Following the same idea, previous studies on visual change detection have made use of the change blindness effect to identify correlates of physical changes in visual input and conscious perception of such changes (Beck, Muggleton, Walsh, & Lavie, 2006; Pourtois, De Pretto, Hauert, & Vuilleumier, 2006; Reddy, Quiroga, Wilken, Koch, & Fried, 2006; Pessoa & Ungerleider, 2004; Beck, Rees, Frith, & Lavie, 2001).
Previous work investigating auditory target detection under informational masking report similar early auditory cortex responses to detected and undetected target sounds, indicating that even sounds not consciously perceived are processed at the sensory level in auditory cortex (Königs & Gutschalk, 2012; Gutschalk, Micheyl, & Oxenham, 2008). On the other hand, recent studies indicate that even early auditory cortex responses to physically identical stimuli are modulated by the listeners' perception of sound (Bernasconi et al., 2011; Kilian-Hütten, Valente, Vroomen, & Formisano, 2011), suggesting that both the sensory processing of physical change in auditory stimulation and the conscious perception of change are represented at the level of auditory cortex. However, to our knowledge, no previous study has investigated both effects in one experiment.
Here we present an fMRI experiment aiming to separate BOLD responses related to physical changes in auditory stimulation (independent of conscious change perception) and perception of change (independent of a true physical change) making use of the change deafness effect. We adapted a one-shot change detection paradigm similar to those used in earlier studies on change deafness (Gregg & Samuel, 2008, 2009; Eramudugolla et al., 2005, 2008; Pavani & Turatto, 2008). The complex auditory stimulation consisted of six simultaneously presented streams, differing in rhythm, frequency, and sound source location. Changes in the auditory stimulation were induced by increasing the frequency of one stream, whereas the rest of the scene remained identical.
Change deafness was previously shown to occur despite successful initial stimulus encoding (Gregg & Samuel, 2008), indicating that undetected changes are processed correctly in auditory sensory areas. Therefore, we hypothesized that in our experiment hits (i.e., correctly perceived changes in auditory input) and misses (i.e., unperceived changes in auditory input) should share a common representation in auditory cortex related to similar encoding of the physical stimulus information. In addition, we expected perception-induced responses, both to hits and false alarms (i.e., erroneously perceived changes in the absence of a physical change in auditory input), in auditory cortex and in several nonsensory brain areas including the ACC, the insula, as well as several frontal and parietal regions (Sabri, Liebenthal, Waldron, Medler, & Binder, 2006; Molholm, Martinez, Ritter, Javitt, & Foxe, 2005; Opitz, Schröger, & von Cramon, 2005; Linden et al., 1999). Because previous fMRI data have revealed auditory cortex responses to frequency (or pitch) deviants close to the lateral portion of Heschl's gyrus and Heschl's sulcus (Molholm et al., 2005; Opitz et al., 2005), a region that is also discussed as playing a role in pitch processing per se (Puschmann, Uppenkamp, Kollmeier, & Thiel, 2010; Penagos, Melcher, & Oxenham, 2004; Warren, Uppenkamp, Patterson, & Griffiths, 2003; Patterson, Uppenkamp, Johnsrude, & Griffiths, 2002), we here also assumed that early change-related responses would occur in this part of auditory cortex.
Twenty-five healthy adult volunteers (11 women, age range = 20–29 years, average age = 24 ± 3 years) participated in the experiment. All participants were right-handed, had normal hearing (hearing loss less than 15 dB HL between 100 Hz and 8 kHz), and no history of neurological disorder. The study was conducted in accordance with the Declaration of Helsinki (World Medical Association, 2008). All experimental procedures were approved by the ethics committee of the University of Oldenburg, and written informed consent was obtained from the participants. Two participants did not complete all experimental sessions; six participants had to be excluded because of severe head movements during fMRI measurements (head movement > 3 mm) or because of incidental findings on their MR scans. Therefore, 17 fMRI data sets were statistically analyzed.
In previous experiments on change deafness, complex auditory scenes consisted of different simultaneously presented natural sounds such as animal voices or musical instruments (Gregg & Samuel, 2008, 2009; Eramudugolla et al., 2005, 2008; Pavani & Turatto, 2008). Therefore, auditory streams did not only differ in their physical properties but also carried different semantic information. As shown by Gregg and Samuel (2009), stimulus semantics play an important role in both auditory object encoding and change detection. To avoid interactions between semantic and acoustic cues, we used here a much simpler auditory setting, in which individual streams lacked any higher-order semantic information and only differed in pitch, rhythm, and sound source location.
Figure 1 depicts the complex auditory scenes used in the experiment. Each scene comprised six simultaneously presented auditory streams of 666-msec duration. The streams consisted of series of band-pass-filtered noise stimuli with short (1/48 sec), intermediate (1/24 sec), or long (1/12 sec) durations, interrupted by intervals of silence (also 1/12 sec, 1/24 sec, or 1/48 sec in duration). Sounds were faded in and out using 5-msec Hanning onset and offset ramps. Noise bands were centered at 200, 400, 800, 1600, 3200, and 6400 Hz, with stimulus bandwidths set to 25% of the respective center frequency. Sound source locations were separated in space using head-related impulse responses from dummy-head recordings obtained by Kayser et al. (2009) (anechoic room recordings, azimuth: −90°, −60°, −30°, +30°, +60°, +90°; source elevation: 0°; source distance: 300 cm). We used four different auditory scenes to prevent participants from becoming too familiar with the complex acoustic input. As shown in Figure 1, these were generated by a pairwise exchange of the center frequency in four of the six streams.
Changes within a scene were induced by increasing the center frequency in one stream, whereas all other streams remained identical. This resulted in six possible changes in each scene. The center frequency shifts (Δf) were adjusted individually for each participant and stream to obtain a hit rate of 70.7% (see Procedure section for details). All stimuli were created using MATLAB (The MathWorks, Inc., Natick, MA) and presented to the participants via headphones at a level of 80-dB superior parietal lobe. Within each scene, the streams were matched for loudness using loudness level contours (ISO 226:2003).
To test whether these non-naturalistic complex auditory scenes are capable of inducing change deafness, we conducted additional psychophysical measurements (see supplementary data). As predicted by previous work on change deafness (Eramudugolla et al., 2005), listeners showed a significantly increased change detection performance under selective attention (i.e., participants received a cue indicating which stream should be monitored) as compared with divided attention (i.e., participants had to monitor the whole scene).
As depicted in Figure 2, each trial consisted of two successively presented complex auditory scenes, separated by 666 msec of silence. Listeners were instructed to respond as fast as possible and to indicate whether both complex scenes were identical or not. Key presses had to be made with the index finger (“change”) or middle finger (“no change”) of the right hand. No feedback on the correctness of the answer was provided. The response interval was restricted to 2000 msec after onset of the second scene presentation. During the whole task, participants viewed a fixation cross shown on a screen. The onset of each trial was indicated by a short flashing of the fixation cross presented 500 msec in advance.
The experiment consisted of four sessions and was conducted on four consecutive days. The first session was a training session designed to improve change detection performance and to minimize learning effects during the subsequent measurements. This training session contained ten blocks of 48 trials. The intertrial interval was 2000 msec. Between blocks, participants were allowed to pause the experiment. In half of the trials no change occurred, the other half contained a pitch change in one of the streams. Change and no-change trials were balanced within each block, and trial order was randomized. During the training, ten different magnitudes of change (Δf = 0.1%, 0.5%, 1%, 2%, 4%, 6%, 8%, 10%, 15%, or 20%) were used. Only in this session, participants received feedback on the correctness of their response. A correct response was indicated by the fixation cross flashing in green for 500 msec, an incorrect response by the fixation cross flashing in red.
On the second day, we determined individual subjective detection thresholds using a yes–no task with a 1-up–2-down staircase procedure converging at a hit rate of 70.7% (Levitt, 1971). Thresholds were measured separately for each of the six streams in an interleaved order. All trials presented during this session contained a change in auditory input. The staircase procedure started with an initial value of Δf = 25%. After two correctly detected changes in succession, the change magnitude was reduced by a predefined alternation factor; after each undetected change, the change magnitude was increased by this amount. After each second turning point, the size of the alternation factor was adapted (10%, 1%, 1%, 1%, 1%, 0.1%, 0.1%, 0.1%). For each stream, we recorded 14 turns in direction. Detection thresholds were calculated as the mean of the last eight turning points. The average Δf obtained by this procedure is stated in the Results section. Before the threshold measurements, participants performed a short training session of two blocks. As on the first day, we used different levels of change magnitude (Δf = 0.5%, 2%, 8%, or 20%); half of the trials did not contain a change. The Δf values obtained in the adjustment procedure were applied in the subsequent experimental sessions to equalize the amount of hits and misses across subjects. Because participants were instructed that in the actual experiment 50% of all trials contained a change, this approach also indirectly modulated the percentage of false alarms, resulting in a sufficient number of trials in all four response conditions.
In the third and fourth sessions, fMRI measurements were performed. Data were acquired using a silent sparse-temporal sampling sequence with clustered volume acquisition (Edmister, Talavage, Ledden, & Weisskoff, 1999; Hall et al., 1999) in which data acquisition and stimulus presentation alternated. Thus, scanner background noise did not disturb scene encoding. Between each data acquisition block one trial was presented. Trials started 2000, 3000, or 4000 msec after the end of fMRI data acquisition (see Figure 2). At the beginning of each day, the participants passed a short training session under scanner conditions to get used to the fMRI environment. This training lasted for about 7 min and consisted of 15 change trials, 15 no-change trials, and 14 null events, in which no auditory stimulation was presented. Afterwards, participants performed two runs of the experiment, each taking about 18 min and containing 36 change trials, 36 no-change trials, and 36 null events (no trial was presented). On the third day, we conducted an additional functional localizer measurement to individually assess the listeners' pitch-sensitive part of auditory cortex in the lateral portion of Heschl's gyrus (see next section).
Functional Localizer of Pitch-sensitive Areas in Lateral Heschl's Gyrus
Previous fMRI work on auditory change detection reported significant activations to frequency deviants predominantly close to the lateral portion of Heschl's gyrus, whereas deviants in other stimulus categories showed no effects in this region (Molholm et al., 2005; Opitz et al., 2005). In addition, the lateral portion of Heschl's gyrus is thought to play an important role in the cortical processing of pitch information (Puschmann et al., 2010; Penagos et al., 2004; Krumbholz, Patterson, Seither-Preisler, Lammertmann, & Lutkenhoner, 2003; Warren et al., 2003; Gutschalk, Patterson, Rupp, Uppenkamp, & Scherg, 2002; Patterson et al., 2002). Therefore, we hypothesized that in our experiment change-related activity should occur within this part of auditory cortex. To access this functional ROI, we applied an experimental paradigm that was previously shown to robustly activate pitch-sensitive areas in the lateral portion of Heschl's gyrus (Patterson et al., 2002).
Similar to the change detection paradigm, we used a sparse-temporal sampling sequence. Between data acquisition blocks, participants were presented with sound sequences of iterated rippled noise (IRN), random noise, or silent blocks. In total, we presented 32 trials of each condition. Sequences were presented in random order. Participants were instructed to fixate on a centrally presented cross during the measurements. IRN is generated by delaying a copy of random noise and adding it back repeatedly to the original signal. These stimuli induce a sensation of pitch similar to a pure tone with a frequency corresponding to the inverse of the used delay (Yost, 1996). IRN stimuli were produced as described in the work of Patterson and coworkers (2002). We generated eight different IRN sounds of 200-msec duration with pitches ranging from 50 to 110 Hz (16 iterations, delays ranging from 9.45 to 19.26 msec, bandpass-filtered between 500 Hz and 8 kHz). Using these sounds, we created eight sequences of constant pitch, each containing 24 IRN stimuli, separated by 50 msec of silence. Similarly, we generated four different sequences of random noise. All stimuli were created using MATLAB (The MathWorks, Inc., Natick, MA).
The first two experimental sessions (training, adjustment procedure) took place in a sound-attenuating chamber (Industrial Acoustics Company, Winchester, UK). Auditory stimuli were generated using a 24-bit 96-kHz USB audio processor (UA-25 EX, Roland Corporation, Hamamatsu, Japan), amplified via a stereo amplifier (TA-FA5ES, Sony Corporation, Minato, Tokyo, Japan), and delivered by MR-compatible in-ear headphones (Sensimetrics S14, Sensimetrics Corporation, Malden, MA). Key presses were recorded using a millisecond-accurate keyboard (DirectIN, Empirisoft Corporation, New York, NY). Experimental control software was programmed in MATLAB using Psychtoolbox-3 (Kleiner, Brainard, & Pelli, 2007; Brainard, 1997).
Functional MRI data were acquired on a 1.5-T Siemens MAGNETOM Sonata MRI scanner (Siemens AG, Erlangen, Germany) with an eight-channel head array. Auditory stimuli were generated using a 24-bit 96-kHz USB audio processor (UA-25 EX, Roland Corporation, Hamamatsu, Japan), amplified via a stereo amplifier (A-9510, Onkyo Corporation, Osaka, Japan), and delivered by MR-compatible in-ear headphones (Sensimetrics S14, Sensimetrics Corporation, Malden, MA). Key presses were recorded using an MR-compatible optical response keypad (LUMItouch, Photon Control, Inc., Burnaby, BC, Canada). Participants viewed a fixation cross presented on a screen via a mirror located above the head coil. Experimental control software was programmed in MATLAB using Cogent 2000 (Cogent 2000 team, FIL and ICN, University College London, London, UK).
For the functional measurements, we used a silent sparse temporal sampling sequence with clustered volume acquisition. During each run of the change detection task, 109 T2*-weighted gradient EPI volumes (time of repetition [TR] = 10,000 msec, delay in TR = 7500 msec, time to echo = 50 msec, flip angle α = 90°, field of view = 200 × 200 mm2, voxel size = 3.0 × 3.1 × 3.1 mm3) with BOLD contrast were obtained. Volumes consisted of 29 transverse slices with a gap of 0.6 mm in-between and were recorded in an ascending order. For each participant, we obtained a high-resolution structural volume using a T1-weighted magnetization-prepared rapid acquisition gradient-echo sequence.
For the functional localizer, we obtained 73 T2*-weighted EPI volumes with a TR of 8500 msec and a delay in TR of 6000 msec. All other settings were kept similar to the change detection paradigm.
We analyzed the listeners' responses as a function of physical change (present/absent) and change perception (yes/no). This resulted in four different response categories: hits (i.e., correctly perceived changes in auditory input), misses (i.e., unperceived changes in auditory input), false alarms (i.e., perception of change in the absence of a physical change in auditory input), and correct rejections (no change perception in the absence of a physical change).
To test for learning effects in the course of the fMRI measurements, we individually calculated the sensitivity index d′ and the response bias index c as well as the median RTs (across all conditions) for each fMRI run and tested for significant differences using one-factorial repeated-measures ANOVAs. Additionally, to control for differences in change detection performance between the four basic auditory scene versions used in the experiment, we also investigated d′ as a function of scene (Versions 1–4) using a one-way repeated-measures ANOVA. Effects were deemed to be statistically significant when passing a threshold of p < .05 (uncorrected). The Results section states hit and false alarm rates, the average sensitivity d′, and the average response bias c obtained across all fMRI runs. Values are given as the mean and the SEM.
fMRI Data Preprocessing
Functional MRI data were processed and analyzed using SPM8 (FIL, Welcome Trust Centre for Neuroimaging, University College London, London, UK). To correct for head motion, each functional time series was spatially realigned to the first image of the first session. The structural T1-weighted volume was registered to a mean functional image and segmented to obtain spatial normalization parameters. Using these parameters, functional and structural images were normalized to the Montreal Neurological Institute template brain. Finally, normalized functional volumes were smoothed with a three-dimensional Gaussian kernel of 6 mm FWHM.
Functional Localizer of Pitch-sensitive Regions
The functional localizer data were statistically analyzed using a mixed-effects model. On the single-subject level, we calculated estimation parameters for two regressors, one for pitch-inducing IRN stimuli and one for noise, using the general linear model. In addition, we included movement parameters as obtained in the SPM realignment procedure to account for signal changes related to head movement. The time series of each voxel were high-pass filtered to 1/128 Hz and modeled for temporal autocorrelation across scans with an AR(1) process.
We determined the contrast pitch > noise for each participant and tested it for significant differences at group level using a one-sample t test with a combined threshold of p < .05, corrected for family-wise errors (FWE), and a minimal cluster size of k = 10. Because we were only interested in auditory cortex effects, we restricted our volume of interest to Heschl's gyrus, the superior temporal gyrus, and the superior temporal pole, which were specified using the WFU PickAtlas extension for SPM (Maldjian, Laurienti, Kraft, & Burdette, 2003). The analysis resulted in statistically significant activation clusters in both hemispheres. The larger cluster was localized to left lateral Heschl's gyrus, extending into Heschl's sulcus, anterior planum temporale, and medial Heschl's gyrus (activation peak at [x, y, z] = [−58, −10, 6], cluster size k = 211). In the right hemisphere, activation was mainly present in Heschl's sulcus, extending into right lateral Heschl's gyrus (activation peak at [x, y, z] = [52, −2, 2], cluster size k = 65). This pattern of activity was used as the functional ROI for analysis of the change detection data.
Change Detection Paradigm
A mixed-effects model was used for statistical analysis of the change detection data. For the single-subject model, we specified four regressors modeling the four different auditory scenes to account for overall variance in the auditory input (Scene Versions 1–4) as well as four regressors describing the participants' responses (hits, misses, false alarms, correct rejections). Additionally, we included one regressor for trials in which no response was given by the participant. Regressors were modeled separately for each fMRI run. Please note that not all participants generated false alarms in all runs. In these cases, a dummy onset was inserted at the end of the corresponding regressor. Movement parameters were included to account for signal changes related to head movement. The time series of each voxel were high-pass filtered to 1/128 Hz and modeled for temporal autocorrelation across scans with an AR(1) process.
On the single subject level, we calculated baseline contrasts for hits, misses, false alarms, and correct rejections (responses in the respective condition against silence). On the group level, we analyzed these contrasts using a flexible-factorial three-way ANOVA design as implemented in SPM8 including the factors Subject (17 levels), Physical Change (present/absent), and Change Perception (yes/no). To account for the differences in trial numbers between conditions, we assumed unequal variances for the factors physical change and change perception.
To isolate brain areas showing an increased response to physical change, we tested the t contrast physical change > no change, that is, hits and misses versus false alarms and correct rejections. Increases in brain activity related to conscious change perception were gauged with the t contrast change perception > no perception, which was established by comparing hits and false alarms versus misses and correct rejections. Our main analysis predominantly focused on auditory cortex using the regions obtained from the functional localizer as a ROI (see previous section). In addition, we also conducted a whole-brain analysis to reveal change-related responses beyond auditory cortex. Effects were deemed to be statistically significant for a threshold of p < .05 (FWE corrected on cluster level). To control for a potentially confounding bias related to the chosen ROI in auditory cortex, we additionally report all activations in temporal lobe at a more liberal threshold of p < .001 (uncorrected).
To ensure that significant activations observed in these analyses were not driven by only one condition but truly reflect effects of physical change or change perception, we extracted average contrast estimates for the crucial subcontrasts from the activation clusters. For example, if a cluster reflects physical change, which was isolated by comparing (hits + misses) − (false alarms + correct rejections), this effect should also hold for both the critical subcontrasts hits–false alarms and misses–correct rejections. Note that the contrasted conditions differ only with regard to the presence of a physical change in auditory input, but are equal with regard to subjective change perception. In addition, we also directly compared contrast estimates of hits and misses within these regions to test for differences related to change perception. Accordingly, for clusters obtained in the analysis of change perception, we extracted contrast weights for hits–misses and false alarms–correct rejections. These conditions differ with regard to subjective change perception but are equal with regard to the presence of a physical change in auditory input. Additionally, we here also directly compared contrast estimates of hits and false alarms to test for differences related to correct change perception. Statistical significance of all post hoc comparisons was tested using t tests at p < .05.
Our main analysis revealed similar BOLD responses to hits and misses in right auditory cortex (see Results section). But because only hits trigger responses in hierarchically higher nonsensory regions presumably involved in the conscious processing of change, we assumed differences in the effective connectivity of right auditory cortex between both conditions. To investigate such effects we performed a psychophysiological interaction analysis (PPI) within SPM8. A PPI analysis aims to explain BOLD responses in one brain area in terms of the interaction between influences of another brain region and a cognitive/sensory process (Friston et al., 1997). In other words, a PPI identifies voxels in which the single-trial BOLD response to an event is differentially modulated by the BOLD response in a given seed region (here: right auditory cortex) as a function of experimental condition (here: hits, misses). In contrast, brain regions showing similar connectivity in both experimental conditions are not revealed by a PPI.
A mixed-effect model was used for the PPI analysis. On the single subject level, the design matrix contained four regressors modeling the auditory stimulation in the four basic scenes and three PPI regressors: (i) a “physiological variable” representing the BOLD response in right auditory cortex, (ii) a “psychological variable” representing successful change detection (i.e., hits–misses), and (iii) the interaction of the factors. The time series of each voxel were high-pass filtered to 1/192 Hz and modeled for temporal autocorrelation across scans with an AR(1) process. On the group level, we performed simple t tests on the interaction term. Effects of the whole-brain analysis were reported as statistically significant for a combined threshold of p < .001 (uncorrected) and a minimum cluster size of k = 20.
For each subject, we selected the local activation peak of the contrast physical change > no change within a 12-mm sphere (i.e., twice the smoothing kernel) around the group activation maximum for this contrast in right auditory cortex ([x, y, z] = [58, −16, 4]). Because the strength of this effect varied between participants, we used a liberal threshold of p < .1 (uncorrected) on the single-subject level to obtain effects in all participants. The first eigenvariate of the BOLD response in right auditory cortex that was used for the PPI analysis was then extracted from all activated voxels located within a sphere of radius 6 mm around the individual activation peak. The median of the obtained individual peak coordinates was [x, y, z] = [58 ± 4, −16 ± 7, 6 ± 3]. The median number of voxels entering the eigenvariate calculation was k = 35 ± 23.
For a post hoc analysis of the results of the PPI analysis, we computed the correlation between responses in right auditory cortex and responses in regions obtained in the PPI analysis as a function of condition (hits/misses). On the single subject level, we extracted the first eigenvariates in these regions (p < .1, 6 mm sphere) and calculated the single trial responses as a function of condition (hits/misses). Similarly, we separated right auditory cortex responses to hits and misses and performed regression analyses estimating the modulation of the response amplitudes in the obtained regions as a function of the auditory cortex response, separately for each condition. On the group level, we applied two-tailed t tests to investigate whether the estimated slopes differed significantly from zero and to analyze significant differences between conditions. Results were reported as statistically significant for p < .05.
In the adjustment procedure, which targeted a hit rate of 70.7%, listeners reached an average detection threshold of Δf = 6.1 ± 0.6%. For the individual streams, average values ranged from Δf = 2.6 ± 0.4% to Δf = 13.4 ± 1.5%. Applying the individually adjusted thresholds in the fMRI task, participants reached an average hit rate of 70.8 ± 2.8% and an average false alarm rate of 18.1 ± 3.0%. The mean sensitivity index d′ was 1.79 ± 0.26; the average response bias c was 0.32 ± 0.14. One-factorial ANOVAs on the behavioral data showed no learning-related changes in d′ [F(3, 14) = 1.8, p > .1], c [F(3, 14) = 1.7, p > .1], or RTs [F(3, 14) = 0.7, p > .5] across the four fMRI runs. In addition, there was no statistical difference in d′ between the four different auditory scenes used in the experiment [one-factorial ANOVA, F(3, 14) = 1.3, p > .2].
Correlates of Physical Change in Auditory Input
The contrast physical change > no change [i.e., (hits + misses) − (false alarms + correct rejections)] revealed a statistically significant activation within the right ROI, which encompassed parts of lateral Heschl's gyrus and sulcus showing an increased BOLD response to pitch-containing stimuli as compared with noise (p < .05, FWE corrected on cluster level). The activation peak was at [x, y, z] = [58, −16, 4], the cluster size was k = 5 voxels. As depicted in Figure 3, using a more liberal threshold of p < .001 (uncorrected) resulted in an additional activation cluster close to the right ROI, but in no other parts of the temporal lobe. To ensure that the observed pattern of activity held for both hits and misses as compared with false alarms and correct rejections, respectively, we extracted average contrast estimates for hits–false alarms and misses–correct rejections from the activation cluster (see Figure 3, right). BOLD responses were significantly increased for both hits and misses as compared with the perceptually matched conditions in which no change occurred (both p < .05). No differences in contrast estimates for hits and misses were observed in this region (p > .7). No further activations related to physical changes were found on the whole-brain level. Likewise, the inverse contrast no physical change > change revealed no significant effects.
Correlates of Change Perception
For the contrast change perception > no perception [i.e., (hits + false alarms) − (misses + correct rejections)], we found two significant clusters of activation (p < .05, FWE corrected on cluster level; see Figure 4). One was located in ACC (activation peak at [x, y, z] = [−2, 36, 20], cluster size k = 111), the other one was in the right insula (activation peak at [x, y, z] = [30, 8, −16], k = 119). No significant perception-related activations were found in any part of the temporal lobe. However, at a more liberal threshold of p < .001 (uncorrected), we obtained an activation cluster in right posterior STS (activation peak at [x, y, z] = [44, −38, 0], k = 25).
Again, we extracted contrast estimates to ensure that the observed effects held for both hits and false alarms. As depicted in Figure 4, BOLD responses in these conditions were significantly increased as compared with misses and correct rejections, in which no change was perceived (all ps < .01). In addition, the contrast estimates for false alarms were significantly increased as compared with hits in both regions (p < .01). For the inverse contrast no change perception > perception, we found no significant activations.
In a second step, we performed a PPI analysis to investigate differences in functional connectivity as a function of change detection (hits/misses) between the part of right auditory cortex, which showed increased responses for both hits and misses, and other brain regions. As depicted in Figure 5A, we found a cluster encompassing parts of the left insula and left inferior frontal gyrus, showing a significant positive interaction effect (p < .001, uncorrected; minimum cluster size of k > 20). The activation peak was located at [x, y, z] = [−42, 14, 10]; the cluster contained k = 66 voxels. A post hoc regression analysis estimating the response modulation in this region as a function of the auditory cortex response revealed a significant positive relationship for hits (slope s = 0.14 ± 0.03, p < .001), but not for misses (s = 0.05 ± 0.04, p > .3). A direct comparison between conditions also resulted in a significant difference (p < .05).
In addition, a cluster in the right STS showed a significant negative interaction effect (peak at [x, y, z] = [52, −10, −16], k = 38 voxels). The post hoc regression analysis provided evidence for a significant relationship between responses in the cluster and right auditory cortex for misses (s = 0.15 ± 0.04, p < .05), but not for hits (s = 0.06 ± 0.04, p > .1). Again, a direct comparison showed that both slopes differed significantly (p < .05). Both the activation cluster and the estimated slopes are depicted in Figure 5B. Note that the post hoc regression analysis in right STS was based on only 15 of 17 data sets, because two participants did not show any substantial activity in this region (at p < .1, uncorrected).
We separated the neural correlates of physical change in auditory stimulation and conscious change perception making use of the change deafness effect. Increased BOLD responses related to physical change were found in right auditory cortex, close to the lateral portion of Heschl's gyrus, but not in hierarchically higher brain regions. Conscious perception of change induced increased responses in the ACC and the right insula. In addition, at a lower significance level, there was also some evidence for perception-induced effects in right STS.
Auditory Cortex Responses to Physical Change
Functional MRI studies have identified change-related responses in both primary and secondary auditory cortex (Sabri, Humphries, Binder, & Liebenthal, 2011; Stewart, Overath, Warren, Foxton, & Griffiths, 2008; Altmann, Bledowski, Wibral, & Kaiser, 2007; Deouell, Heller, Malach, D'Esposito, & Knight, 2007; Sabri et al., 2006; Molholm et al., 2005; Opitz et al., 2005; Opitz, Rinne, Mecklinger, von Cramon, & Schröger, 2002; Linden et al., 1999). However, the specific location of change-related responses is shown to depend on the changing acoustical sound feature (Altmann et al., 2007; Molholm et al., 2005). Frequency deviants are mainly reported to induce increased BOLD responses close to the lateral portion of Heschl's gyrus (Molholm et al., 2005; Opitz et al., 2005). In line with these findings, we also observed activity related to frequency changes close to the lateral portion of Heschl's gyrus. Our results, however, extend these prior findings in showing that BOLD responses were similarly increased for both perceived and unperceived changes. Hence, neural activity in lateral Heschl's gyrus seems to represent early stages of sensory processing not influenced by conscious perception of change. Supporting this assumption, previous electrophysiological studies report common early responses for both detected and undetected target sounds (Königs & Gutschalk, 2012; Gutschalk et al., 2008). For example, Königs and Gutschalk (2012) presented magnetoencephalography data on neural activity related to the detection of target sequences, which were informationally masked by random tone bursts. Both detected and undetected targets were shown to evoke similar P1m responses (45–85 msec), whereas later auditory cortex components showed perception-related differences.
Opitz and colleagues (2005) suggested that change-related activations in the anterior rim of Heschl's gyrus are likely to represent memory-based processes, whereas increased BOLD responses in the posterior rim of Heschl's gyrus might be related to a release from adaptation during sensory processing of acoustic deviants (“fresh neurons”). Moreover, the lateral portion of Heschl's gyrus was previously shown to be sensitive to pitch information in general and to changes in pitch (Krumbholz et al., 2003; Patterson et al., 2002). Therefore, we propose that the change-related activation observed in our study might reflect the recruitment of a different, nonhabituated set of neurons processing the changes in pitch. This view is supported by recent electrophysiological studies reporting early change-related responses in auditory cortex at the level of the Nb component, peaking about 35 msec after stimulus onset (Grimm, Escera, Slabu, & Costa-Faidella, 2011). Interestingly, this effect was found for frequency changes but not for other acoustic deviants (Leung, Cornella, Grimm, & Escera, 2012). Therefore, the response is unlikely to reflect explicit change detection mechanisms but rather early stages of pitch processing (Alho, Grimm, Mateo-Leon, Costa-Faidella, & Escera, 2012; Marmel, Perrin, & Tillmann, 2011). The spread of activity along the posterior rim of Heschl's gyrus might reflect the tonotopic gradient observed in this region (Langers, de Kleine, & van Dijk, 2012). Because we induced changes across a large frequency range covering five octaves, release from adaptation should occur along different parts of the posterior rim of Heschl's gyrus.
Auditory Cortex Responses to Change Perception
Recent work provides evidence that auditory cortex responses to physically identical sounds are modulated by the listeners' perception (Bernasconi et al., 2011; Brancucci, Franciotti, D'Anselmo, Della Penna, & Tommasi, 2011; Hill, Bishop, Yadav, & Miller, 2011; Kilian-Hütten et al., 2011). For example, neuroimaging data on perceptual streaming by Hill and colleagues (2011) indicate a significant modulation of BOLD responses in auditory cortex related to the perception (grouped/split) of bistable auditory streams.
Investigating target detection under informational masking, Wiegand and Gutschalk (2012) reported significantly increased BOLD responses for detected as compared with undetected target sequences in medial Heschl's gyrus, indicating perception-related effects already at the level of primary auditory cortex. However, other studies on target detection did not reveal any differences between detected and missed targets in Heschl's gyrus (Sabri et al., 2011; Sadaghiani, Hesselmann, & Kleinschmidt, 2009). Instead, Sadaghiani and co-workers (2009) reported differences between hits and misses only outside the sensory core regions in the right STS, whereas Sabri et al. (2011) did not report any perception-related differences in any part of the temporal lobe. In our study, the data indicate some evidence for perception-related effects in the right STS, but not along Heschl's gyrus. Activity in this more posterior portion of STS was previously associated with voluntary attention shifts to the changing stream (Huang, Belliveau, Tengshe, & Ahveninen, 2012). In line with this, we also suggest that the effect observed in our experiment might be related to shifts of attention in response to the perception of change in one of the presented auditory streams.
In addition to this effect, our PPI analysis indicated a positive correlation between sensory-driven BOLD responses in right lateral Heschl's gyrus and neural activity in right STS. This effect was only observed for misses but not for hits. Change-related responses in this more anterior portion of STS were previously associated with involuntary attention shifts toward salient or novel auditory stimuli (Huang et al., 2012; Sabri et al., 2006). This may indicate that misses still capture the listeners' attention to some extent. However, it remains open why only misses, but not hits, should trigger such an involuntary attention shift. A similar effect was recently described in an electroencephalography study on change deafness by Gregg and Snyder (2012). They showed an increased auditory P2 response (between 235 and 315 msec after deviance onset) for misses as compared with correct rejections. In contrast, no difference was observed at this latency range for hits. So far, the experimental data cannot explain these differences between hits and misses. Further studies will be needed to investigate change-related responses under change deafness in more detail.
Note that in active change detection tasks that have long gaps between stimulus presentations, listeners may make use of memorizing strategies to perform the task. Recent neuroimaging work investigating maintenance effects in auditory cortex in a change detection task showed a stimulus-specific suppression during the memorizing phase in large parts of primary and secondary auditory cortex (Linke, Vicente-Grabovetsky, & Cusack, 2011). Although maintenance periods in that study were significantly longer (2 or 10 sec) than in our experiment (666 msec), it cannot be excluded that memory-specific suppression affected auditory cortex responses (both to physical and perceptual change) observed in our paradigm.
Effects Related to Change Perception beyond Auditory Cortex
Perceiving changes in auditory input was found to be mainly associated with increased BOLD responses in the ACC and in the right insula. Previous neuroimaging studies on auditory change and target detection reported significantly increased BOLD responses in these regions related to detected changes as compared with no-change stimuli (Deouell et al., 2007; Linden et al., 1999) as well as for correctly detected targets as compared with missed targets or baseline activity (Sadaghiani et al., 2009; Kiehl, Laurens, Duty, Forster, & Liddle, 2001). In addition, similar effects have also been observed in studies on visual change or target detection (Huettel, Misiurek, Jurkowski, & McCarthy, 2004; Pessoa & Ungerleider, 2004; Huettel, Guzeldere, & McCarthy, 2001). Menon and Uddin (2010) proposed that the anterior insula and ACC form a saliency network that detects behaviorally relevant sensory stimuli and initiates further attentive processing. According to this model, the insula receives bottom–up deviance signals from the sensory cortices and evaluates the behavioral relevance of this information. The ACC triggers top–down inputs into sensory and nonsensory regions involved in executive control, which initiate attention-switching and a behavioral response (Crottaz-Herbette & Menon, 2006). In agreement with this model, our PPI analysis revealed a significant correlation between BOLD responses in right auditory cortex and the left insula for successfully detected changes, but not for undetected changes, strengthening the view that the insula plays a key role for conscious change detection. However, note that the different lateralization of insula activations in the main analysis (right insula) and the PPI (left insula) is unlikely to reflect specific hemispheric differences. In both analyses, applying a more liberal statistical threshold resulted in bilateral insula activation.
The finding that effective connectivity between auditory cortex and insula was only observed for hits suggests that the signal from right lateral Heschl's gyrus is not passed directly to the insula but is regulated by a low-level change detection mechanism in auditory cortex. In electrophysiological studies, change detection in auditory cortex is usually reflected by the MMN component, elicited about 150–200 msec after the onset of a deviating sound (Bendixen, Sanmiguel, & Schröger, 2012; Näätänen, Kujala, & Winkler, 2011; Näätänen, Astikainen, Ruusuvirta, & Huotilainen, 2010; Näätänen, Paavilainen, Rinne, & Alho, 2007; Näätänen, Jacobsen, & Winkler, 2005; Näätänen, Tervaniemi, Sussman, Paavilainen, & Winkler, 2001). Some previous studies indicate that the MMN can also be elicited by unperceived changes (Van Zuijen, Simoens, Paavilainen, Näätanen, & Tervaniemi, 2006; Allen, Kraus, & Bradlow, 2000). However, recent experiments that investigated this component in active target detection tasks reported no MMN-like responses to unperceived changes (Gregg & Snyder, 2012; Wiegand & Gutschalk, 2012; Schröger, Bendixen, Trujillo-Barreto, & Roeber, 2007). Therefore, it seems plausible that the low-level change detection mechanism reflected by the MMN response regulates sensory change-induced inputs into the insula (Menon & Uddin, 2010). However, our data show no significant differences between hits and misses in auditory cortex and therefore provide no evidence for the involvement of such an early change detection mechanism. We suggest that this might be related to our experimental design, which is different from the auditory oddball designs usually applied to investigate MMN and low-level auditory change detection. In these experiments, a sensory memory representation of the auditory input is induced by repeated presentation of standard stimuli, whereas in our paradigm the standard scene (consisting of several stimuli) is only presented once and changed on a trial-to-trial basis. Therefore, a potentially evoked MMN-like response might be comparatively small in our experiment.
Perception-induced effects in the insula and the ACC were found for both correctly and erroneously perceived changes. However, the effects were stronger for false alarms as compared with hits in both areas. We suggest that this increased activity might be related to specific aspects of performance monitoring. Previous studies on these processes reported increased responses in both the cingulate cortex and the insula in the context of error processing (Ullsperger, Harsay, Wessel, & Ridderinkhof, 2010; Taylor, Stern, & Gehring, 2007; Debener et al., 2005), indicating that the observed effects might be related to some kind of implicit knowledge about the response error. In this case, it might be assumed that misses show a similar modulatory effect in both regions. However, post hoc t tests showed no evidence for this (at p < .05, uncorrected). On the other hand, activity in the cingulate cortex was also shown to be modulated by response uncertainty and preresponse conflicts (Ridderinkhof, Ullsperger, Crone, & Nieuwenhuis, 2004). Hence, listeners may show a higher degree of uncertainty in the case of false alarms as compared with hits, which might be associated with an increased BOLD response.
This experiment shows that the change deafness effect provides a useful tool to separate neural responses related to physical change in auditory stimulation and conscious change perception. Increased BOLD responses related to changes in stimulus frequency were confined to the lateral portion of Heschl's gyrus and occurred similarly for both hits and misses. This provides evidence that, under change deafness, undetected changes are still encoded correctly but fail to trigger conscious change detection. This finding not only complements earlier electrophysiological experiments on target detection showing similar early event-related responses to both detected and undetected targets (Gutschalk et al., 2008) but is also consistent with previous psychophysical work reporting successful stimulus encoding under change deafness (Gregg & Samuel, 2008). No effects related to the conscious perception of change were found in auditory cortex core regions. Instead, both hits and false alarms were associated with increased BOLD responses in the insula and the ACC. Our functional connectivity analysis corroborates this finding in showing increased coupling between the lateral right auditory cortex and the insula in response to hits. This strengthens the view that the insula and the ACC form a saliency network, which receives change-related responses from auditory cortex and initiates further conscious processing of change.
This work was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG, SFB/TRR31). Sebastian Puschmann was supported by a grant from the graduate school “Neurosenses.” The authors thank Dr. Jeremy Thorne for proofreading the final version of the manuscript. The helpful comments by three anonymous reviewers are acknowledged.
Reprint requests should be sent to Sebastian Puschmann, Carl-von-Ossietzky Universität Oldenburg, 26111 Oldenburg, Germany, or via e-mail: firstname.lastname@example.org.