Functional Overlap between Regions Involved in Speech Perception and in Monitoring One's Own Voice during Speech Production

In the study of human communication, the relationship between speech perception and speech production has long been controversial (e.g., Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967). That the two sides of spoken language are linked somehow is not in dispute—prelingual deafness without immediate cochlear implantation severely hinders the development of normal speech in children (Schauwers et al., 2004; Geers, Nicholas, & Sedey, 2003), and hearing impairment in adults affects aspects of speech production such as the control of fundamental frequency and intensity (Cowie, Douglas-Cowie, & Kerr, 1982). Talking, like all motor control, must require sensory feedback to maintain the accuracy and stability of movement (Levelt, 1983). However, the nature of the link between perception and action in speech, its cognitive structure and its neural organization, are not well understood. Whether the same neural system subserves auditory perception both for speech comprehension and for the purpose of articulatory control is not known.

Clinical data indicate that if auditory feedback is eliminated, speech fluency is affected (Schauwers et al., 2004; Geers et al., 2003; Cowie et al., 1982), and if feedback is delayed, fluency is disrupted (Fukawa, Yoshioka, Ozawa, & Yoshida, 1988; Howell & Powell, 1987; Siegel, Schork, Pick, & Garber, 1982; Mackay, 1968; Black, 1951). Given the rapidity of speech, it would be difficult for acoustic feedback to contribute to control as part of a servomechanism (Lashley, 1951) because the articulation of speech sounds would be finished by the time acoustic feedback could be processed. Alternatively, feedforward mechanisms, capable of predictive motor planning, have been proposed (Kawato, 1989). Such “internal models” of the motor system (Jordan & Rumelhart, 1992) are hypothesized to learn through sensory (e.g., auditory) feedback and thus involve representations of actions and their consequences.

One source of evidence for internal models in speech comes from auditory perturbation studies, in which talkers are observed to alter their speech production in response to altered acoustic feedback. For example, if the fundamental frequency of auditory feedback is shifted or if frequencies of vowel formants are altered, talkers accommodate rapidly as if to normalize their production so that it is acoustically closer to a desired output (Chen, Liu, Xu, & Larson, 2007; Purcell & Munhall, 2006a, 2006b; Jones & Munhall, 2000; Houde & Jordan, 1998; Elman, 1981). Such studies yield two important conclusions. First, that subjects rapidly compensate for the perturbation in subsequent trials indicates the presence of an error detection and correction mechanism (e.g., Jones & Munhall, 2000; Houde & Jordan, 1998). Second, the compensation persists for a short period after acoustic feedback is returned to normal (e.g., Purcell & Munhall, 2006a, 2006b), suggesting that the error detection mechanism is producing learning that tunes the speech motor controller.

The neural network supporting this acoustic error detection and correction has been investigated in both nonhuman primates and humans. A recent study in marmosets reveals that vocalization-induced suppression within neurons in the auditory cortex is markedly reduced (firing rate increases) if feedback is altered by frequency shifting (Eliades & Wang, 2008). Recent functional neuroimaging studies in humans reveal increases in activity in the superior temporal areas when vocalization with altered feedback is compared with a condition with normal feedback (Tourville, Reilly, & Guenther, 2008; Christoffels, Formisano, & Schiller, 2007; Fu et al., 2006; Hashimoto & Sakai, 2003). This signal increase has been interpreted as evidence for an error signal that could be used to tune ongoing speech production (Guenther, 2006).

The engagement of the superior temporal region in processing speech during listening has been extensively documented. Superior temporal activation is observed for processing of speech stimuli (see Table 1) when compared with noise (Obleser et al., 2006; Rimol, Specht, Weis, Savoy, & Hugdahl, 2005; Jancke, Wustenberg, Scheich, & Heinze, 2002; Binder et al., 1994; Zatorre, Evans, Meyer, & Gjedde, 1992), tones (Burton & Small, 2006; Ashtari et al., 2004; Poeppel et al., 2004; Zaehle, Wustenberg, Meyer, & Jancke, 2004; Joanisse & Gati, 2003; Specht & Reul, 2003; Benson et al., 2001; Vouloumanos, Kiehl, Werker, & Liddle, 2001; Binder et al., 2000; Burton, Small, & Blumstein, 2000; Binder, Frost, Hammeke, Rao, & Cox, 1996; O'Leary et al., 1996), and other nonspeech sounds (Benson, Richardson, Whalen, & Lai, 2006; Uppenkamp, Johnsrude, Norris, Marslen-Wilson, & Patterson, 2006; Meyer, Zysset, Yves von Cramon, & Alter, 2005; Gandour et al., 2003; Thierry, Giraud, & Price, 2003; Belin, Zatorre, & Ahad, 2002; Giraud & Price, 2001). Recent functional imaging studies in which speech and nonspeech conditions are very closely matched acoustically also elicit signal change in this region, suggesting that it is the processing of speech qua speech and not the acoustic structure of speech that is responsible for the signal change (Desai, Liebenthal, Waldron, & Binder, 2008; Mottonen et al., 2006; Scott, Rosen, Lang, & Wise, 2006; Uppenkamp et al., 2006; Dehaene-Lambertz et al., 2005; Liebenthal, Binder, Spitzer, Possing, & Medler, 2005; Narain et al., 2003; Scott, Blank, Rosen, & Wise, 2000).

Despite the evidence for the involvement of the superior temporal region in the processing of both error signal and speech, whether and how these two functions are linked has never been investigated in the same context. In the experiment presented here, we examined whether the same regions within subjects are processing the speaker's own auditory feedback as well as subserving the perception of speech. We argued that simply observing a change in auditory cortical activity during altered, compared with normal, feedback is not sufficient evidence of an error detection mechanism. The characteristics of a speech error signal include that it be generated uniquely during speech production and not during listening. Accordingly, it is best identified as an interaction, where the same acoustic difference between altered and normal feedback in the context of talking provokes a different pattern of activity than when the identical stimuli are presented in the context of listening. In addition, we will examine whether this error detection mechanism recruits regions that are also active—albeit in a different way—when listeners passively hear speech.

Accordingly, we will compare speech production to passive listening, with identical acoustic stimuli present in both conditions. On production trials, participants either hear their own voice as normal (unaltered feedback) or hear voice-gated, signal-correlated noise (manipulated feedback). On listening trials, participants hear either recordings of their own voice or noise yoked to previous production trials. We identified brain regions that respond more to noise feedback than unaltered feedback during speech production (consistent with an error signal), but that are not sensitive to speech during passive listening, or that respond more to unaltered feedback than to noise feedback during passive listening. By assessing the interaction between speaking condition and feedback, we will examine whether the same brain regions contribute to speech perception for the purpose of ongoing articulatory control and for the purpose of comprehension.

Written informed consent was obtained from 21 individuals (mean age = 23 years, range = 18–45 years, 16 women). All were right-handed, without any history of neurological or hearing disorder, and spoke English as their first language. Each participant received $15 to compensate them for their time. Procedures were approved by the Queen's Health Sciences Research Ethics Board.

We adopted a 2 × 2 factorial design (four experimental conditions) and a low-level silence/rest control condition. The four experimental conditions were as follows: production-clear—producing whispered speech (“Ted”) and hearing this through headphones; production-masked—producing whispered speech (“Ted”) and hearing voice-gated, signal-correlated masking noise (Schroeder, 1968), which is created by applying the amplitude envelope of the utterance to white noise; listen-clear—listening to the stimuli of production-clear trials (without production); and listen-masked—listening to the stimuli of production-masked trials (without production). Whispered speech was used to minimize bone conducted auditory feedback (Barany, 1938) and to make sure that noise would effectively mask speech (Houde & Jordan, 2002).

fMRI data were collected on a 3-T Siemens Trio MRI system, using a rapid sparse-imaging procedure (Orfanidou, Marslen-Wilson, & Davis, 2006). To hear responses and to minimize acoustic interference, we presented stimuli and recorded responses (i.e., a single trial occurred) in a 1400-msec silent period between successive 1600-msec scans (EPI; 26 slices; voxel size = 3.3 × 3.3 × 4.0 mm). A high-resolution T1-weighted MPRAGE structural scan was also acquired on each subject.

In the two production conditions, participants spoke into the optical microphone (Phone-Or; Magmedix, Fitchburg, MA), and their utterances were digitized at 10 Hz with 16-bit precision using a National Instruments PXI-6052E input/output board. Real-time analysis was achieved using a National Instruments PXI-8176 embedded controller. Processed signals were converted back to analog by the input/output board at 10 kHz with 16-bit precision and played over high-fidelity magnet-compatible headphones (NordicNeuroLab, Bergen, Norway) in real time. The processing delays were short enough (iteration delay less than 10 msec) that they would not be noticeable to listeners. The processed signals were also recorded and stored on a computer to be used for the yoked trials of the listen-only conditions. Sound was played at a comfortable listening level (approximately 85 dB; see Figure 1A).

In the production-clear condition, the whispered utterance was simply digitized and recorded and played out unaltered. The masking noise in the production-masked condition was produced by applying the amplitude envelope of the utterance to uniform Gaussian white noise so that the resulting noise would have the envelope of the original speech signal or signal-correlated noise (Schroeder, 1968). The masking noise was also temporally gated with the onset and offset of speaking. All our subjects reported in the exiting interviews that they could only hear masking noise and not their own speech through the headphones when they were speaking in the production-masked condition.

Participants were scanned in three functional runs, each lasting 9 min and comprising 180 trials; 36 of each of the five conditions. Conditions were presented in pseudorandom order, with the limitation that transitional probabilities were approximately equal and all five conditions were presented once in a block of five trials (conditions could repeat at the transition from one block of five trials to the next). The stimuli for listening trials were taken from the production trials in the preceding block, except in the first run when the stimuli for listening trials were from the production trials in the same block. Trials were 3000 msec long, comprising a 1400-msec period without scanning, followed by a 1600-msec whole-brain EPI acquisition (see Figure 1B). Each trial began with a fixation cross appearing in the middle of a black screen (viewed through mirrors placed on the head coil and in the scanner bore) 100 msec before the offset of the previous scan; this signaled the beginning of the trial, and the color of the cross indicated whether the volunteer should whisper “Ted” (if green) or remain silent (if red). Of the five conditions types, the two production conditions were cued with a green cross, and the two listening conditions and the rest condition were cued with a red cross (see Figure 1B). Pilot testing in five subjects revealed that it took volunteers at least 200 msec to respond to the green or red cue, so we could present it 100 msec before the end of the scan and still ensure that subjects (on production trials) would not commence speaking during the scan, thereby using our 1400-msec silent period effectively.

Subjects practiced the whispering-on-cue task before scanning commenced, and we monitored their behavior during scanning: Subject's vocal production and auditory feedback signal were segregated into two different channels and therefore monitored and recorded simultaneously. Their performance in each trial was monitored in real time in the control room for possible errors. In addition, we also inspected the recordings of both production and auditory feedback in each trial afterward to ensure that every incorrect trial was identified and properly counted. The auditory feedback was not picked up by the microphone, and thus there was no danger of the recorded utterances being masked by it. Sessions in which the error rate exceeded 5% were excluded from analysis, as explained in the Results section.

SPM2 (www.fil.ion.ucl.ac.uk/spm/spm2.html) was used for data analysis and visualization. Data were first realigned, within subjects, to the first true functional scan of the session (after discarding two dummy scans), and individual's structural image was coregistered to the mean fMRI image. The coregistered structural was spatially normalized to the ICBM 152 T1 template, and the realigned functional data were normalized using the same deformation parameters. The fMRI data were then smoothed using a Gaussian kernel of 10 mm (FWHM).

Data from each subject were entered into a fixed-effects general linear model using an event-related analysis procedure (Josephs & Henson, 1999). Four event types were modeled for each run. We included six parameters from the motion correction (realignment) stage of preprocessing as regressors in our model to ensure that variability due to head motion was properly accounted for. We chose the hemodynamic response function as the basis function. High-pass filter (cutoff = 128 sec) and AR(1) correction for serial autocorrelation were applied. Contrast images assessing main effects, simple effects, and interactions were created for each subject, and these were entered into random-effects analyses (one-sample t tests) comparing the mean parameter-estimate difference over subjects to zero. Clusters were deemed significant if they exceeded a statistical threshold of p < .05 after correction for multiple comparisons at the cluster level.

Errors occurred when volunteers (a) spoke after a red cross, (b) remained silent after a green cross, or (c) spoke so quietly that the gated noise was not triggered on production-masked trials. We excluded any trial in which an error occurred. In addition, we excluded from analysis any run in which errors exceeded 5% of the trials; this happened in three runs—one run from each of three individuals. Performance exceeded 95% correct in all remaining runs. For the three individuals with missing data, contrast images were computed at the single-subject level from the remaining two runs, and these were included in the random-effects analyses across subjects.

We analyzed main effects of our two factors (production vs. listening and speech feedback vs. noise feedback) and interactions between these factors at the group level. When interactions were significant, we also analyzed simple effects. We started by reporting interactions because they are the main focus of the study and because their presence influences the interpretation of the main effects.

We reasoned that regions involved in processing auditory feedback during talking ought to exhibit a greater increase in activity for noise compared with normal speech, when these are heard as the auditory concomitants of one's own utterances compared with when one is simply listening to them, without talking. We observed such an interaction in the posterior superior temporal gyrus (STG) bilaterally (see Table 2 and Figure 2). To better understand how differences among conditions produced this significant interaction, we explored the simple effects that constitute this interaction. We observed that, within the bilateral STG regions where this interaction yielded significant activity, activation for production-masked trials was significantly greater than for production-clear trials. In the left hemisphere, there was a significant cluster (see Table 3) in the posterior STG, extending into the middle temporal gyrus (MTG) and posteriorly into the supramarginal gyrus. In the right hemisphere, there was one cluster in the STG. The contrast of listen-clear − listen-masked yielded activation that was strongly lateralized to the left hemisphere, with a peak cluster in the left MTG extending into the middle STS and a cluster in the right anterior STG (see Table 4). The left MTG activation peak for this contrast is also in the neighborhood of areas shown in previous studies that contrasted speech stimuli with nonspeech sounds (see Table 1 and Figure 3). The overlap between regions exhibiting a significant Feedback Type × Production/Perception interaction and regions sensitive to speech (activated more by speech than noise during passive listening) is shown in Figure 4. As can be seen, a region of the STG/MTG is sensitive to speech and exhibits increased activation during production when masking noise is heard.

The opposite interaction contrast revealed areas where normal feedback yielded more activation than masking noise during talking compared with passive listening. These regions included the left inferior frontal gyrus, the left superior parietal lobule, the left anterior cingulum, the left middle occipital gyrus, the left caudate, and the right fusiform (see Table 5). Again, to understand this interaction, we explored the simple effects. The contrast of production-clear − production-masked yielded no significant activation. However, the contrast of listen-masked − listen-clear revealed significant clusters in regions highlighted by the interaction (see Table 5). Thus, the interaction appears to be due to a greater difference in these areas when passively listening to noise compared with speech, relative to hearing these sounds while talking.

Production–listening activated an extremely large region involving both hemispheres centered on the left inferior frontal gyrus. Much greater activity during production than listening is consistent with a number of previous studies of speech motor control (Dhanjal, Handunnetthi, Patel, & Wise, 2008; Riecker et al., 2005; Blank, Scott, Murphy, Warburton, & Wise, 2002; Wise, Greene, Buchel, & Scott, 1999; Wildgruber, Ackermann, Klose, Kardatzki, & Grodd, 1996). For example, Wilson, Saygin, Sereno, and Iacoboni (2004) threshold activation maps at p < 10⁻⁴ for listening conditions and at p < 10⁻¹² for speech production conditions to achieve comparable levels of activity. This suggests that speech production must have yielded much more activation relative to listening in their study. When we increased the threshold to p < 10⁻¹¹, we observed clusters in the left inferior frontal gyrus, left postcentral gyrus, and right thalamus (see Figure 5 and Table 6), consistent with previous studies investigating vocal production (Moser et al., 2009; Riecker, Brendel, Ziegler, Erb, & Ackermann, 2008; Kleber, Birbaumer, Veit, Trevorrow, & Lotze, 2007). The reverse contrast, in which activity during production conditions was subtracted from that during listening conditions, did not reveal any significant activation.

We did not observe significant activation at the whole-brain level when comparing speech with noise. This is probably due to the strong Feedback Type × Production/Perception interaction in speech-sensitive regions described earlier; the effect of this crossover interaction is to markedly attenuate the main effect, rendering it not significant. As noted above, during passive listening, signal increases in response to speech as opposed to noise were statistically significant in a left MTG region that many other studies have identified as speech sensitive.

Brain regions more responsive to noise than to speech include the left inferior frontal gyrus, the left postcentral gyrus, the left putamen, the left cerebellum, and the right supramarginal gyrus (see Table 7). Significant activation in auditory regions for this contrast was not observed.

In the bilateral STG, greater activity is elicited during speech production when the auditory concomitants of one's own speech are masked with noise compared with when they are heard clearly. These regions, together with more inferior STS/MTG regions, also exhibit greater activity when listening to speech compared with noise, consistent with them being speech sensitive. That the same STG/MTG region is recruited both for the perception of speech and for processing an error signal during production when the predictive signal and auditory concomitant do not match implies that speech-sensitive regions are implicated in an on-line articulatory control system.

The signal modulation that we observed here was an increase for masked speech compared with clear speech during production but not listening. Although the relationship between hemodynamic response and neural activity is uncertain, this pattern is consistent with neurophysiological work, which demonstrates a release from neural suppression when feedback is altered during production. For example, Eliades and Wang (2008) have observed that, in marmosets, the majority of auditory cortex neurons exhibited suppression during vocalization. However, this vocalization-induced suppression of neural activity was significantly reduced when auditory feedback was altered during vocal production but not during passive listening (Eliades & Wang, 2005, 2008). The attenuation of suppression implies changes in balance of excitatory and inhibitory processes, which would affect regional metabolic energy demands and alter vascular response (Logothetis, 2008). This could in principle manifest as changes in the blood oxygen level that would be detectable as fMRI BOLD signal. Our observation of activity in the bilateral posterior STG in response to altered feedback in production is consistent with such a pattern of neural activity. Models of speech motor control (e.g., Guenther, 2006) suggest that vocal motor centers generate specific predictions about the expected sensory consequences of articulatory gestures, which are compared with the actual sensory outcome. Elevated activity in the STG in the presence of mismatched auditory feedback may reflect the involvement of this region in an error detection mechanism when vocalization occurs (Bays, Flanagan, & Wolpert, 2006; Sommer & Wurtz, 2006; Matsuzawa, Matsuo, Sugio, Kato, & Nakai, 2005; Blakemore, Wolpert, & Frith, 1998).

The observed bilateral STG regions (and in particular the left posterior STG) in the interaction contrast are in accordance with a number of previous studies of speech processing (see Table 1), including studies in which speech and nonspeech stimuli are matched acoustically [e.g., Dehaene-Lambertz et al., 2005: (−60, −24, +4); Mottonen et al., 2006: (−61, −39, +2); Narain et al., 2003: (−52, −54, +14); and Scott et al., 2006: (−60, −44, +10)]. This lends support to the idea that this area is sensitive, if not specific, for speech sounds.

In the macaque monkey, the extreme capsule (EmC) interconnects the rostral part of the lateral and medial frontal cortex with the midpart of the STG and the cortex of the STS. These frontal regions are connected with inferior parietal cortex through the middle longitudinal fasciculus (MdLF; Makris & Pandya, 2009; Petrides & Pandya, 1988, 2006). This EmC–MdLF pathway, spanning frontal–parietal–temporal cortex, has been suggested to play a crucial role in language functions (Makris & Pandya, 2009; Schmahmann et al., 2007). Another fiber system, the superior longitudinal fasciculus–arcuate fasciculus, courses between the caudal-dorsal pFC and the caudal part of the STG. This pathway has been implicated in sensory-motor processing (Makris & Pandya, 2009). A recent study using diffusion tensor imaging-based tractography in humans has found that sublexical repetition of speech is mediated by the superior longitudinal fasciculus–arcuate fasciculus pathway, whereas auditory comprehension is subserved by the EmC–MdLF system (Saur et al., 2008). Therefore, the anatomical location of the posterior STG demonstrating signal increases for masked speech over clear speech during production, and the opposite during passive listening (i.e., the interaction), is such that it may be involved in both of these processing pathways, thereby serving both functions of speech processing and sensory-motor integration (Okada & Hickok, 2006; Buchsbaum et al., 2005; Warren, Wise, & Warren, 2005; Hickok, Buchsbaum, Humphries, & Muftuler, 2003; Okada, Smith, Humphries, & Hickok, 2003; Scott & Johnsrude, 2003).

A number of previous neuroimaging studies have reported bilateral STG activation in response to manipulated auditory feedback compared with normal feedback during reading aloud (Christoffels et al., 2007; Toyomura et al., 2007; Fu et al., 2006; Hashimoto & Sakai, 2003; McGuire, Silbersweig, & Frith, 1996). One feature shared by our study and these studies is that we all manipulated auditory feedback in a way that caused it to be different from what was expected. What distinguishes our study is that we have crossed the feedback-type manipulation common to these studies (normal vs. altered feedback) with a task manipulation (production vs. listening). It is precisely the specificity of this interaction that indicates that activity in brain regions involved in speech perception is modulated by speech production.

We used whispered speech to control the bone-conducted acoustic signals in the masking noise condition more effectively (Houde & Jordan, 2002; Paus, Perry, Zatorre, Worsley, & Evans, 1996). Previous studies indicate that the auditory regions activated by whispered speech are very similar to those activated by vocalized speech, although the level of activation could be somewhat different (Haslinger et al., 2005; Schulz, Varga, Jeffires, Ludlow, & Braun, 2005). This suggests that the use of whispered speech did not qualitatively affect our results.

Vocal production invariably entails both auditory and somatosensory goals (Nasir & Ostry, 2008). We did not manipulate somatosensory signals due to the complexity of such experimental setups (e.g., Tremblay, Shiller, & Ostry, 2003). In addition, because speech production in our study only involves repetitively vocalizing a single consonant–vowel–consonant word (“Ted”), what we have observed is undoubtedly an underestimate of the functional network subserving the speech production and speech perception. Future work could investigate how speech production and perception are coupled using somatosensory perturbations and more naturalistic speech stimuli.

We have observed enhanced activity in the STG region bilaterally during speech production when the auditory feedback and the predicted auditory consequences of speaking do not match. The same region is sensitive to speech during listening. This suggests a self-monitoring/feedback system at work, presumably involved in controlling on-line articulatory planning. Furthermore, the network supporting speech perception appears to overlap with this self-monitoring system, in the STG at least, highlighting the intimate link between perception and production.

The authors thank the referees for their valuable comments. This work was supported by an R-01 operating grant from the U.S. National Institutes of Health, NIDCD grant DC08092 (K. M.), and by grants from the Canadian Institutes of Health Research (I. J.), the Natural Sciences and Engineering Research Council of Canada (I. J. and K. M.), the Ontario Ministry of Research and Innovation, and the Queen's University (I. J. and Z. Z.). I. J. was supported by the Canada Research Chairs Program.

Reprint requests should be sent to Zane Z. Zheng, Centre for Neuroscience Studies, Queen's University, 62 Arch Street, Kingston, Ontario, Canada K7L 3N6, or via e-mail: 5zz2@queensu.ca.