Abstract

Prediction is held to be a fundamental process underpinning perception, action, and cognition. To examine the time course of prediction error signaling, we recorded intracranial EEG activity from nine presurgical epileptic patients while they listened to melodies whose information theoretical predictability had been characterized using a computational model. We examined oscillatory activity in the superior temporal gyrus (STG), the middle temporal gyrus (MTG), and the pars orbitalis of the inferior frontal gyrus, lateral cortical areas previously implicated in auditory predictive processing. We also examined activity in anterior cingulate gyrus (ACG), insula, and amygdala to determine whether signatures of prediction error signaling may also be observable in these subcortical areas. Our results demonstrate that the information content (a measure of unexpectedness) of musical notes modulates the amplitude of low-frequency oscillatory activity (theta to beta power) in bilateral STG and right MTG from within 100 and 200 msec of note onset, respectively. Our results also show this cortical activity to be accompanied by low-frequency oscillatory modulation in ACG and insula—areas previously associated with mediating physiological arousal. Finally, we showed that modulation of low-frequency activity is followed by that of high-frequency (gamma) power from approximately 200 msec in the STG, between 300 and 400 msec in the left insula, and between 400 and 500 msec in the ACG. We discuss these results with respect to models of neural processing that emphasize gamma activity as an index of prediction error signaling and highlight the usefulness of musical stimuli in revealing the wide-reaching neural consequences of predictive processing.

INTRODUCTION

In the field of music cognition, it is widely held that listeners' expectations about how music will unfold provide a rich source of its aesthetic value and meaning (Huron, 2006; Meyer, 1956). According to this theory, predictive uncertainty and violations of expectation introduce tension, whereas events that confirm expectations convey resolution of tension. This theory of music processing is consistent with proposals that prediction is a fundamental process underpinning many aspects of perception and cognition (Clark, 2013; Friston, 2010), including decision-making, motor action, visual perception, and language comprehension (Gentsch, Weber, Synofzik, Vosgerau, & Schütz-Bosbach, 2016; Lewis, Schoffelen, Schriefers, & Bastiaansen, 2016; Summerfield & De Lange, 2014; Rauss, Schwartz, & Pourtois, 2011). For musical stimuli, the likelihood of different continuations and, therefore, the strength of expectation violation can be precisely specified using computational models. This is of great benefit in studying the neural mechanisms underlying expectations in music (Omigie, 2015). However, it is also of great potential benefit in improving our understanding of neural mechanisms of predictive processing more generally.

Electrophysiological Signatures of Melodic and Harmonic Expectancy Processing

Expectancy may be described as the anticipation of a future event based on its probability of occurrence. The concept of musical syntax (Pearce & Rohrmeier, 2018; Rohrmeier & Pearce, 2018) refers to the structural regularities of a musical style that govern these probabilities of occurrence (e.g., Koelsch & Siebel, 2005), whether in terms of melodic, rhythmic, or harmonic structure (Large & Palmer, 2002). A learned, internal model of the syntactic structure of a musical style is thought to determine listeners' musical expectations (Rohrmeier & Koelsch, 2012) and has been shown to influence not just auditory perception across expertise levels (Marmel, Tillmann, & Dowling, 2008; Margulis & Levine, 2006; Bharucha & Stoeckig, 1986) but also music production in expert musicians (e.g., Bianco et al., 2016).

Over the decades, there has been a great deal of research on the EEG correlates of violations of melodic and harmonic structure. One early study, which compared differential ERPs elicited by incongruous events in sentences to those in well-known melodies, revealed that incorrect musical events elicited an amplified N100, as well as a positivity around 300 msec post note onset (Besson & Macar, 1987). Similar EEG signatures were found for terminal deviant notes in well-known melodies in a study incorporating a delay period to allow the build-up of expectancies (Paller, McCarthy, & Wood, 1992). Since these pioneering works, the early negativity and the late positive potential (or P300) have been shown to be influenced by listeners' musical expertise, the music's familiarity, and the strength of the events' incongruity (Omigie, Pearce, Williamson, & Stewart, 2013; Peretz, Brattico, Järvenpää, & Tervaniemi, 2009; Miranda & Ullmann, 2007; Brattico, Tervaniemi, Näätänen, & Peretz, 2006; Besson & Faïta, 1995). Harmonically incongruent events have similarly been shown to elicit early anterior negativities, the amplitudes of which are sensitive to age and expertise, the degree of unexpectedness of the deviant, as well as the presence or absence of veridical knowledge regarding the occurrence of the deviant (e.g., Guo & Koelsch, 2016; Koelsch, Schmidt, & Kansok, 2002; Koelsch, Gunter, Friederici, & Schröger, 2000).

Interestingly, however, although commonalities have been observed, previous work also reveals differences between melodic and harmonic expectancy processing. For instance, the late positivity seen for incongruous musical events has been shown to be more discriminating of harmonic than melodic stimuli (Hantz, Kreilick, Kananen, & Swartz, 1997). Irregular melody endings elicit an ERP effect peaking at 125 msec after the onset of notes but not the later ERP signature observable for irregular final chords. Melodic and harmonic deviants also elicit slightly different scalp distributions of activity (Koelsch & Jentschke, 2010), and neuropsychological work points to differences in the neural substrates of melodic and harmonic syntax processing (Samson & Zatorre, 1988). Taken together, whereas the literature, as a whole, points to some common basis for harmonic and melodic structure processing, these results highlight the importance of studying each domain of musical syntax alone in its own right. A previous study has used intracranial recording to better localize processing of harmonic syntax in the temporal and frontal cortices (Sammler et al., 2013), but intracranial recording has not been used with melodic stimuli.

Oscillatory and Anatomical Correlates of Musical Expectancy

Accompanying the characterization of ERPs to incongruent harmonic and melodic events, there is increasing interest in describing the patterns of oscillatory activity that are associated with such events. It is noteworthy that the degree of violation of musical expectancy has long been associated with power variations in various frequency bands, primarily in right frontal brain regions (Janata & Petsche, 1993). More recently, however, variations in beta power in particular (Pearce, Ruiz, Kapasi, Wiggins, & Bhattacharya, 2010) and long-range synchronization across frontal and temporal regions (Herrojo-Ruiz, Koelsch, & Bhattacharya, 2009) have also been reported as a function of expectancy violations.

When considering predictive processing during music listening, it is relevant to examine not just the electrophysiological signatures of expectation violation but also the anatomical substrates involved in these processes. Neuropsychological studies have shown that patients who have undergone right temporal excisions show deficits in melodic discrimination (Samson & Zatorre, 1988). These studies emphasize the importance of superior temporal gyrus (STG) in musical pitch processing (Zatorre & Samson, 1991). However, neuroimaging studies that have associated computational models of tonal processing with neural processing have further implicated rostral frontal areas in the processing of musical key (Alluri et al., 2012; Janata et al., 2002). A recent meta-analysis confirms that, although regions of the STG are heavily implicated, frontal areas especially are essential for the processing of higher order tonality (Janata, 2015). Some previous computational approaches that have been used to examine tonality processing are based on low-level models of time-varying acoustic features (Alluri et al., 2012). Others have been able to show the anatomical substrates of tonality processing by modeling a higher level concept of tonality. For example, Janata et al. (2002) used domain-general unsupervised machine learning methods (self-organizing maps), trained on a specially constructed melody that rotates between closely aligned keys, to associate low-level pitch percepts with key strengths. The model used in the present research uses domain-general information theoretical principles of statistical learning and probabilistic prediction to learn the syntax of a musical style through exposure. Combined with a high spatial resolution methodology like intracranial EEG, this modeling approach makes it possible to elucidate the anatomical substrates of violations expectations in a precise way.

Although studies specifically examining tonal expectancies have primarily implicated the inferior frontal gyrus (IFG; Musso et al., 2015; Kim, Kim, & Chung, 2011; Sammler, Koelsch, & Friederici, 2011; Tillmann et al., 2006; Koelsch, Fritz, Schulze, Alsop, & Schlaug, 2005; Maess, Koelsch, Gunter, & Friederici, 2001), it is important to note the heterogeneity of this subregion of the frontal lobe (Hartwigsen, Neef, Camilleri, Margulies, & Eickhoff, 2018; Amunts et al., 2010). Indeed, it is worth noting that, whereas previous studies have tended to report sensitivity to musical syntax in Brodmann's area (BA) 44 and BA 45 (Cheung, Meyer, Friederici, & Koelsch, 2018; Bianco et al., 2016; Koelsch & Siebel, 2005), others have rather associated the subjective experience induced by incongruous musical events (i.e., the tension associated with expectation violations) with BA 47 and more orbital frontal areas (Mikutta et al., 2015; Lehne, Rohrmeier, & Koelsch, 2014). In one single-case study, using depth electrode recordings, higher theta power was associated with harmonically unexpected chords in the left OFC (Mikutta et al., 2015). Furthermore, using fMRI, BOLD increases in the pars orbitalis of the left IFG have been associated with music-induced subjective tension (Lehne et al., 2014).

Finally, although early work on expectancy processing has tended to focus on lateral cortical areas, it is interesting to note that recent studies have begun to highlight an important role of subcortical structures. Reanalyzing fMRI data using an ROI analysis, Koelsch, Fritz, and Schlaug (2008) reported greater BOLD activation in both left and right amygdala during the processing of irregular chord endings. Since then, evidence has suggested the involvement of left amygdala theta oscillations in the processing of harmonically unexpected chords (Mikutta et al., 2015, but note data from only one patient was evaluated) and right amygdala activity in mediating subjective tension in response to longer musical pieces (Lehne et al., 2014). Thus, taken together, a growing literature suggests that not only cortical but also subcortical areas may be involved in processing violations of melodic expectation. However, very few studies have characterized the neural signatures of expectation violation using methods, like intracranial EEG, that possess both fine temporal and spectral resolution. Furthermore, despite it being necessary to conclusively demonstrate the neural signatures of predictive error signaling during music listening, few studies have parametrically manipulated the strength of expectation violation induced by the musical stimuli in a highly refined way.

Computational Approaches to Melodic Expectation Modeling

Research examining the where and when of neural oscillatory activity in response to musical expectation violations point to the potential fruitfulness of using music to examine the predictive coding framework (Heilbron & Chait, 2018; Clark, 2013; Friston, 2010). Essentially, contemporary models, within this framework, suggest that predictive processing may involve the coordination of slow and fast brain oscillations. In particular, it has been suggested that whereas a lower (e.g., beta) frequency channel from deep cortical layers propagates top–down information, bottom–up information from superficial cortical layers is propagated using the gamma frequency channel (Wang, 2010). Similar patterns have been reported on a large-scale cortical level by Arnal, Wyart, and Giraud (2011), who exploited the specificity with which visual input predicts auditory input in audiovisual speech and suggested that the gamma power activity they observed for incongruent events (mismatched audiovisual content) in lower sensory regions reflected prediction error signaling.

Just as for audiovisual speech, musical stimuli contain probabilistic regularities that an enculturated listener can be expected to have internalized. Accordingly, musical stimuli whose probabilistic structure has been characterized using computational modeling have the potential to elucidate the neural correlates of predictive processing (e.g., Omigie, Dellacherie, Hasboun, George, et al., 2015). The information dynamics of music model (or IDyOM; Pearce, 2005, 2018) implements the theory that listeners weigh the probability of different possible continuations to a musical excerpt based on the frequency with which different continuations followed similar contexts in their previous experience (Pearce, 2005; Meyer, 1956). Although many studies rely on music-theoretical approaches to characterize musical syntax, IDyOM embodies the hypothesis that listeners internalize the syntactic structure of musical styles through domain-general processes of implicit statistical learning based on long-term exposure to music (Bigand & Poulin-Charronnat, 2006; Tillmann, Bharucha, & Bigand, 2000) and generates probabilistic expectations based on this internalized syntax. Accordingly, models like IDyOM offer the opportunity to consider musical processing in a more domain-general way.

Support for the notion that statistical learning influences expectancy formation comes from listeners' subjective rating of expectedness of specific events in a musical context. For instance, listeners rate small intervals as more expected than large ones, reflecting the relative frequency with which such intervals occur in melodies (Huron, 2006). When required to give subjective ratings of how well each of a set of notes fits a musical pattern (Cuddy & Badertscher, 1987), listeners also produce rating profiles that reflect the tonal hierarchy present in western music. IDyOM has proven very reliable in predicting listeners' explicit unexpectedness and uncertainty ratings (Hansen, Vuust & Pearce, 2016; Hansen & Pearce, 2014; Pearce et al., 2010), as well as their RTs in implicit tasks that probe listeners' melodic expectations (Omigie, Pearce, & Stewart, 2012). IDyOM has also demonstrated high discriminative power. Specifically, it has been able to predict listeners' style-specific expectations (Hansen et al., 2016) and successfully simulate other music cognitive processes such as recognition memory (Agres, Abdallah, & Pearce, 2018), perceptual similarity (Pearce & Müllensiefen, 2017), and metrical inference (Van der Weij, Pearce, & Honing, 2017).

In allowing precise characterization of all events in a melody, computational models of melodic expectation like IDyOM allow the careful interrogation of neural responses to events with respect to their information content (IC; an information theoretical measure of unpredictability) in the context of a naturalistic listening condition. Indeed, an earlier study combining scalp EEG and IDyOM output demonstrated parametric sensitivity to the IC of melodic events of an observed early negativity and later positivity (Omigie et al., 2013). However, that scalp EEG study did not and could not address some of the claims put forward by a predictive coding framework with respect to the anatomical specificity of underlying oscillatory mechanisms. Similarly, although both statistical learning and music-theoretical methods (Egermann, Pearce, Wiggins, & McAdams, 2013; Koelsch, Kilches, Steinbeis, & Schelinski, 2008; Steinbeis, Koelsch, & Sloboda, 2006) have emphasized the ability of music's probabilistic pitch structure to influence listeners' physiological arousal, there is still only sparse evidence for the involvement of the very subcortical regions known to mediate such autonomic nervous system responses. Here, we aimed to identify any consistencies existing between cortical and subcortical responses to parametric manipulations of expectation violation that may confirm a role of the latter in predictive processing of music. In other words, we aimed to examine the possibility that the network of regions involved in predictive processing include subcortical as well as cortical areas.

The Current Study

Our study capitalized on the excellent temporal and spatial resolution of intracranial depth electrode recordings (e.g., Omigie, Dellacherie, Hasboun, Clément, et al., 2015; Omigie, Dellacherie, Hasboun, George, et al., 2015) and state-of-the-art computational modeling of music to shed light on predictive processing of melodic structure, both in terms of patterns of oscillatory activity and the underlying anatomical regions. To take advantage of the detailed stimulus characterization afforded by computational modeling, we analyzed oscillatory activity in response to all notes in all the melodic stimuli presented to participants. Specifically, we examined detailed timing of activity in the STG, middle temporal gyrus (MTG) and pars orbitalis of IFG—all held to be involved in auditory deviant and musical syntax processing—as well as the amygdala, anterior cingulate gyrus (ACG), and insula, three key structures of the limbic network previously implicated in error, syntax, and salience processing as well as, most importantly, in mediating physiological arousal (Lappe, Steinsträter, & Pantev, 2013; Sander, Grafman, & Zalla, 2003; Critchley, Mathias, & Dolan, 2001).

Melodic stimuli used in the current study contained between 32 and 64 notes, each with a duration of 600 msec and separated from the next note by 100 msec. To constrain the scope of our investigation, we use the notion of a prediction unit cycle beginning with prediction-related activity reflecting the likelihood of different note continuations and continuing with prediction error signals following the onset of a given note. Here, it is important to note that we focus on the latter half of this cycle, namely, the time course of prediction error signaling following note onset, which is nevertheless dependent on the existence of predictions regarding what that note would be. We suggest that the feedback-related activity or prediction error signaling following a given note's onset reflects the extent to which predictions mismatch with the actual incoming sensory information and may be expected to occur from as early as within 300 msec of the occurrence of the given incoming note (Dürschmid et al., 2016). Critically, we used IDyOM estimates of IC as our measure of the extent to which listeners' predictions are violated and as the variable with which to predict neural consequences of these violations. In Figure 1, we provide a schematic view of the oscillatory activity we expected to see in different ROIs as a function of increasing IC during the prediction error signaling phase. First, based on a range of EEG studies examining the neural time course of musical syntax processing in frontal and temporal cortices, we also anticipated early increases in low-level frequency bands (theta-alpha power) as a function of increasing note unexpectedness (IC), and this not just in STG but also in syntax processing areas such as MTG and pars orbitalis of the IFG (e.g., Sammler et al., 2011). Furthermore, based on the results of studies examining novelty processing in auditory MMN paradigms in general (Todorovic, Schoffelen, van Ede, Maris, & de Lange, 2015; Haenschel, Baldeweg, Croft, Whittington, & Gruzelier, 2000), we anticipated a modulation of beta power by IC in the STG in particular. Finally, and most importantly, in line with the proposed role of higher frequency activity (gamma) in propagating prediction errors especially in low-level sensory processing areas, we expected the strength of gamma activity to increase with increasing note unexpectedness, also specifically in the STG (Mikutta et al., 2015; Fuentemilla, Marco-Pallarés, Münte, & Grau, 2008). We anticipated that this prediction error-signaling gamma modulation would start at around 140 msec and peak at approximately 230 msec in line with previous studies using oddball paradigms (Dürschmid et al., 2016; El Karoui et al., 2015). Thus, in summary and based on the predictive coding and more general music and auditory deviance detection literature, we predicted a modulation of theta–alpha power in lateral frontal and temporal cortices more generally (STG, MTG, and IFG), followed by a modulation of beta and gamma power in auditory cortices (STG) specifically.

Figure 1. 

A schematic of a unit of the prediction cycle with hypothesized effects.

Figure 1. 

A schematic of a unit of the prediction cycle with hypothesized effects.

Finally, a number of subcortical areas have been associated with salience detection and musical syntax processing, so we examined the possibility that these areas show graded responses to parametric manipulations of expectation violation that would confirm their role in predictive processing. First, in line with evidence of its involvement in the detection of salient events (Sander et al., 2003), the processing of unexpected musical chords (Koelsch, Fritz, et al., 2008), as well as tension in real music (Lehne et al., 2014), we predicted we would observe sensitivity to IC in the amygdala (Mikutta et al., 2015). Furthermore, we predicted that the insula, which has been associated with both musical syntax processing and the mediation of physiological arousal, would, along with the amygdala and ACG, also show evidence of modulation by IC (Cheung et al., 2018; Bianco et al., 2016; Hsu, Lin, Hsu, & Lee, 2014; Lappe et al., 2013). Finally, we assumed that, as ACG is associated with modulating physiological arousal (Critchley et al., 2001) and as increased physiological arousal has been associated with the occurrence of high-IC musical events (Egermann et al., 2013; Steinbeis et al., 2006), we might see modulation of ACG activity as a function of IC. It was difficult to formulate specific hypotheses regarding frequency bands for subcortical oscillatory activity because studies that have revealed a role for these areas in music processing have generally used BOLD activation in fMRI. Thus, although we entertained the idea that these subcortical regions would show gamma activity (potentially indicative of prediction error signaling), we made the conservative hypothesis, based on a previous study that showed expectation violation effects constrained to theta power in the amygdala (Mikutta et al., 2015), that any modulation observed would be constrained to lower frequency bands.

METHODS

Participants

Data were collected from 12 patients implanted with depth intracerebral electrodes for presurgical evaluation at the Epilepsy Unit in the Pitié-Salpêtrière Hospital. However, data from only nine patients (M = 34.4 years, SD = 12.65 years; two men) were analyzed due to highly artifacted data in one patient and no electrodes in relevant regions in the other two (see Table 1). The study was approved by the ethical committee for Biomedical Research of Pitié-Salpêtrière Hospital in Paris (CPP Paris VI, INSERM C11-16). All patients gave informed written consent and clinical investigations were conducted according to the principles expressed in the Declaration of Helsinki.

Table 1. 
Demographic and Background Information on Participating Patients
Patient IDDate of RecordingAgeSexHandednessSeizure Frequency (per Week)Age of Epilepsy OnsetEpileptic Focus (Seizure Onset Regions)LesionCerebral Localization of ImplantationAreas ExaminedOther RegionsMedication
pat_1 03/06/14 62 Left 32 Bilateral anterior hippocampus Right hippocampal sclerosis Right and left temporal lobes rMTG, lMTG, rSTG, lSTG BA 38, HIPP Lamictal, Trileptal Lysanxia 
pat_2 01/07/14 30 Right 16 Superior frontal lobe (BA 6, 8) Dilation of the right temporal horn Right frontal lobe rACG BA 6, 8, 10 Keppra, Lamictal 
pat_4 26/01/15 32 Right 0.25 24 Left temporopolar cortex No lesion Left temporal lobe lAMYG, lMTG, lSTG BA 20, 34, 36, 37, 38, HIPP Keppra, Lamictal, Tegretol 
pat_5 29/04/15 22 Right Right medial frontal lobe No lesion Right frontal lobe rACG, rIFG, rIns, BA 6, 8, 9, 31 Dépakine, Lamictal 
pat_6 20/05/15 34 Right 2.5 Left Parahippocampal gyrus No lesion Left frontal and temporal lobes lACG, lAMYG, lIFG, lInsula lMTG BA 10, 34, 38, 46, HIPP Lamictal, Zonégran 
pat_7 22/06/15 40 Right 23 Left Inferior temporal lobe No lesion Left frontal and temporal lobes lMTG, lSTG BA 20, 28, 38, HIPP Vimpat, Diphantoine 
pat_8 12/01/16 46 Right — — Left temporopolar cortex, middle temporal cortex Right hippocampal sclerosis Right frontal and temporal and left temporal lobe rSTG, rMTG, rIFG, lAMYG BA 20, 35, 38, HIPP Lamictal, Keppra, Urbanyl 
pat_9 08/03/16 24 Right Left temporopolar cortex No lesion Left frontal and temporal lobes lAMYG, lMTG, lSTG BA 20, 38, 46, HIPP Tégrétol, Lamictal 
pat_10 24/05/16 20 Right Left Hippocampus MRI: Hypersignal in Right Sub cortical paraventricular region Bilateral temporal lobe lMTG, rMTG BA 20, 34, 36, 37, 38, HIPP Trileptal, Vimpat, Zonégran, Lamictal, Urbanyl 
Patient IDDate of RecordingAgeSexHandednessSeizure Frequency (per Week)Age of Epilepsy OnsetEpileptic Focus (Seizure Onset Regions)LesionCerebral Localization of ImplantationAreas ExaminedOther RegionsMedication
pat_1 03/06/14 62 Left 32 Bilateral anterior hippocampus Right hippocampal sclerosis Right and left temporal lobes rMTG, lMTG, rSTG, lSTG BA 38, HIPP Lamictal, Trileptal Lysanxia 
pat_2 01/07/14 30 Right 16 Superior frontal lobe (BA 6, 8) Dilation of the right temporal horn Right frontal lobe rACG BA 6, 8, 10 Keppra, Lamictal 
pat_4 26/01/15 32 Right 0.25 24 Left temporopolar cortex No lesion Left temporal lobe lAMYG, lMTG, lSTG BA 20, 34, 36, 37, 38, HIPP Keppra, Lamictal, Tegretol 
pat_5 29/04/15 22 Right Right medial frontal lobe No lesion Right frontal lobe rACG, rIFG, rIns, BA 6, 8, 9, 31 Dépakine, Lamictal 
pat_6 20/05/15 34 Right 2.5 Left Parahippocampal gyrus No lesion Left frontal and temporal lobes lACG, lAMYG, lIFG, lInsula lMTG BA 10, 34, 38, 46, HIPP Lamictal, Zonégran 
pat_7 22/06/15 40 Right 23 Left Inferior temporal lobe No lesion Left frontal and temporal lobes lMTG, lSTG BA 20, 28, 38, HIPP Vimpat, Diphantoine 
pat_8 12/01/16 46 Right — — Left temporopolar cortex, middle temporal cortex Right hippocampal sclerosis Right frontal and temporal and left temporal lobe rSTG, rMTG, rIFG, lAMYG BA 20, 35, 38, HIPP Lamictal, Keppra, Urbanyl 
pat_9 08/03/16 24 Right Left temporopolar cortex No lesion Left frontal and temporal lobes lAMYG, lMTG, lSTG BA 20, 38, 46, HIPP Tégrétol, Lamictal 
pat_10 24/05/16 20 Right Left Hippocampus MRI: Hypersignal in Right Sub cortical paraventricular region Bilateral temporal lobe lMTG, rMTG BA 20, 34, 36, 37, 38, HIPP Trileptal, Vimpat, Zonégran, Lamictal, Urbanyl 

F = Female; M = Male; BA = Brodmann area; MTG = middle temporal gyrus; STG = superior temporal gyrus; ACG = anterior cingulate gyrus; AMYG = amygdala; IFG = inferior frontal gyrus; INS = insula; HIPP = Hippocampus; r = right; l = left.

Stimuli

Stimuli were the same as those used in previous behavioral and scalp EEG studies (Omigie et al., 2012, 2013) and were characterized using IDyOM (Pearce, 2005). The melodies varied in length from 32 to 64 notes and were in a range of major keys. Individual notes were created using the grand piano instrument of a Roland sound canvas MIDI synthesizer before being converted to audio files. To focus on pitch expectations in particular, the rhythmic structure of the melodies had been removed in a musically sensitive manner by a skilled musicologist so that each note had the same duration of 600 msec and an equivalent interonset interval of 700 msec (100 msec between notes). Each note was rendered at the same sound intensity. Finally, to provide an incidental detection task for the patients to engage in, a single note in 6 of the 56 melodies was modified to play in a different timbre, specifically the marimba instrument.

Information Theoretical Characterization of Stimuli

The computational model IDyOM (Pearce, 2005, 2018) learns from a corpus of melodies and the current melody being processed to generate an IC value for each note in a given melody. The configuration of the model used here was exactly as described in previous studies whereby the model was trained on an independent corpus of western tonal melodies as well as the melody itself as it unfolded (Omigie et al., 2012, 2013). The IC of a note is the negative log of the probability of the note's occurrence and indicates the degree of unexpectedness of the given note in its given context (Pearce, 2005). We sought to verify whether a parametric relationship could be observed between note IC and oscillatory activity in particular ROIs encompassing lateral and medial frontal and temporal areas. Accordingly, all the notes of each melody were sorted by their IC, irrespective of which melody they belonged to, and assigned to 10 bins of increasing IC by decile (following the main analysis in Omigie et al., 2013). Figure 2 shows the IC profile of a sample melody used in the experiment, the IC distribution for all notes in the 56 melodies, and descriptive statistics for the IC of notes assigned to the 10 bins. Table 2 shows the mean tonal stability values (Krumhansl & Kessler, 1982), the size of the preceding interval, and the mean position of the notes within the melody. In the western tonal system, the stability of a pitch within a key is related to its position in a hierarchy, and higher ranking/more stable pitches are often more frequent and more predictable than lower ranking ones (Krumhansl, 1990). Also, larger pitch intervals between notes are less frequent than smaller ones in western melodies (Huron, 2001). Accordingly, as can be seen in Table 2, the higher the mean IC of notes in a bin, the more tonally unstable they tended to be, and the greater their tendency to follow larger intervals. Both mean tonal stability per bin and mean interval size were shown to correlate strongly with mean IC per bin (IC and tonal stability: r = −.92, p < .001; IC and interval size, r = .9, p < .001) confirming a contribution of both to the model's IC estimates.

Figure 2. 

Musical stimuli. (A) The IC profile of a single sample melody. (B) The distribution of IC across all notes in the corpus. (C) The mean IC (and SD) of notes in the 10 IC bins.

Figure 2. 

Musical stimuli. (A) The IC profile of a single sample melody. (B) The distribution of IC across all notes in the corpus. (C) The mean IC (and SD) of notes in the 10 IC bins.

Table 2. 
Descriptive Statistics of Notes in the 10 Bins of Increasing IC
 IC Mean (SD)Size of Preceding Interval (Semitones) Mean (SD)Tonal Stability Mean (SD)Midi Pitch Mean (SD)Position Mean (SD)
Bin 1 0.67 (0.22) 1.57 (0.74) 4.77 (0.98) 68.66 (3.51) 21.42 (11.99) 
Bin 2 1.19 (0.09) 1.43 (0.67) 4.90 (1.20) 67.88 (3.24) 20.48 (11.10) 
Bin 3 1.50 (0.10) 1.39 (0.89) 4.79 (1.10) 68.47 (3.27) 19.68 (11.98) 
Bin 4 1.83 (0.09) 1.08 (1.15) 4.72 (1.09) 68.19 (3.26) 21.08 (12.56) 
Bin 5 2.13 (0.08) 1.20 (1.26) 4.54 (1.09) 68.37 (3.56) 20.9 (11.52) 
Bin 6 2.44 (0.10) 1.37 (1.35) 4.45 (1.18) 68.37 (3.26) 19.20 (11.43) 
Bin 7 2.77 (0.10) 1.50 (1.47) 4.42 (1.17) 69.19 (3.33) 19.51 (11.70) 
Bin 8 3.23 (0.17) 2.44 (1.58) 4.28 (1.07) 69.56 (3.13) 18.72 (11.43) 
Bin 9 4.10 (0.36) 3.59 (1.98) 4.27 (1.21) 68.85 (3.67) 19.49 (9.81) 
Bin 10 6.09 (1.21) 5.42 (2.62) 4.11 (1.33) 69.17 (4.02) 19.05 (10.66) 
 IC Mean (SD)Size of Preceding Interval (Semitones) Mean (SD)Tonal Stability Mean (SD)Midi Pitch Mean (SD)Position Mean (SD)
Bin 1 0.67 (0.22) 1.57 (0.74) 4.77 (0.98) 68.66 (3.51) 21.42 (11.99) 
Bin 2 1.19 (0.09) 1.43 (0.67) 4.90 (1.20) 67.88 (3.24) 20.48 (11.10) 
Bin 3 1.50 (0.10) 1.39 (0.89) 4.79 (1.10) 68.47 (3.27) 19.68 (11.98) 
Bin 4 1.83 (0.09) 1.08 (1.15) 4.72 (1.09) 68.19 (3.26) 21.08 (12.56) 
Bin 5 2.13 (0.08) 1.20 (1.26) 4.54 (1.09) 68.37 (3.56) 20.9 (11.52) 
Bin 6 2.44 (0.10) 1.37 (1.35) 4.45 (1.18) 68.37 (3.26) 19.20 (11.43) 
Bin 7 2.77 (0.10) 1.50 (1.47) 4.42 (1.17) 69.19 (3.33) 19.51 (11.70) 
Bin 8 3.23 (0.17) 2.44 (1.58) 4.28 (1.07) 69.56 (3.13) 18.72 (11.43) 
Bin 9 4.10 (0.36) 3.59 (1.98) 4.27 (1.21) 68.85 (3.67) 19.49 (9.81) 
Bin 10 6.09 (1.21) 5.42 (2.62) 4.11 (1.33) 69.17 (4.02) 19.05 (10.66) 

Task

Participants were presented with the 56 melodies in three blocks. They were asked to listen to the individual melodies and indicate after each whether they had heard a note played in a deviant timbre using keys labeled “YES” or “NO” on the presentation laptop. Performance on this task was implemented to keep participants' attention high, and notes from those melodies containing a deviant timbre were not included in the analysis. With regard to performance, participants showed a high hit rate (M = 0.83, SD = 0.12, range = 0.67–1), although the false alarm rate showed a wider range due to two participants who expressed great hesitance with the task and accordingly were encouraged by the experimenter to respond based on their subjective feeling (M = 0.14, SD = 0.28, range = 0–0.77). When these two participants were excluded, a much smaller false alarm rate (M = 0.04, SD = 0.039), in line with expectations, was obtained. Given the irrelevance of the task to the study, no participants were excluded based on their performance.

Intracranial EEG Recordings

Participating patients had been stereotactically implanted with depth electrodes to localize the epileptogenic focus for possible resection. Invasive exploration was planned based on brain locations suspected to be at the origin of epileptic seizures. The hypotheses regarding localization resulted from analysis of data previously gathered in the clinic (examination and history, continuous surface EEG video, MRI and, in some cases, PET and ictal SPECT). Local field potential data were acquired at a sampling rate of 4 kHz (ATLAS System, Neuralynx, Bozeman, MT) and bandpass filtered between 0.05 Hz and 1 kHz. Electrodes were composed of 4–12 contacts, each 2.3 mm long, 5–10 mm apart, mounted on a 1.12-mm wide flexible plastic probe (Ad-Tech Medical Instruments, Racine, WI). These were stereotactically inserted using a Leksell frame (Adam et al., 1996). To determine localization of contacts, the postoperative CT scans were coregistered with the preoperative 1.5-T MRI. The EpiLoc toolbox (Stereotaxy: Techniques, Images, Models) from the Institut du Cerveau et de la Moelle Epinière was used to automatically establish the MNI coordinates, and those coordinates that fell within the regions of the MNI template for each ROI were adopted in a first step. In a second step, such contacts were examined on the postoperative MRI scans using Slicer (Fedorov et al., 2012) and, using further visual inspection, were either confirmed or not as belonging to the given ROIs. To allow conclusions to be drawn with anatomical specificity, only those contacts for which at least one adjacent contact was present in the (ROI) were considered for inclusion in the analysis, although this resulted in fewer contacts available for analysis.

Data Preprocessing and Time–Frequency Analyses

Contact and artifact rejection as well as data preprocessing were carried out using Fieldtrip (Oostenveld, Fries, Maris, & Schoffelen, 2011) and custom-written MATLAB scripts. Data were epoched from 2000 msec before to 2000 msec after the onset of each note. Artifacts related to epileptic activity were automatically removed by excluding (1) trials whose maximum amplitude exceeded the mean amplitude of the trial by at least 5 SDs and (2) contacts in which more than 5% of trials were excluded (following the previous criterion). Trials were also visually checked for spikes and abnormal rhythmic activity. Signals were analyzed in a bipolar montage to minimize the influence of distant sources (Bollimunta, Chen, Schroeder, & Ding, 2008) after subtraction of the signals of adjacent contacts. The Cartesian coordinates (x, y, z) corresponding to the new bipolar signals were calculated as the mid-distance location between two adjacent recording sites, where both sites were located in the ROI. To examine oscillatory activity, data were bandpass filtered to allow the examination of five frequency bands of interest: Delta (1–3 Hz), Theta (4–7 Hz), Alpha (8–12 Hz), Beta (12–30 Hz), and Gamma (30–80 Hz). A Hilbert transform was then carried out on the bandpass filtered data to obtain the analytic signal and accordingly the oscillatory power time course for each epoch. The time course of all epochs was then baselined to the 100-msec window directly preceding the onset of the sound by subtracting the mean power during this period from the entire epoch.

Statistical Analysis

Predictive processing was studied in 81 intracranial contacts across nine epileptic patients. Activity was recorded from cortical (67 contacts) and subcortical structures (14 contacts). The distribution, number, and location of contacts in each of the ROIs for all nine participants are shown in Figure 3 and Table 3, respectively. To control for the variance associated with different contacts and patients, data were analyzed using linear mixed-effects models with the lme4 package in R (Bates, Mächler, Bolker, & Walker, 2015). Each trial, representing an individual note in an individual melody, was split into equal-sized time windows (e.g., TW1: 0–100 msec, TW2: 100–200 msec) to determine the relative latency of significant modulations of neural activity by IC. Within these windows, mean spectral amplitude for each contact, for each patient, and for each frequency band in a particular ROI was estimated. The amplitude of oscillatory power served as the dependent variable in the linear mixed-effect models (for each band, for each time window) with IC bin (IC) as fixed effect and contacts and participants as random factors with intercept to account for any variation across individual contacts and participants. To boost statistical power (necessary given the limited number of contacts), we analyze contacts from the left and right hemispheres together initially (El Karoui et al., 2015). However, we examine the lobes separately in case of nonsignificant effects to ensure that nonsignificant effects were not due to contacts showing contrasting effects. p Values reported are obtained using the anova function from the car toolbox in R and are false discovery rate-corrected for multiple comparisons (across time windows and across frequency bands) for each ROI separately. Finally, given the heterogeneous regions measured from and to complement the previous analysis, further models examining modulation of activity in each contact individually were estimated. As using false discovery rate to correct across time window, frequency bands, and contacts resulted in no effects surviving this conservative approach, results using less conservative alpha values of p < .01 are presented.

Figure 3. 

Distribution of contacts in cortical (small circles) and subcortical (large circles) ROIs. STG: blue, MTG: black, pars orbitalis of IFG: red, ACG: pink, amygdala: yellow, insula: green. Note that right hemisphere data have been projected onto the left hemisphere.

Figure 3. 

Distribution of contacts in cortical (small circles) and subcortical (large circles) ROIs. STG: blue, MTG: black, pars orbitalis of IFG: red, ACG: pink, amygdala: yellow, insula: green. Note that right hemisphere data have been projected onto the left hemisphere.

Table 3. 
Distribution of Contacts in the Different ROIs
ROINumber of Participants with Contacts in ROITotal Number of Contacts in the ROIMNI Coordinates of Contacts
STG 21 [50.9 −8.7 −7.5], [48.1 −13.5 −9.7], [54.5 −10.9 −8.1], [−48.1 −21.3 −6.8], [−46.7 −18.8 −7.5], [−52.3 −19.9 −4.5], [−48.6 −33.4 8.9], [−51.5 −35.1 12.3], [−55.4 −36.0 16.5], [−59.3 −38.0 19.6], [−47.9 0.6 −17.3], [−51.2 0.7 −13.9], [−59.0 0.9 −8.1], [48.7 −32.0 6.2], [51.8 −33.4 9.4], [57.0 −36.0 12.6], [60.2 −37.5 15.6], [64.5 −39.8 18.9], [−49.5 −17.9 −7.5], [−55.3 −18.0 −4.0], [−63.9 −23.7 −0.3] 
MTG 33 [45.9 −10.6 −10.9], [44.1 2.2 −29.9], [50.7 3.1 −30.3], [56.2 3.6 −28.5], [−52.0 4.7 −25.0], [−52.1 −2.8 −18.4], [−56.8 −0.8 −18.1], [−62.7 0.3 −17.8], [−55.6 −8.0 −22.0], [−60.3 −45.1 −11.7], [−64.1 −46.1 −8.6], [−57.8 −25.3 −12.3], [−64.7 −26.6 −11.0], [−71.6 −28.6 −8.1], [−45.1 −11.9 −13.4], [−55.5 −9.2 −19.1], [−52.8 −26.3 −15.1], [−57.6 −28.7 −13.7], [−62.2 −30.6 −13.2], [−67.0 −32.7 −13.0], [40.2 −1.4 −32.1], [47.5 0 −30.8], [−56.3 −14.8 −21.9], [−54.8 −2.0 −21.3], [−54.1 8.0 −30.3], [−58.3 5.9 −29.1], [−53.2 −41.5 −6.7], [−58.3 −43.3 −7.0], [−62.5 −45.2 −5.8], [−52.4 −19.8 −15.6], [−59.7 −19.7 −14.3], [52.4 −18.8 −18.0], [58.8 −19.3 −17.9] 
IFG pars orbitalis 13 [37.7 29.5 −3.4], [41.9 31.8 −3.0], [46.1 34.3 −2.6], [49.3 37.0 −2.1], [−29.1 25.9 −12.7], [−44.1 36.5 −4.9], [22.1 15.4 −23.8], [26.7 19.6 −20.5], [29.9 23.3 −18.4], [34.2 26.0 −16.1], [37.3 28.8 −13.9], [41.6 33.1 −10.7], [45.9 35.6 −8.8] 
ACG [9.9 30.6 30.0], [8.8 46.0 6.9], [9.6 17.7 34.5], [18.5 22.3 31.0], [−15.9 48.4 −11.4] 
Left AMY [−19.4 −5.1 −21.3], [−26.7 −3.4 −22.5], [−13.7 −4.2 −25.7], [−19.8 −1.4 −27.7] 
Insula [33.5 26.9 −3.7], [42.8 15.7 8.8], [47.0 17.0 10.9], [−39.8 8.5 8.3], [−42.5 10.7 11.1] 
ROINumber of Participants with Contacts in ROITotal Number of Contacts in the ROIMNI Coordinates of Contacts
STG 21 [50.9 −8.7 −7.5], [48.1 −13.5 −9.7], [54.5 −10.9 −8.1], [−48.1 −21.3 −6.8], [−46.7 −18.8 −7.5], [−52.3 −19.9 −4.5], [−48.6 −33.4 8.9], [−51.5 −35.1 12.3], [−55.4 −36.0 16.5], [−59.3 −38.0 19.6], [−47.9 0.6 −17.3], [−51.2 0.7 −13.9], [−59.0 0.9 −8.1], [48.7 −32.0 6.2], [51.8 −33.4 9.4], [57.0 −36.0 12.6], [60.2 −37.5 15.6], [64.5 −39.8 18.9], [−49.5 −17.9 −7.5], [−55.3 −18.0 −4.0], [−63.9 −23.7 −0.3] 
MTG 33 [45.9 −10.6 −10.9], [44.1 2.2 −29.9], [50.7 3.1 −30.3], [56.2 3.6 −28.5], [−52.0 4.7 −25.0], [−52.1 −2.8 −18.4], [−56.8 −0.8 −18.1], [−62.7 0.3 −17.8], [−55.6 −8.0 −22.0], [−60.3 −45.1 −11.7], [−64.1 −46.1 −8.6], [−57.8 −25.3 −12.3], [−64.7 −26.6 −11.0], [−71.6 −28.6 −8.1], [−45.1 −11.9 −13.4], [−55.5 −9.2 −19.1], [−52.8 −26.3 −15.1], [−57.6 −28.7 −13.7], [−62.2 −30.6 −13.2], [−67.0 −32.7 −13.0], [40.2 −1.4 −32.1], [47.5 0 −30.8], [−56.3 −14.8 −21.9], [−54.8 −2.0 −21.3], [−54.1 8.0 −30.3], [−58.3 5.9 −29.1], [−53.2 −41.5 −6.7], [−58.3 −43.3 −7.0], [−62.5 −45.2 −5.8], [−52.4 −19.8 −15.6], [−59.7 −19.7 −14.3], [52.4 −18.8 −18.0], [58.8 −19.3 −17.9] 
IFG pars orbitalis 13 [37.7 29.5 −3.4], [41.9 31.8 −3.0], [46.1 34.3 −2.6], [49.3 37.0 −2.1], [−29.1 25.9 −12.7], [−44.1 36.5 −4.9], [22.1 15.4 −23.8], [26.7 19.6 −20.5], [29.9 23.3 −18.4], [34.2 26.0 −16.1], [37.3 28.8 −13.9], [41.6 33.1 −10.7], [45.9 35.6 −8.8] 
ACG [9.9 30.6 30.0], [8.8 46.0 6.9], [9.6 17.7 34.5], [18.5 22.3 31.0], [−15.9 48.4 −11.4] 
Left AMY [−19.4 −5.1 −21.3], [−26.7 −3.4 −22.5], [−13.7 −4.2 −25.7], [−19.8 −1.4 −27.7] 
Insula [33.5 26.9 −3.7], [42.8 15.7 8.8], [47.0 17.0 10.9], [−39.8 8.5 8.3], [−42.5 10.7 11.1] 

RESULTS

Superior Temporal Gyrus

Figure 4 shows the time course of oscillatory power as a function of IC across all trials across the 21 contacts in bilateral STG (13 left and 8 right contacts) with gray shading indicating the specific time windows (TWs) in which IC significantly predicted oscillatory power. Significant modulation was found from within 100 msec and continued until 400 msec after note onset in the theta band (TW1: β = 0.03, SE = 0.005, p < .001; TW2: β = 0.05, SE = 0.008, p < .001: TW3: β = 0.05, SE = 0.01, p < .001; TW4: β = 0.03, SE = 0.01 p = .005), until 300 msec after onset in the alpha band (TW1: β = 0.03, SE = 0.007, p ≤ .001; TW2: β = 0.07, SE = 0.01, p < .001; TW3: β = 0.05, SE = 0.01, p < .001), and until 200 msec after onset in the beta band (TW1: β = 0.02, SE = 0.009, p < .05; TW2: β = 0.04, SE = 0.009, p < .001). A medium latency modulation was found in the gamma band between 200 and 400 msec (TW3: β = 0.01, SE = 0.003, p < .001; TW4: β = 0.01, SE = 0.003, p < .001) whereas, finally, modulation by IC was found between 500 and 600 msec after onset in both the alpha (TW6: β = −0.03, SE = 0.01, p < .05) and in the gamma band (TW6: β = 0.008, SE = 0.003, p < .05).

Figure 4. 

The time course of spectral power changes in the different frequency bands in bilateral STG, MTG, and orbital IFG as a function of IC bin. Blue lines show time courses for the three lowest IC bins, green for the four middle bins, and red for the three highest bins. Gray shading indicates the time windows showing significant modulation by IC bin. STG = superior temporal gyrus; MTG = middle temporal gyrus; IFG = inferior temporal gyrus; r = right; l = left.

Figure 4. 

The time course of spectral power changes in the different frequency bands in bilateral STG, MTG, and orbital IFG as a function of IC bin. Blue lines show time courses for the three lowest IC bins, green for the four middle bins, and red for the three highest bins. Gray shading indicates the time windows showing significant modulation by IC bin. STG = superior temporal gyrus; MTG = middle temporal gyrus; IFG = inferior temporal gyrus; r = right; l = left.

Figure 5 shows the results of the linear regression modeling on the single contact level. Here, 50% (4 of the 8) of contacts in the right temporal lobe and 31% (4 of 13) of contacts in the left temporal lobe showed modulation at a significance level of p < .01. The previously observed pattern of lower frequency activity being followed by higher frequency activity was corroborated in these analyses as low-frequency activity (theta to beta) for individual contacts was observed from within 100 to 300 msec (and then again at 500–600 msec), whereas effects in the gamma band were only present from 200 to 400 msec. Finally, it is interesting to note that, although both anterior and posterior STG contacts showed modulation in the theta and beta bands, only posterior STG contacts showed modulation in the alpha and gamma band.

Figure 5. 

The distribution of contacts in the STG, MTG, and orbital IFG showing modulation by IC significant at p < .01. STG: blue, MTG: black, pars orbitalis of the IFG BA 47: red. Note that right hemisphere data have been projected onto the left hemisphere. STG = superior temporal gyrus; MTG = middle temporal gyrus; IFG = inferior temporal gyrus; r = right; l = left.

Figure 5. 

The distribution of contacts in the STG, MTG, and orbital IFG showing modulation by IC significant at p < .01. STG: blue, MTG: black, pars orbitalis of the IFG BA 47: red. Note that right hemisphere data have been projected onto the left hemisphere. STG = superior temporal gyrus; MTG = middle temporal gyrus; IFG = inferior temporal gyrus; r = right; l = left.

Middle Temporal Gyrus

Thirty-three contacts in the left (25 contacts) and right (8 contacts) MTG were examined. Linear mixed-model analysis with all contacts in the MTG showed some evidence for modulation of activity in lower frequency bands, followed by modulation in gamma bands, but these modulations did not survive correction for multiple comparisons. Running the analysis on the right temporal lobe contacts alone revealed a significant modulation in beta power between 100 and 200 msec (Beta: TW2: β = 0.04, SE = 0.01, p = .05) and a trend in the gamma band (β = −0.007, SE = 0.002, p = .08) whereby gamma power was reduced for higher IC notes. No significant modulation of activity in the left temporal lobe was observed. Finally, 37.5% (3 of a possible 8) of contacts in the right and 24% (6 of a possible 25) of contacts in the left lobe showed modulation either within the first 200 msec in lower frequency bands (predominantly in delta but also in theta and alpha bands; see Figure 5) or at later time windows in alpha and gamma (300–600 msec) whereby power in the alpha and gamma bands were reduced for high-IC events. Visual inspection suggests that contacts showing modulation across the different frequency bands were distributed across both anterior and posterior MTG.

IFG Pars Orbitalis

Linear mixed-effect model analysis of 13 contacts in both lobes (2 left and 11 right) suggested modulation in alpha (0–100 msec) and delta (200–500 msec) power, but neither this grand analysis nor the analysis of the hemispheres separately survived correction for multiple comparisons, although, as in STG and MTG, the distribution and timing of activity on a single-electrode level (p < .01) showed gamma frequency activity having the longest latency. Specifically, alpha–beta activity within the first 100 msec and delta between 300 and 500 msec was later followed by beta–gamma band activity between 500 and 600 msec. Modulation was observed in 50% (1 of a possible 2) of the left contacts and 18.2% (2 of the possible 11) of contacts in the right lobe.

Anterior Cingulate Gyrus, Insula, and Amygdala

Figure 6 shows the time course of spectral power in bilateral ACG, with gray shading indicating the specific time windows at which IC significantly predicted amplitude of oscillatory power. Five ACG contacts (one left, four right) were examined. The ACG showed significant modulation by IC in the theta band between 200 and 500 msec (TW3: β = 0.03, SE = 0.01, p < .05; TW4: β = 0.04, SE = 0.01, p < .05; TW5: β = 0.03, SE = 0.01, p = .05), in the alpha band between 100 and 300 msec (TW2: β = 0.03, SE = 0.01, p < .05; TW3: β = 0.03, SE = 0.01, p < .05), and in the gamma band between 400 and 500 msec (TW5: β = 0.01, SE = 0.004, p < .05). Single-electrode analysis showed two contacts (one in the left lobe [100%] and one of a possible four in the right [25%]) in the ventral anterior cingulate region showing significant modulation in the theta band (0–300 msec) and then in the alpha band (200–400 msec; see Figure 7).

Figure 6. 

The time course of spectral power changes in the different frequency bands in bilateral ACG, left amygdala, and bilateral insula as a function of IC bin. Blue lines show time courses for the three lowest IC bins, green for the four middle bins, and red for the three highest bins. Gray shading indicates the time windows showing significant modulation by IC bin. ACG = anterior cingulate gyrus; AMYG = amygdala; INS = insula; r = right; l = left.

Figure 6. 

The time course of spectral power changes in the different frequency bands in bilateral ACG, left amygdala, and bilateral insula as a function of IC bin. Blue lines show time courses for the three lowest IC bins, green for the four middle bins, and red for the three highest bins. Gray shading indicates the time windows showing significant modulation by IC bin. ACG = anterior cingulate gyrus; AMYG = amygdala; INS = insula; r = right; l = left.

Figure 7. 

The distribution of contacts in the ACG, insula, and amygdala showing modulation by IC significant at p < .01. ACG: pink, amygdala: yellow, insula: green. Note that right hemisphere data have been projected onto the left hemisphere. ACG = anterior cingulate gyrus.

Figure 7. 

The distribution of contacts in the ACG, insula, and amygdala showing modulation by IC significant at p < .01. ACG: pink, amygdala: yellow, insula: green. Note that right hemisphere data have been projected onto the left hemisphere. ACG = anterior cingulate gyrus.

At the single-contact level, five contacts in the left (two) and right (three) insula and four contacts in the left amygdala were examined. Although a grand linear mixed-effect analysis of insula contacts did not show any effects that survived correction for multiple comparison (see Figure 6), analysis of contacts in the left insula alone revealed significant modulation in the delta band (TW1: β = 0.03, SE = 0.007, p < .001; TW2: β = 0.06, SE = 0.014, p < .001; TW3: β = 0.07, SE = 0.019, p < .001; TW4: β = 0.08, SE = 0.02, p =.001; TW5: β = 0.07, SE = 0.02, p < .05) and gamma band (TW4: β = 0.017, SE = 0.006, p < .05), but no significant modulation in the right hemisphere. Single-electrode analysis confirmed that two insula contacts (of a possible two in the left lobe, 100%) showed modulation from 0 to 400 msec in delta band followed by modulation in gamma band from 300 to 400 msec. Finally, analysis of amygdala contacts together did not survive correction for multiple comparisons, although analysis of single contacts showed a single contact displaying significance between 100 and 200 msec in the beta band (see Figure 7).

DISCUSSION

The current study examined the extent to which the information theoretical predictability of a musical note modulates the oscillatory activity in both (a) lateral frontal and temporal areas involved in auditory sequence processing and (b) amodal limbic areas involved in salience detection and physiological arousal. As we were particularly interested in the time course of prediction error signaling in these regions, we used IC as a measure of expectation violation (or prediction error) to predict oscillatory activity post onset of the musical notes. Contrary to our hypotheses, neither the observed modulation of IFG pars orbitalis nor the observed modulation of the left amygdala activity survived correction for multiple comparisons across time windows and frequency bands. However, we were able to provide robust evidence of parametric modulation of oscillatory activity by expectation violations in the STG, the right MTG, the ACG (primarily in ventral regions), and the left insula. Presence and timing of the observed oscillatory activity, notably in the gamma frequency band, showed congruence with patterns found in predictive processing in other domains. Thus, our findings emphasize the utility of musical stimuli in revealing the wide-reaching neural consequences of predictive processing in general.

The Role of the Lateral Temporal Lobe in Melodic Expectancy and Predictive Processing

We predicted that STG would show robust modulation by IC in lower (theta to beta) and higher (gamma) frequency bands, with the latter, in particular, reflecting auditory prediction error signaling (Arnal et al., 2011). Accordingly, our results showed a broadband increase in bilateral STG oscillatory activity as a function of increasing IC that was most extensive in the theta (0–400 msec) and most short-lived in the beta band (0–200 msec). Furthermore and most critically, our results in the STG confirmed robust gamma modulation by IC, which, compared with lower frequency bands, had the longest latency—emerging at approximately 200 msec and then again at 500 msec post note onset.

On the one hand, the pattern of oscillatory activity observed here is easily reconcilable with a large number of studies that link theta–alpha and beta–gamma activity to auditory change detection and music syntax processing more specifically (e.g., El Karoui et al., 2015; Choi et al., 2013; Fuentemilla et al., 2008). Greater theta, alpha, and beta power have been reported for higher IC musical events (Carrus, Pearce, & Bhattacharya, 2013; Pearce et al., 2010), whereas beta power in the STG, in general, is often associated with novelty processing in auditory MMN paradigms (Haenschel et al., 2000; Todorovic et al., 2015). On the other hand, the findings are also particularly important in corroborating a pattern of activity that has often been described in association with the predictive coding framework (Arnal et al., 2011). Specifically, here, we show robust IC modulation of activity in the gamma band, which has been argued to propagate prediction errors following the comparison of predictions with the incoming sensory events. That this gamma activity modulation is limited to STG in the temporal lobe and specifically to posterior STG contacts is consistent with the notion that such bottom–up information originates from lower level sensory processing areas. Furthermore, although its latency may seem to be longer than in previous studies (e.g., Dürschmid et al., 2016), it is important to note that our use of 100-msec time windows limits the precision with which the onset of this gamma sensitivity can be reported, and indeed, it can be seen that the sensitivity does seem to emerge as early as half way into the preceding time window (100–200 msec). Finally, that beta activity, which has been argued to both carry top–down predictions (Arnal et al., 2011) as well as index expectation violations (Todorovic et al., 2015) also showed robust modulation by IC here, emphasizes this frequency band as playing a critical role in predictive processing in the brain.

Although STG houses the primary auditory areas and may thus be expected to play the most critical role, an extensive literature, documenting studies carried out with patients and healthy participants, demonstrates a more general role of the right lateral temporal lobe in melodic pitch processing (e.g., Warrier & Zatorre; 2004; Warren & Griffiths, 2003; Patterson, Uppenkamp, Johnsrude, & Griffiths, 2002; Johnsrude, Penhune, & Zatorre, 2000). In addition to the IFG and the STG, the MTG has been implicated in music syntactic (Sammler et al., 2013) and syntax processing more generally (Brennan et al., 2012; Kaan, & Swaab, 2002). However, the timing and nature of oscillatory activity in response to predictive processing in this region remained unclear. Here, we anticipated we would observe lower (theta–alpha) modulation in MTG as a function of IC. Interestingly, we observed that parametric modulation of oscillatory activity in this region was primarily in the right hemisphere and particularly in the beta range. Here, it important to note that, whereas previous studies have shown beta power to be modulated by musical note IC (Carrus et al., 2013), ours is the first study to show that such beta oscillations may emanate not just from the bilateral STG but also from the right MTG. In other words, we show for the first time a specific involvement of right MTG beta power in the processing of melodic violations.

In addition to these beta band effects in the right MTG, we also observed a trend toward reduced MTG gamma power (between 300 and 600 msec) for high-IC events that was corroborated by the single contact analyses. Although it is necessary to be cautious with these findings, it is interesting to note that they resonate with that from a scalp EEG study that also reported decreases in gamma power in response to irregular musical chords (Herrojo-Ruiz et al., 2009). Those authors interpreted their results within the framework of the match-and-utilization model and under the assumption that tonic chords constitute a match. However, the limited spatial resolution afforded by that scalp EEG study would have masked the MTG as the source of this reduced gamma signal. Critically, it would also have masked the presence and source of an effect that we see here and which is more in line with the predictive coding framework—namely an increase in gamma activity reflecting prediction error signaling. Thus, taken together, previous noninvasive EEG methods have contributed greatly to our understanding of the neural correlates of predictive processing. However, by providing here, electrophysiological evidence of differing patterns of oscillatory activity in close-lying lateral temporal areas (the STG and MTG), our study highlights the utility of intracranial EEG, combined with computational modeling of music, in this field of inquiry.

The Role of Ventral Anterior Cingulate Gyrus and Insula in Melodic Expectancy and Predictive Processing

In addition to aiming to throw light on the timing of predictive error signaling in lateral cortical areas, a key aim of the current research was to examine whether parallels may be seen across cortical and subcortical areas. Our motivation to investigate this emerged from the observation that areas like the insula and ACG are not only involved with salience detection, attention orientation, and musical syntax processing but also in the mediation of physiological arousal. Here, we provide valuable confirmation of a role in music predictive processing of key subcortical structures that are difficult to isolate using noninvasive EEG (Deouell, 2007). Specifically, in addition to showing modulation of the STG and the right MTG oscillatory activity by IC, our results also showed systematic IC modulation in ventral regions of the ACG and insula, whereby increasingly high-IC events resulted in a sustained increase of oscillatory activity.

Modulation of ventral ACG oscillatory activity by IC was observed first in the theta (within 200 msec of note onset) and then in the alpha band. (within 100 msec). However, interestingly and mirroring STG activity to a degree not fully anticipated, ACG modulation was also observable later (from 400 to 500 msec) in the gamma frequency band. With regard to what the activity in this region may reflect during the prediction error signaling process, one possibility is that it is related to the greater attentional resources that are likely being allocated to increasingly high-IC events (Bush, Luu, & Posner, 2000; Carter et al., 1998). However, a further compelling explanation for this observed modulation is that it reflects the role of the ventral ACG in the processing and integration of emotional information (Mayberg, 1997) and in mediating an interplay between cognitive processing and sympathetic nervous system activity (Beissner, Meissner, Bär, & Napadow, 2013; Critchley et al., 2001, 2003). Initial evidence that high IC notes in a melody can lead to modulation of the autonomic nervous system response was provided by Egermann et al. (2013), who demonstrated that unpredictable passages characterized by high IC led to greater physiological arousal. Electrical stimulation of both ventral and dorsal regions of the ACG have long been shown to result in the modulation of heart rate and blood pressure in animals (Chefer, Talan, & Engel, 1997; Ward, 1948), whereas in more recent work on humans, sympathetic activity has also been observed to be related to ACG activity (Beissner et al., 2013; Critchley et al., 2001, 2003). Therefore, one possibility is that, despite using isochronous melodies only, the observed ventral ACG modulation reflects a mechanism by which violations of melodic expectation can induce changes in emotional arousal (Meyer, 1956). Such a potential role is supported by our findings of a similar modulation (albeit in delta and gamma power) in the insula, which, in addition to being implicated in deviance detection (Hsu et al., 2014) and musical violation processing (Cheung et al., 2018; Bianco et al., 2016; Lappe et al., 2013), has also been associated with mediating physiological arousal (Singer, Critchley, & Preuschoff, 2009).

Finally, given our intention to examine consistencies across cortical and subcortical areas, it is interesting to note that the patterns of low- and high-frequency activity we observed in ventral ACG and insula were highly consistent in terms of time course (from low- to high-frequency bands) with those found in the STG and MTG. No study to date has shown a sensitivity of gamma activity in the ventral ACG or insula to the strength of violated expectations, and we argue that one of the valuable contributions of this study is showing this similar pattern of neuro-oscillatory responding across cortical and subcortical areas. Without being able to estimate connectivity between these cortical and subcortical areas (e.g., as in Omigie, Dellacherie, Hasboun, George, et al., 2015), we, unfortunately, cannot comment on the extent to which the effects in these different regions are related. However, we suggest that this constitutes a very interesting avenue of research that could be fruitfully pursued in future studies using intracranial recordings combined with the computational modeling approach we demonstrate here.

IFG Pars Orbitalis and Amygdala during Music Listening

The IFG is widely held to be involved in predictive processing more generally and in music syntax processing specifically. However, our failure to show a strong role of pars orbitalis IFG in melodic expectation processing here may not be considered so surprising given that previous studies have mainly reported responses to music-syntactic violations in BA 44 and BA 45 subregions of the IFG (Cheung et al., 2018; Bianco et al., 2016; Koelsch & Siebel, 2005). Despite the fact that BA 47 has not been as widely implicated in predictive processing as BA 44 and BA 45, our expectation of modulation in this region was primarily based on findings that tension resulting from complex musical structure modulates the OFC and particularly the pars orbitalis of the IFG. Bearing that in mind, our failure to show robust oscillatory activity modulation in IFG goes against the results of Lehne et al. (2014) (who showed the left pars orbitalis IFG), as well as against those of Mikutta and colleagues (who showed the left OFC activity to be involved in tension processing; Mikutta et al., 2015). It is, however, important to note that we did observe patterns somewhat in line with those studies. Specifically, we observed a greater proportion of the left than the right contacts showing modulation by IC and single contacts in the pars orbitalis of the IFG showed modulation from within 100 to 600 msec after sound onset, although these effects failed to survive correction for multiple comparisons. Furthermore, although lack of power cannot be excluded as a reason for the absent effect here, it is worth noting that the hierarchy of stability may be less robust for individual tones than for chords, and therefore, the neural correlates of deviant processing in the IFG may not be as pronounced for melodies (as employed here) as for harmony (Koelsch, 2012).

Lastly, because the amygdala is considered part of a network that, along with the OFC, modulates attention to stimuli (Vuilleumier, 2005), we hypothesized a role for this region in responding to expectation-violating melodic events (e.g., Zarcone, Van Schijndel, Vogels, & Demberg, 2016). However, our results showed little evidence of modulation of amygdala power by IC. Once again, it is possible that melodic structure does not modulate the amplitude of activity in this region to the extent that manipulations of harmonic structure are able to. After all, here, even the highest IC notes modulating neural activity are not deviant per se, compared with the highly irregular events used in other studies (Koelsch, Fritz, et al., 2008). Other possibilities, however, are that the right amygdala, which was not recorded from in this study, is more involved than the left amygdala in processing musically salient events (Lehne et al., 2014), or that, quite simply, the current stimuli used lack the complexity and richness necessary for recruiting emotion and reward areas during the cognitive processing of syntax.

Conclusions

In summary, our findings throw light on the time course of neural activation involved in music prediction error signaling, extending previous work on melodic and harmonic processing that has relied on noninvasive EEG/MEG and neuroimaging. Critically, our results emphasize the usefulness of a computational approach that, by allowing the estimation of conditional probabilities for individual events, offers a fine-grained quantification of the predictability of musical events that is difficult to obtain using purely music-theoretical analysis. Our use of a model based on domain-general principles of information theory yields observations about brain activity that are potentially generalizable across domains (Strange, Duggins, Penny, Dolan, & Friston, 2005) and may thus be profitably extended to provide further domain-general understanding of predictive processing in the brain.

Nevertheless, the current study has a number of shortcomings, which, though shared with most intracranial EEG studies (Mukamel & Fried, 2012), deserve elaboration. First, the findings rely on a small number of contacts from a restricted number of patients, thus limiting the generalizability of significant findings and throwing doubt on some null results, which may be Type II errors. Second, because of little overlap in contact distribution across patients, the current analysis had to focus on the effect of IC on different regions in isolation and could not investigate how these regions interact, as has proven highly enlightening in previous work (Omigie, Dellacherie, Hasboun, George, et al., 2015). Third, to minimize participation effort, our paradigm did not require participants to give feedback on subjective feelings of surprise or emotional arousal, which limits our capacity to draw strong conclusions about affective processing of the stimuli. Future studies in which data are collected from a greater number of patients, each implanted in different key ROIs simultaneously, would allow valuable insight into interaction within the network of brain areas mediating the processing of musical expectancy. Most importantly, however, studies that combine (a) objective quantification using computational models of the kind employed here with (b) stimuli with the capacity to induce greater affective responses and reward (Alluri et al., 2012) will throw more light—than was possible here—on the role that predictive processing plays in the aesthetic and emotional experiencing of music.

Acknowledgments

We would like to extend our sincerest thanks to all patients who took part in this research. We would also like to thank our reviewers for their comments and suggestions on earlier drafts of this paper.

Reprint requests should be sent to Diana Omigie, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322 Frankfurt am Main, Germany, or via e-mail: d.omigie@gmail.com.

REFERENCES

REFERENCES
Adam
,
C.
,
Clemenceau
,
S.
,
Semah
,
F.
,
Hasboun
,
D.
,
Samson
,
S.
,
Aboujaoude
,
N.
, et al
(
1996
).
Variability of presentation in medial temporal lobe epilepsy: A study of 30 operated cases
.
Acta Neurologica Scandinavia
,
94
,
1
11
.
Agres
,
K.
,
Abdallah
,
S.
, &
Pearce
,
M.
(
2018
).
Information-theoretic properties of auditory sequences dynamically influence expectation and memory
.
Cognitive Science
,
42
,
43
76
.
Alluri
,
V.
,
Toiviainen
,
P.
,
Jääskeläinen
,
I. P.
,
Glerean
,
E.
,
Sams
,
M.
, &
Brattico
,
E.
(
2012
).
Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm
.
Neuroimage
,
59
,
3677
3689
.
Amunts
,
K.
,
Lenzen
,
M.
,
Friederici
,
A. D.
,
Schleicher
,
A.
,
Morosan
,
P.
,
Palomero-Gallagher
,
N.
, et al
(
2010
).
Broca's region: Novel organizational principles and multiple receptor mapping
.
PLoS Biology
,
8
,
e1000489
.
Arnal
,
L. H.
,
Wyart
,
V.
, &
Giraud
,
A. L.
(
2011
).
Transitions in neural oscillations reflect prediction errors generated in audiovisual speech
.
Nature Neuroscience
,
14
,
797
801
.
Bates
,
D.
,
Mächler
,
M.
,
Bolker
,
B.
, &
Walker
,
S.
(
2015
).
Fitting linear mixed-effects models using lme4
.
Journal of Statistical Software
,
67
,
1
48
.
Beissner
,
F.
,
Meissner
,
K.
,
Bär
,
K. J.
, &
Napadow
,
V.
(
2013
).
The autonomic brain: An activation likelihood estimation meta-analysis for central processing of autonomic function
.
Journal of Neuroscience
,
33
,
10503
10511
.
Besson
,
M.
, &
Faïta
,
F.
(
1995
).
An event-related potential (ERP) study of musical expectancy: Comparison of musicians with nonmusicians
.
Journal of Experimental Psychology: Human Perception and Performance
,
21
,
1278
1296
.
Besson
,
M.
, &
Macar
,
F.
(
1987
).
An event-related potential analysis of incongruity in music and other non-linguistic contexts
.
Psychophysiology
,
24
,
14
25
.
Bharucha
,
J. J.
, &
Stoeckig
,
K.
(
1986
).
Reaction time and musical expectancy: Priming of chords
.
Journal of Experimental Psychology: Human Perception and Performance
,
12
,
403
410
.
Bianco
,
R.
,
Novembre
,
G.
,
Keller
,
P. E.
,
Kim
,
S. G.
,
Scharf
,
F.
,
Friederici
,
A. D.
, et al
(
2016
).
Neural networks for harmonic structure in music perception and action
.
Neuroimage
,
142
,
454
464
.
Bigand
,
E.
, &
Poulin-Charronnat
,
B.
(
2006
).
Are we experienced listeners? A review of the musical capacities that do not depend on formal musical training
.
Cognition
,
100
,
100
130
.
Bollimunta
,
A.
,
Chen
,
Y.
,
Schroeder
,
C. E.
, &
Ding
,
M.
(
2008
).
Neuronal mechanisms of cortical alpha oscillations in awake-behaving macaques
.
Journal of Neuroscience
,
28
,
9976
9988
.
Brattico
,
E.
,
Tervaniemi
,
M.
,
Näätänen
,
R.
, &
Peretz
,
I.
(
2006
).
Musical scale properties are automatically processed in the human auditory cortex
.
Brain Research
,
1117
,
162
174
.
Brennan
,
J.
,
Nir
,
Y.
,
Hasson
,
U.
,
Malach
,
R.
,
Heeger
,
D. J.
, &
Pylkkänen
,
L.
(
2012
).
Syntactic structure building in the anterior temporal lobe during natural story listening
.
Brain and Language
,
120
,
163
173
.
Bush
,
G.
,
Luu
,
P.
, &
Posner
,
M. I.
(
2000
).
Cognitive and emotional influences in anterior cingulate cortex
.
Trends in Cognitive Science
,
4
,
215
222
.
Carrus
,
E.
,
Pearce
,
M. T.
, &
Bhattacharya
,
J.
(
2013
).
Melodic pitch expectation interacts with neural responses to syntactic but not semantic violations
.
Cortex
,
49
,
2186
2200
.
Carter
,
C. S.
,
Braver
,
T. S.
,
Barch
,
D. M.
,
Botvinick
,
M. M.
,
Noll
,
D.
, &
Cohen
,
J. D.
(
1998
).
Anterior cingulate cortex, error detection, and the online monitoring of performance
.
Science
,
280
,
747
749
.
Chefer
,
S. I.
,
Talan
,
M. I.
, &
Engel
,
B. T.
(
1997
).
Central neural correlates of learned heart rate control during exercise: Central command demystified
.
Journal of Applied Physiology
,
83
,
1448
1453
.
Cheung
,
V. K.
,
Meyer
,
L.
,
Friederici
,
A. D.
, &
Koelsch
,
S.
(
2018
).
The right inferior frontal gyrus processes nested non-local dependencies in music
.
Scientific Reports
,
8
,
3822
.
Choi
,
J. W.
,
Lee
,
J. K.
,
Ko
,
D.
,
Lee
,
G. T.
,
Jung
,
K. Y.
, &
Kim
,
K. H.
(
2013
).
Fronto-temporal interactions in the theta-band during auditory deviant processing
.
Neuroscience Letters
,
26
,
120
125
.
Clark
,
A.
(
2013
).
Whatever next. Predictive brains, situated agents, and the future of cognitive science
.
Behavioral and Brain Sciences
,
36
,
181
204
.
Critchley
,
H. D.
,
Mathias
,
C. J.
, &
Dolan
,
R. J.
(
2001
).
Neural activity in the human brain relating to uncertainty and arousal during anticipation
.
Neuron
,
29
,
537
545
.
Critchley
,
H. D.
,
Mathias
,
C. J.
,
Josephs
,
O.
,
O'Doherty
,
J.
,
Zanini
,
S.
,
Dewar
,
B. K.
, et al
(
2003
).
Human cingulate cortex and autonomic control: Converging neuroimaging and clinical evidence
.
Brain
,
126
,
2139
2152
.
Cuddy
,
L. L.
, &
Badertscher
,
B.
(
1987
).
Recovery of the tonal hierarchy: Some comparisons across age and levels of musical experience
.
Perception & Psychophysics
,
41
,
609
620
.
Deouell
,
L. Y.
(
2007
).
The frontal generator of the mismatch negativity revisited
.
Journal of Psychophysiology
,
21
,
188
203
.
Dürschmid
,
S.
,
Edwards
,
E.
,
Reichert
,
C.
,
Dewar
,
C.
,
Hinrichs
,
H.
,
Heinze
,
H.-J.
, et al
(
2016
).
Hierarchy of prediction errors for auditory events in human temporal and frontal cortex
.
Proceedings of the National Academy of Sciences, U.S.A.
,
113
,
6755
6760
.
Egermann
,
H.
,
Pearce
,
M. T.
,
Wiggins
,
G. A.
, &
McAdams
,
S.
(
2013
).
Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music
.
Cognitive, Affective, & Behavioral Neuroscience
,
13
,
533
553
.
El Karoui
,
I.
,
King
,
J. R.
,
Sitt
,
J.
,
Meyniel
,
F.
,
Van Gaal
,
S.
,
Hasboun
,
D.
, et al
(
2015
).
Event-related potential, time–frequency, and functional connectivity facets of local and global auditory novelty processing: An intracranial study in humans
.
Cerebral Cortex
,
25
,
4203
4212
.
Fedorov
,
A.
,
Beichel
,
R.
,
Kalpathy-Cramer
,
J.
,
Finet
,
J.
,
Fillion-Robin
,
J. C.
,
Pujol
,
S.
, et al
(
2012
).
3D Slicer as an image computing platform for the quantitative imaging network
.
Magnetic Resonance Imaging
,
30
,
1323
1341
.
Friston
,
K.
(
2010
).
The free-energy principle: A unified brain theory?
Nature Review Neuroscience
,
11
,
127
138
.
Fuentemilla
,
L. L.
,
Marco-Pallarés
,
J.
,
Münte
,
T. F.
, &
Grau
,
C.
(
2008
).
Theta EEG oscillatory activity and auditory change detection
.
Brain Research
,
1220
,
93
101
.
Gentsch
,
A.
,
Weber
,
A.
,
Synofzik
,
M.
,
Vosgerau
,
G.
, &
Schütz-Bosbach
,
S.
(
2016
).
Towards a common framework of grounded action cognition: Relating motor control, perception and cognition
.
Cognition
,
146
,
81
89
.
Guo
,
S.
, &
Koelsch
,
S.
(
2016
).
Effects of veridical expectations on syntax processing in music: Event-related potential evidence
.
Scientific Reports
,
6
,
19064
.
Haenschel
,
C.
,
Baldeweg
,
T.
,
Croft
,
R. J.
,
Whittington
,
M.
, &
Gruzelier
,
J.
(
2000
).
Gamma and beta frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (EEG) data with in vitro models
.
Proceedings of the National Academy of Sciences, U.S.A.
,
97
,
7645
7650
.
Hansen
,
N. C.
, &
Pearce
,
M. T.
(
2014
).
Predictive uncertainty in auditory sequence processing
.
Frontiers in Psychology
,
5
,
1052
.
Hansen
,
N. C.
,
Vuust
,
P.
, &
Pearce
,
M.
(
2016
).
“If you have to ask, you'll never know”: Effects of specialised stylistic expertise on predictive processing of music
.
PLoS One
,
11
,
e0163584
.
Hantz
,
E. C.
,
Kreilick
,
K. G.
,
Kananen
,
W.
, &
Swartz
,
K. P.
(
1997
).
Neural responses to melodic and harmonic closure: An event-related potential study
.
Music Perception
,
15
,
69
98
.
Hartwigsen
,
G.
,
Neef
,
N. E.
,
Camilleri
,
J. A.
,
Margulies
,
D. S.
, &
Eickhoff
,
S. B.
(
2018
).
Functional segregation of the right inferior frontal gyrus: Evidence from coactivation-based parcellation
.
Cerebral Cortex
,
29
,
1532
1546
.
Heilbron
,
M.
, &
Chait
,
M.
(
2018
).
Great expectations: Is there evidence for predictive coding in auditory cortex?
Neuroscience
,
389
,
54
73
.
Herrojo-Ruiz
,
M.
,
Koelsch
,
S.
, &
Bhattacharya
,
J.
(
2009
).
Decrease in early right alpha band phase synchronization and late gamma band oscillations in processing syntax in music
.
Human Brain Mapping
,
30
,
1207
1225
.
Hsu
,
C. H.
,
Lin
,
S. K.
,
Hsu
,
Y. Y.
, &
Lee
,
C. Y.
(
2014
).
The neural generators of the mismatch responses to Mandarin lexical tones: An MEG study
.
Brain Research
,
1582
,
154
166
.
Huron
,
D.
(
2001
).
Tone and voice: A derivation of the rules of voice-leading from perceptual principles
.
Music Perception
,
19
,
1
64
.
Huron
,
D.
(
2006
).
Sweet anticipation: Music and the psychology of expectation
.
Cambridge, MA
:
MIT Press
.
Janata
,
P.
(
2015
).
Neural basis of music perception
. In
G.
Celesia
&
G.
Hickok
(Eds.),
Handbook of clinical neurology
(
Vol. 129
, pp.
187
205
).
Philadelphia
:
Elsevier
.
Janata
,
P.
,
Birk
,
J. L.
,
Van Horn
,
J. D.
,
Leman
,
M.
,
Tillmann
,
B.
, &
Bharucha
,
J. J.
(
2002
).
The cortical topography of tonal structures underlying Western music
.
Science
,
298
,
2167
2170
.
Janata
,
P.
, &
Petsche
,
H.
(
1993
).
Spectral analysis of the EEG as a tool for evaluating expectancy violations of musical contexts
.
Music Perception
,
10
,
281
304
.
Johnsrude
,
I. S.
,
Penhune
,
V. B.
, &
Zatorre
,
R. J.
(
2000
).
Functional specificity in the right human auditory cortex for perceiving pitch direction
.
Brain
,
123
,
155
163
.
Kaan
,
E.
, &
Swaab
,
T. Y.
(
2002
).
The brain circuitry of syntactic comprehension
.
Trends in Cognitive Sciences
,
6
,
350
356
.
Kim
,
S. G.
,
Kim
,
J. S.
, &
Chung
,
C. K.
(
2011
).
The effect of conditional probability of chord progression on brain response: An MEG study
.
PLoS One
,
6
,
e17337
.
Koelsch
,
S.
(
2012
).
Brain and music
.
Sussex, United Kingdom
:
Wiley-Blackwell
.
Koelsch
,
S.
,
Fritz
,
T.
, &
Schlaug
,
G.
(
2008
).
Amygdala activity can be modulated by unexpected chord functions during music listening
.
NeuroReport
,
19
,
1815
1819
.
Koelsch
,
S.
,
Fritz
,
T.
,
Schulze
,
K.
,
Alsop
,
D.
, &
Schlaug
,
G.
(
2005
).
Adults and children processing music: An fMRI study
.
Neuroimage
,
25
,
1068
1076
.
Koelsch
,
S.
,
Gunter
,
T.
,
Friederici
,
A. D.
, &
Schröger
,
E.
(
2000
).
Brain indices of music processing: “Nonmusicians” are musical
.
Journal of Cognitive Neuroscience
,
12
,
520
541
.
Koelsch
,
S.
, &
Jentschke
,
S.
(
2010
).
Differences in electric brain responses to melodies and chords
.
Journal of Cognitive Neuroscience
,
22
,
2251
2262
.
Koelsch
,
S.
,
Kilches
,
S.
,
Steinbeis
,
N.
, &
Schelinski
,
S.
(
2008
).
Effects of unexpected chords and of performer's expression on brain responses and electrodermal activity
.
PLoS One
,
3
,
e2631
.
Koelsch
,
S.
,
Schmidt
,
B.
, &
Kansok
,
J.
(
2002
).
Effects of musical expertise on the early right anterior negativity: An event-related brain potential study
.
Psychophysiology
,
39
,
657
663
.
Koelsch
,
S.
, &
Siebel
,
W. A.
(
2005
).
Towards a neural basis of music perception
.
Trends in Cognitive Sciences
,
9
,
578
584
.
Krumhansl
,
C. L.
(
1990
).
Cognitive foundations of musical pitch
.
New York
:
Oxford University Press
.
Krumhansl
,
C. L.
, &
Kessler
,
E. J.
(
1982
).
Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys
.
Psychonomic Review
,
89
,
334
368
.
Lappe
,
C.
,
Steinsträter
,
O.
, &
Pantev
,
C.
(
2013
).
A beamformer analysis of MEG data reveals frontal generators of the musically elicited mismatch negativity
.
PLoS One
,
8
,
e61296
.
Lehne
,
M.
,
Rohrmeier
,
M.
, &
Koelsch
,
S.
(
2014
).
Tension-related activity in the orbitofrontal cortex and amygdala: An fMRI study with music
.
Social Cognitive and Affective Neuroscience
,
9
,
1515
1523
.
Lewis
,
A. G.
,
Schoffelen
,
J. M.
,
Schriefers
,
H.
, &
Bastiaansen
,
M.
(
2016
).
A predictive coding perspective on beta oscillations during sentence-level language comprehension
.
Frontiers in Human Neuroscience
,
10
,
85
.
Maess
,
B.
,
Koelsch
,
S.
,
Gunter
,
T. C.
, &
Friederici
,
A. D.
(
2001
).
Musical syntax is processed in Broca's area: An MEG study
.
Nature Neuroscience
,
4
,
540
545
.
Margulis
,
E. H.
, &
Levine
,
W. H.
(
2006
).
Timbre priming effects and expectation in melody
.
Journal of New Music Research
,
35
,
175
182
.
Marmel
,
F.
,
Tillmann
,
B.
, &
Dowling
,
W. J.
(
2008
).
Tonal expectations influence pitch perception
.
Perception & Psychophysics
,
70
,
841
852
.
Mayberg
,
H. S.
(
1997
).
Limbic-cortical dysregulation: A proposed model of depression
.
Journal of Neuropsychiatry and Clinical Neurosciences
,
9
,
471
481
.
Meyer
,
L. B.
(
1956
).
Emotion and meaning in music
.
Chicago
:
Chicago University Press
.
Mikutta
,
C. A.
,
Dürschmid
,
S.
,
Bean
,
N.
,
Lehne
,
M.
,
Lubell
,
J.
,
Altorfer
,
A.
, et al
(
2015
).
Amygdala and orbitofrontal engagement in breach and resolution of expectancy: A case study
.
Psychomusicology: Music, Mind, and Brain
,
25
,
357
365
.
Miranda
,
R. A.
, &
Ullman
,
M. T.
(
2007
).
Double dissociation between rules and memory in music: An event-related potential study
.
Neuroimage
,
38
,
331
345
.
Mukamel
,
R.
, &
Fried
,
I.
(
2012
).
Human intracranial recordings and cognitive neuroscience
.
Annual Review of Psychology
,
63
,
511
537
.
Musso
,
M.
,
Weiller
,
C.
,
Horn
,
A.
,
Glauche
,
V.
,
Umarova
,
R.
,
Hennig
,
J.
, et al
(
2015
).
A single dual-stream framework for syntactic computations in music and language
.
Neuroimage
,
117
,
267
283
.
Omigie
,
D.
(
2015
).
Music and literature: Are there shared empathy and predictive mechanisms underlying their affective impact?
Frontiers in Psychology
,
6
,
1250
.
Omigie
,
D.
,
Dellacherie
,
D.
,
Hasboun
,
D.
,
Clément
,
S.
,
Baulac
,
M.
,
Adam
,
C.
, et al
(
2015
).
Intracranial markers of emotional valence processing and judgments in music
.
Cognitive Neuroscience
,
6
,
16
23
.
Omigie
,
D.
,
Dellacherie
,
D.
,
Hasboun
,
D.
,
George
,
N.
,
Clement
,
S.
,
Baulac
,
M.
, et al
(
2015
).
An intracranial EEG study of the neural dynamics of musical valence processing
.
Cerebral Cortex
,
25
,
4038
4047
.
Omigie
,
D.
,
Pearce
,
M. T.
, &
Stewart
,
L.
(
2012
).
Tracking of pitch probabilities in congenital amusia
.
Neuropsychologia
,
50
,
1483
1493
.
Omigie
,
D.
,
Pearce
,
M. T.
,
Williamson
,
V. J.
, &
Stewart
,
L.
(
2013
).
Electro-physiological correlates of melodic processing in congenital amusia
.
Neuropsychologia
,
51
,
1749
1762
.
Oostenveld
,
R.
,
Fries
,
P.
,
Maris
,
E.
, &
Schoffelen
,
J. M.
(
2011
).
FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data
.
Computational Intelligence and Neuroscience
,
2011
,
156869
.
Paller
,
K. A.
,
McCarthy
,
G.
, &
Wood
,
C. C.
(
1992
).
Event-related potentials elicited by deviant endings to melodies
.
Psychophysiology
,
29
,
202
206
.
Patterson
,
R. D.
,
Uppenkamp
,
S.
,
Johnsrude
,
I. S.
, &
Griffiths
,
T. D.
(
2002
).
The processing of temporal pitch and melody information in auditory cortex
.
Neuron
,
36
,
767
776
.
Pearce
,
M. T.
(
2005
).
The construction and evaluation of statistical models of melodic structure in music perception and composition
(Doctoral dissertation)
.
Department of Computing, City University
,
London, United Kingdom
.
Pearce
,
M. T.
(
2018
).
Statistical learning and probabilistic prediction in music cognition: Mechanisms of stylistic enculturation
.
Annals of the New York Academy of Sciences
,
1423
,
378
395
.
Pearce
,
M. T.
, &
Müllensiefen
,
D.
(
2017
).
Compression-based modelling of musical similarity perception
.
Journal of New Music Research
,
46
,
135
155
.
Pearce
,
M. T.
, &
Rohrmeier
,
M.
(
2018
).
Musical syntax II: Empirical perspectives
. In
R.
Bader
(Ed.),
Springer handbook of systematic musicology
(pp.
487
505
).
Berlin, Germany
:
Springer-Verlag
.
Pearce
,
M. T.
,
Ruiz
,
M. H.
,
Kapasi
,
S.
,
Wiggins
,
G. A.
, &
Bhattacharya
,
J.
(
2010
).
Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation
.
Neuroimage
,
50
,
302
313
.
Peretz
,
I.
,
Brattico
,
E.
,
Järvenpää
,
M.
, &
Tervaniemi
,
M.
(
2009
).
The amusic brain: In tune, out of key, and unaware
.
Brain
,
132
,
1277
1286
.
Rauss
,
K.
,
Schwartz
,
S.
, &
Pourtois
,
G.
(
2011
).
Top–down effects on early visual processing in humans: A predictive coding framework
.
Neuroscience & Biobehavioral Reviews
,
35
,
1237
1253
.
Rohrmeier
,
M. A.
, &
Koelsch
,
S.
(
2012
).
Predictive information processing in music cognition. A critical review
.
International Journal of Psychophysiology
,
83
,
164
175
.
Rohrmeier
,
M. A.
, &
Pearce
,
M. T.
(
2018
).
Musical syntax I: Theoretical perspectives
. In
R.
Bader
(Ed.),
Springer handbook of systematic musicology
(pp.
473
486
).
Berlin, Germany
:
Springer-Verlag
.
Sammler
,
D.
,
Koelsch
,
S.
,
Ball
,
T.
,
Brandt
,
A.
,
Grigutsch
,
M.
,
Huppertz
,
H. J.
, et al
(
2013
).
Co-localizing linguistic and musical syntax with intracranial EEG
.
Neuroimage
,
64
,
134
146
.
Sammler
,
D.
,
Koelsch
,
S.
, &
Friederici
,
A. D.
(
2011
).
Are left fronto-temporal brain areas a prerequisite for normal music-syntactic processing?
Cortex
,
47
,
659
673
.
Samson
,
S.
, &
Zatorre
,
R. J.
(
1988
).
Melodic and harmonic discrimination following unilateral cerebral excision
.
Brain and Cognition
,
7
,
348
360
.
Sander
,
D.
,
Grafman
,
J.
, &
Zalla
,
T.
(
2003
).
The human amygdala: An evolved system for relevance detection
.
Reviews in the Neurosciences
,
14
,
303
316
.
Singer
,
T.
,
Critchley
,
H. D.
, &
Preuschoff
,
K.
(
2009
).
A common role of insula in feelings, empathy and uncertainty
.
Trends in Cognitive Sciences
,
13
,
334
340
.
Steinbeis
,
N.
,
Koelsch
,
S.
, &
Sloboda
,
J. A.
(
2006
).
The role of harmonic expectancy violations in musical emotions: Evidence from subjective, physiological, and neural responses
.
Journal of Cognitive Neuroscience
,
18
,
1380
1393
.
Strange
,
B. A.
,
Duggins
,
A.
,
Penny
,
W.
,
Dolan
,
R. J.
, &
Friston
,
K. J.
(
2005
).
Information theory, novelty and hippocampal responses: Unpredicted or unpredictable?
Neural Networks
,
18
,
225
230
.
Summerfield
,
C.
, &
De Lange
,
F. P.
(
2014
).
Expectation in perceptual decision making: Neural and computational mechanisms
.
Nature Reviews Neuroscience
,
15
,
745
756
.
Tillmann
,
B.
,
Bharucha
,
J. J.
, &
Bigand
,
E.
(
2000
).
Implicit learning of tonality: A self-organizing approach
.
Psychological Review
,
107
,
885
913
.
Tillmann
,
B.
,
Koelsch
,
S.
,
Escoffier
,
N.
,
Bigand
,
E.
,
Lalitte
,
P.
,
Friederici
,
A. D.
, et al
(
2006
).
Cognitive priming in sung and instrumental music: Activation of inferior frontal cortex
.
Neuroimage
,
31
,
1771
1782
.
Todorovic
,
A.
,
Schoffelen
,
J. M.
,
van Ede
,
F.
,
Maris
,
E.
, &
de Lange
,
F. P.
(
2015
).
Temporal expectation and attention jointly modulate auditory oscillatory activity in the beta band
.
PLoS One
,
10
,
e0120288
.
Van der Weij
,
B.
,
Pearce
,
M. T.
, &
Honing
,
H.
(
2017
).
A probabilistic model of meter perception: Simulating enculturation
.
Frontiers in Psychology
,
8
,
824
.
Vuilleumier
,
P.
(
2005
).
How brains beware: Neural mechanisms of emotional attention
.
Trends in Cognitive Sciences
,
9
,
585
594
.
Wang
,
X. J.
(
2010
).
Neurophysiological and computational principles of cortical rhythms in cognition
.
Physiological Review
,
90
,
1195
1268
.
Ward
,
A. A.
(
1948
).
The cingular gyrus: Area 24
.
Journal of Neurophysiology
,
11
,
13
34
.
Warren
,
J. D.
, &
Griffiths
,
T. D.
(
2003
).
Distinct mechanisms for processing spatial sequences and pitch sequences in the human auditory brain
.
Journal of Neuroscience
,
23
,
5799
5804
.
Warrier
,
C. M.
, &
Zatorre
,
R. J.
(
2004
).
Right temporal cortex is critical for utilization of melodic contextual cues in a pitch constancy task
.
Brain
,
127
,
1616
1625
.
Zarcone
,
A.
,
Van Schijndel
,
M.
,
Vogels
,
J.
, &
Demberg
,
V.
(
2016
).
Salience and attention in surprisal-based accounts of language processing
.
Frontiers in Psychology
,
7
,
844
.
Zatorre
,
R. J.
, &
Samson
,
S.
(
1991
).
Role of the right temporal neocortex in retention of pitch in auditory short-term memory
.
Brain
,
114
,
2403
2417
.