Neural Oscillations and a Nascent Corticohippocampal Theory of Reference

Reference, or the ability to link linguistic representations to the world, makes language an immensely powerful tool for communication. Reference gives language intentionality, or “aboutness,” whether reference is made to the real world or to a hypothetical one. This essential computation presents a complex problem that dominates the philosophy of language to this day (e.g., Recanati, 1993; Martinich, 1985). Cognitive science shows that the crucial role of reference starts as early as language development itself, when children learn the meaning of words through understanding the referential intention of a speaker (e.g., Bloom, 2000). From this perspective, words are the spoken or written symbols that people use to denote referents in the physical world around them or in the conversations and stories that they engage in. Grasping a word's referential meaning is therefore a key challenge when establishing the intended meaning of a speaker. Moreover, in connected text or dialogue, the referential meaning of a word frequently often involves reference to a previously mentioned concept, such that the comprehender must establish a referential relationship between a given referring expression and its antecedent. Thus, the computational problem that reference presents can be, at least in some sense, boiled down to relating two representations that are separated from each other in time and/or by other representations.

Given these circumstances, how is reference computed in the mind and brain? Psycholinguistic theories of reference have a broad consensus on the involvement of memory in referential processing, which we will discuss shortly, but a neurobiological mechanism for reference is still largely unknown. In fact, as of 2016, neurobiological accounts of sentence-level language comprehension have not yet been articulated to the level of reference (e.g., Hagoort & Indefrey, 2014; Bornkessel-Schlesewsky & Schlesewsky, 2013; Friederici, 2012; Price, 2012), probably because these models primarily focus on accounting for syntactic and semantic processing phenomena.

To acquire data relevant to this theoretical gap, the current study performs oscillatory analysis of four encephalography (EEG) experiments on the neural signature of reference (Experiment 1: Nieuwland & Van Berkum, 2006; Experiment 2: Nieuwland, Otten, & Van Berkum, 2007; Experiment 3: Martin, Nieuwland, & Carreiras, 2012; Experiment 4: Nieuwland, 2014). Crucially, these experiments vary in language (Dutch, Spanish, or English), modality (written sentence or spoken story comprehension), and the linguistic expression that was manipulated (pronouns, noun phrases, ellipsis). However, their theoretical aim was similar: All the experiments examined how people understand expressions that refer back to a previously mentioned concept and, more specifically, compared expressions with a straightforward, coherent referential meaning to expressions that are referentially ambiguous or otherwise referentially insufficient. Through oscillatory analysis on these data sets, we show the increased coupling of functional neural systems associated with referential coherence compared with referential ambiguity or insufficiency. Moreover, Experiment 4 involved high-density EEG recording, which allowed us to localize the source of this increased neural coupling using a beamformer procedure (Gross et al., 2001).

A word that refers back to a previously mentioned concept (the antecedent) is an anaphor. The memory-based processing literature on text anaphora argues that anaphor comprehension involves the reactivation of the antecedent from a memory representation of the discourse (e.g., Gerrig & McKoon, 1998; Gernsbacher, 1989; Dell, McKoon, & Ratcliff, 1983; see also Sanford & Garrod, 1989, 2005), followed by the subsequent integration of the antecedent into the overall representation of the narrated event. Psycholinguistic experiments have demonstrated that this process proceeds very rapidly when the anaphor shares sufficient semantic and syntactic features with the antecedent (for a review, see Sturt, 2013; Garnham, 2001). Feature-based antecedent reactivation enables the recognition of the antecedent and, therefore, the establishment of a referential link between multiple instantiations of the same concept despite linguistic form differences, such that anaphora are not understood as mere repetitions of the antecedent (e.g., “The old man laughed. The man/Peter/he was happy”; see Almor, 1999; Garrod, Freudenthal, & Boyle, 1994).

In our view, the above-mentioned broad strokes processing framework on antecedent reactivation equates to or, minimally, involves recognition memory subroutines. Such a hypothesis is consistent with a broader approach to linguistic dependency resolution known as the cue-based retrieval framework (Martin, 2016; Martin & McElree, 2008, 2009, 2011; Lewis, Vasishth, & Van Dyke, 2006; McElree, 2006; Van Dyke & McElree, 2006). Cue-based retrieval builds on the computational architecture of human recognition memory and extends its mechanistic principles to language processing contexts. In the cue-based retrieval framework, memory representations like antecedents are organized and recovered by virtue of their content (content-addressable), and they are elicited directly, without a so-called “search” through memory, based on the contact of antecedent features with memory retrieval cues that are available on the anaphor (direct-access, cue-based retrieval). The extent to which unintended memory representations with content-overlap interfere with this process (cue-based retrieval interference) naturally emerges as the primary determinant of retrieval difficulty (e.g., Öztekin & McElree, 2007; Nairne, 2002). However, we note that there are substantive computational differences between recognition memory paradigms and anaphor resolution, and between anaphor resolution and “mere” word recognition (or pattern completion) per se. First, recognition memory paradigms often study the relationship between target and recognition probe in time, whether the target has been studied in the recent or distant past, as well as the context of retrieval—whether there is similar information present in memory that the recognition probe cue might call on. These task demands might not be identical during anaphor resolution, but aspects of them seem to persist, namely, that retrieval success of an anaphor might depend on how recently the antecedent occurred and how similar the antecedent is to other referents in the discourse. Crucially, the retrieval of an antecedent during anaphor resolution requires the system to compute a new relational representation of that antecedent in its new sentential and referential context. In this sense, anaphor resolution actually requires more relational processing than the standard probe recognition task might.

This cognitive-mechanistic account places anaphoric reference as a systemic computation in a larger processing model of language (e.g., Martin, 2016) and can lay the groundwork for developing an account of its neurobiology. This account, as do related memory-based accounts (Gerrig & McKoon, 1998; Sanford & Garrod, 1989), also offers a straightforward explanation for observed differences in the ease of anaphor comprehension, including differences between referentially coherent and ambiguous expressions as used in the current study. Relative to referentially coherent expressions, ambiguous expressions are associated with greater retrieval interference, which depends on the degree of content-overlap between anaphor and the intended antecedent relative to the content-overlap between the anaphor and other memory representations (Martin, Nieuwland, & Carreiras, 2014; Martin et al., 2012; Van Dyke & McElree, 2011). This explains why ambiguous expressions, whose features fail to immediately elicit a unique target from memory, result in slower processing times and delayed comprehension (e.g., Stewart, Holler, & Kidd, 2007; MacDonald & MacWhinney, 1990; Gernsbacher, 1989).

Memory-based theories of anaphor resolution, including the cue-based retrieval account, also harbor predictions, albeit only implicitly, regarding underlying neural processes. In recognition memory research, successful retrieval (i.e., recognition) is associated with increased activity in and connectivity between the medial-temporal lobe and posterior parietal cortex (Gonzalez et al., 2015; Shannon & Buckner, 2004; for a review, see Wagner, Shannon, Kahn, & Buckner, 2005). This may reflect the reactivation of, and bringing back into the focus of attention information that was previously encoded by the medial-temporal lobe system including the hippocampus (e.g., Gordon, Rissman, Kiani, & Wagner, 2014; Levy & Wagner, 2013; Staresina, Henson, Kriegeskorte, & Alink, 2012; Öztekin, McElree, Staresina, & Davachi, 2009; McClelland, McNaughton, & O'Reilly, 1995; see Discussion section, for a brief discussion of its subdivisions). Such patterns hold for reactivation of information that was encoded in the recent or distant past (e.g., Öztekin, Davachi, & McElree, 2010; Öztekin, Curtis, & McElree, 2009; see also Berryhill & Olson, 2008), blurring the difference between representations in long-term memory and working memory. This is relevant for current purposes because, in most language situations, anaphor resolution involves reactivation of recently encountered antecedents that may still reside still in working memory, whereas recognition memory paradigms sometimes involve reactivation of more distant information (e.g., items studied in a separate study session). In light of this literature, we predict that similar patterns of brain activity in hippocampus may occur if anaphora are understood through the reactivation of an antecedent through recognition memory subroutines.

Consistent with this first prediction, Nieuwland, Petersson, and Van Berkum (2007) observed BOLD activity increases in the hippocampus for pronouns that matched a unique antecedent in the sentence (e.g., “John told Mary that he…”) relative to pronoun that did not (e.g., “Lisa told Mary that he..”). Furthermore, damage to the hippocampus, which leads to hippocampal amnesic syndrome, is associated with impairments in pronoun production and comprehension (e.g., Kurczek, Brown-Schmidt, & Duff, 2013; MacKay, James, Taylor, & Marian, 2007; see also Almor, Kempler, MacDonald, Andersen, & Tyler, 1999, for related findings on pronoun comprehension in Alzheimer's disease) and with reduced use of definite reference to previously discussed items (Duff, Gupta, Hengst, Tranel, & Cohen, 2011).

A second prediction involves the brain's neural oscillatory activity or the synchronization of its neural populations as measured in electrical and magnetic activity (EEG, MEG, ECoG). Neural oscillations reflect the transient coupling or uncoupling of functional neural systems or cell assemblies (e.g., Buzsáki & Draguhn, 2004; Engel, Fries, & Singer, 2001), thereby offering a window into the functional network dynamics of human cognition. Oscillatory activity in particular the theta frequency (3–8 Hz) and gamma frequency band (>30 Hz) increases in power for successful recognition (e.g., Gonzalez et al., 2015; Burke et al., 2014; Jensen, Kaiser, & Lachaux, 2007; Jacobs, Hwang, Curran, & Kahana, 2006; Osipova et al., 2006; Mormann et al., 2005; Herrmann, Munk, & Engel, 2004; for a review, see Lisman & Jensen, 2013; Düzel, Penny, & Burgess, 2010; Klimesch, Freunberger, & Sauseng, 2010; Nyhus & Curran, 2010; Bastiaansen & Hagoort, 2003). Like successful memory recognition, successful anaphor resolution may lead to an increase in gamma band and/or theta activity. Consistent with this prediction, unpublished data from Van Berkum, Zwitserlood, Bastiaansen, Brown, and Hagoort (2004) showed a gamma-band increase around 40–50 Hz for referentially coherent pronouns compared with referentially problematic pronouns. The current study follows up on this work and tests the prediction that referentially coherent expressions elicit increased gamma-/theta-band oscillatory activity compared with referentially problematic expressions.

The observation that referentially coherent expressions elicit more gamma/theta oscillatory activity than referentially problematic expressions and not the other way around would deviate in a crucial way from the commonly observed pattern that problematic utterances lead to increased brain activity compared with unproblematic utterances. Research on the neurobiology of language comprehension often relies on violation paradigms (e.g., syntactically or semantically anomalous sentences), yielding conclusions about language comprehension based on the increase in activity when language processing is atypical or fails (e.g., Nieuwland, Martin, & Carreiras, 2012; Nieuwland, Petersson, et al., 2007). This approach has been generally very fruitful, but its conclusions about “normal” language comprehension are inherently limited. Therefore, finding increased oscillatory brain activity for successful reference, across different modalities and linguistic manipulations, offers an important initial step in describing the common processes involved in reference.

Such a finding on its own does not constitute unequivocal, direct support for the involvement of recognition memory networks during anaphora resolution. Changes in gamma/theta oscillatory activity are not unique to recognition memory and have been observed for different language comprehension processes (e.g., Lam, Schoffelen, Uddén, Hultén, & Hagoort, 2016; Bastiaansen & Hagoort, 2015; Lewis, Wang, & Bastiaansen, 2015; Rommers, Dijkstra, & Bastiaansen, 2013; Wang, Zhu, & Bastiaansen, 2012; Davidson & Indefrey, 2007; Roehm, Schlesewsky, Bornkessel, Frisch, & Haider, 2004). Lewis and Bastiaansen (2015) proposed that gamma-band activity during sentence comprehension indexes predictive processing and that theta-band activity indexes lexical-semantic retrieval. Although we agree that gamma-band and theta-band activity can, in principle, index predictive and lexical semantic processing, respectively, such a frequency-to-function mapping, while intriguing, risks oversimplification of dynamic brain activity, and we will return to this issue in our discussion.

To obtain additional support for our hypothesis about the involvement of recognition memory during anaphor resolution, we therefore investigated whether oscillatory activity increases are at least in part generated by brain regions strongly associated with recognition memory, like the medial-temporal lobe and posterior parietal cortex. Importantly, anaphor comprehension requires not only antecedent reactivation but also the integration and relation of this information into the unfolding sentence context (e.g., Sanford & Garrod, 1989) and therefore also relies on ongoing language processes (e.g., syntactic structure building, lexical-semantic processing). Therefore, we expected to see additional involvement of the traditional frontotemporal language network (e.g., Friederici & Singer, 2015; Fedorenko & Thompson-Schill, 2014; Hagoort & Indefrey, 2014; Hagoort, 2013). To state our hypothesis in neurobiological/neurocognitive terms, the antecedent is represented in the frontotemporal language network and the medial-temporal memory network functions to reorganize and phase reset activation in left inferior frontal gyrus to reactivate the antecedent as needed, along with the posterior parietal cortex, which governs attention-to-memory representations. This contrasts with recognition probes or word recognition alone where retrieval, but not relational processing, is needed.

The current study seeks to observe a signature of reference resolution in brain oscillations. We compared oscillatory activity induced by referentially coherent expressions with activity induced by referentially problematic expressions, using existing data from four previously reported ERP experiments. These experiments differed in the language of study, modality, and the referential expression of interest.

Experiment 1 (Nieuwland & Van Berkum, 2006) examined the comprehension of written Dutch sentences with pronouns that matched either one or two characters (e.g., “John told Mary/David that he needed to buy insurance”). Experiment 2 (Nieuwland, Otten, et al., 2007) examined the comprehension of spoken Dutch stories with noun phrase anaphora that matched either one or two characters (e.g., “the nephew” in a context with one or two nephews). Experiment 3 (Martin et al., 2012) examined the comprehension of written Spanish sentences with noun phrase ellipsis that matched or mismatched its antecedent in gender (e.g., “la camiseta…otra/otro,” with otra/otro being the feminine or masculine gender equivalent of “another”; see also Martin et al., 2014). Experiment 4 (Nieuwland, 2014) examined the comprehension of written English sentences with pronouns that matched or mismatched the only mentioned character in the sentence (e.g., “John said that he/she was a very happy person”).

Despite the differences between the experiments, referentially problematic expressions in each experiment elicit a slowly unfolding, sustained frontal negativity in the ERP waveform¹ compared with referentially coherent expressions (the NRef; Boudewyn et al., 2015; Nieuwland & Van Berkum, 2008a; Van Berkum, Zwitserlood, Hagoort, & Brown, 2003; Van Berkum, Brown, & Hagoort, 1999; for a review, see Nieuwland & Van Berkum, 2008b; Van Berkum, Koornneef, Otten, & Nieuwland, 2007), suggesting the involvement of qualitatively similar processes. The Nref effect is qualitatively and functionally distinct from the modulation of the well-known N400 ERP associated with semantic processing (see, e.g., Nieuwland & Van Berkum, 2008a). Here, we predicted that oscillatory activity in these experiments would show a gamma/theta-band increase for coherence compared with problematic expressions and that at least some of the observed differences would generate from the recognition memory network (posterior parietal cortex) and the frontotemporal language network.

Table 1 shows example materials from each experiment and a brief description of the relevant linguistic manipulation. Note that, in the four experiments reported here, participants also read or listened to a large number of filler sentences along with the experimental sentences described here. Full methodological details regarding participants, materials, and procedure are available in the previously published report for each experiment. Here we describe the methods that are relevant to the current analyses.

In Experiment 1 (Nieuwland & Van Berkum, 2006), 32 native speakers of Dutch read grammatically correct Dutch sentences that described an interaction between two individuals. We only included the ambiguous and nonambiguous sentences from this experiment, which did not contain semantically or syntactically unexpected or malformed expressions. Sixty nonambiguous sentences described two individuals of different gender and contained a referentially coherent pronoun that matched exactly one individual (e.g., “Anton forgave Linda because she…”), and the 60 ambiguous sentences described two individuals of the same gender and contained a referentially ambiguous pronoun that matched both individuals (e.g., “Mary forgave Linda because she…”). The pronouns were thus only a few words downstream from the antecedents. Participants read the sentences one word at a time from the center of a display. For the sentence context, word duration depended on word length, but from the word before the pronoun onwards, word duration was 350 msec (followed by a blank screen for 150 msec). Participants did not perform a secondary task.

In Experiment 2 (Nieuwland, Otten, et al., 2007), 31 native speakers of Dutch listened to 90 naturally spoken Dutch ministories of five sentences that described a scenario with one protagonist and two secondary characters. The two secondary characters were always denoted with a noun phrase followed by a relative clause of at least four words, and they could be denoted with different noun phrases (nephew/uncle) or with the same noun phrase (nephew). When the same noun phrase was used, the last word of the relative clause disambiguated the temporarily ambiguous expression (e.g., “the nephew who was really into politics/sports”). The third and the fifth sentence of each story mentioned one of the secondary characters using a referential expression that was either referentially coherent or that was temporarily ambiguous. The noun phrase anaphors were thus always at least one full sentence downstream from the antecedents. In the current analysis, we only used the conditions where the anaphor was coherent at both story positions (1-1 Referent Condition) or ambiguous at both positions (2-2 Referent condition), and we excluded stories in which the third sentence contained an ambiguity but in which the fifth sentence was referentially coherent (2-1 Reference condition) to match ambiguous and coherent conditions on story position. This selection resulted in a maximum of 60 trials per condition. Participants did not perform a secondary task.

In Experiment 3 (Martin et al., 2012), 22 native speakers of Spanish each read 60 Spanish sentences in which the gender of the critical word (otro or otra; the remaining determiner of the elided noun phrase) was grammatically correct or incorrect given that of the antecedent (e.g., “camiseta” in “Marta se compró la camiseta que estaba al lado de la falda y Miren cogió otra/otro para salir de fiesta,” see Table 1 for approximate translation into English). The sentences were from the correct/incorrect attractor–same condition from the original study, which meant that the attractor always had the same gender as the antecedent (la camiseta/la falda). The critical words and antecedents were always intervened by another noun in a relative clause (“falda”) and by another discourse referent (“Miren”). Critical words were always followed by at least three other words. Participants read the sentences one word at a time from the center of the display and answered intermittent comprehension questions about the presented sentences throughout the experiment.

In Experiment 4 (Nieuwland, 2014), 19 native speakers of English each read 180 grammatically correct English sentences that introduced a female or male character, followed by “verb-ed that” and subsequently a male or female pronoun and another four words. The pronouns were thus only two words downstream from the antecedents. Eighty sentences contained a referentially coherent pronoun that matched the character (Match condition), and 80 sentences contained a referentially ambiguous pronoun that did not match the character (Mismatch), for example in “Clifford mentioned that he/she was getting a divorce.” Participants read the sentences one word at a time at a pace of two words per second (300 msec word duration followed by blank screen for 200 msec) and were not asked to perform any secondary task.

In Experiment 1, continuous EEG data were collected from 30 standard channels (10/20 system) using an ActiCap (Brain Products, Munich, Germany), plus two additional EOG electrodes. The EEG was recorded with a 5-sec time constant and a 100-Hz low pass filter and sampled at 500 Hz. All electrode impedances were kept below 5 kΩ. In this and all following experiments, we used Brain Vision Analyzer software 2.0 (Brain Products) to preprocess the raw EEG data. The EEG data were re-referenced offline to the average of both mastoids and filtered with a 1-Hz high pass filter (48 dB/Oct). Data segments from −1 to 2.5 sec relative to the onset of the pronoun in each sentence were extracted and corrected for ocular artifacts and steady muscle artifacts using a method based on independent component analysis. After that, we applied automatic artifact rejection based on three rejection criteria simultaneously: an amplitude criterion of ±90 μV, a gradient criterion (i.e., the maximum admissible voltage step between two adjacent time points) of 50 μV, and a difference criterion (i.e., the maximum admissible absolute difference between two values within each EEG epoch) of 120 μV. Only participants with at least 40 trials in each of the conditions were included, leaving 28 participants for analysis. The referentially coherent and ambiguous conditions retained on average 54 and 55 trials per subject, respectively.

In Experiment 2, EEG data recording and preprocessing was identical to that in Experiment 1. Only participants with at least 40 trials in each of the conditions were included, leaving 26 participants for analysis. The referentially coherent and ambiguous conditions retained on average 53 and 52 trials per subject, respectively.

In Experiment 3, EEG data were collected from 29 standard channels (10/20 system) using an ActiCap (Brain Products), plus four additional EOG electrodes. The EEG was recorded continuously with a 10-sec time constant and a 100-Hz low pass filter and sampled at 250 Hz. The preprocessing procedure was identical to that of Experiments 1 and 2. Only participants with at least 20 trials in each of the conditions were included, leaving 18 participants for analysis. The two conditions had the same average number of trials (28) per subject entering the analysis.

In Experiment 4, EEG data were recorded at a sample rate of 512 Hz and with 24-bit AD conversion using the Biosemi ActiveTwo system (BioSemi BV, Amsterdam, The Netherlands). This system's hardware is completely DC coupled and applies digital low-pass filtering through its ADC's decimation filter (the hardware bandwidth limit), which has a fifth-order sinc response with a −3 dB point at one fifth of the sample rate (i.e., approximating a low-pass filter at 100 Hz). Data were recorded from 64 EEG, 4 EOG, and 2 mastoid electrodes using the standard 10/20 system. The preprocessing procedure was identical to that of Experiments 1–3 except for the automated artifact rejection procedure. A slightly more liberal artifact criterion was used to retain enough participants for analysis, using an amplitude criterion of ±100 μV, a gradient criterion (i.e., the maximum admissible voltage step between two adjacent time points) of 50 μV, and a difference criterion (i.e., the maximum admissible absolute difference between two values within each EEG epoch) of 150 μV. Only participants with at least 50 trials in each of the conditions were included, leaving 16 participants for analysis. The two conditions had the same average number of trials (70) per subject entering the analysis.

Time–frequency analysis was performed following the same procedure in Experiments 1–4, using the Fieldtrip software package (Oostenveld, Fries, Maris, & Schoffelen, 2011). First, a 50-Hz notch-filter was applied to all data to reduce spurious effects of line noise. To optimize the trade-off between time and frequency resolution, time–frequency analysis was then performed using the extracted epochs in two different, partially overlapping frequency ranges. In the low-frequency range (2–30 Hz), a 400-msec Hanning window was used to compute power changes in frequency steps of 1 Hz and time steps of 10 msec. In the high-frequency range (25–90 Hz²), time–frequency analysis was performed using a multitaper approach (Mitra & Pesaran, 1999), computing power changes with a 400-msec time-smoothing and a 5-Hz frequency-smoothing window, in 2.5-Hz frequency steps and 10-msec time steps. Subsequently, power changes per trial in the poststimulus interval were computed as a relative change from a baseline interval spanning from −0.5 to −0.3 sec relative to critical word onset, and average power changes per subject were computed separately for referentially coherent and referentially problematic trials.

For each experiment, statistical evaluation of the time–frequency responses was performed with a cluster-based random permutation test (Maris & Oostenveld, 2007). This statistical analysis was performed separately for the lower- and higher-frequency bands (2–30 and 30–90 Hz, respectively). Because of our specific interest in theta activity, we also performed a frequency-of-interest analysis for each experiment based on the average power in the 4–7 Hz range (Bastiaansen & Hagoort, 2003).

For Experiment 1, analysis was performed in the 200–1500 msec latency window after onset of the pronoun (thereby excluding data associated with sentence-final words). For every data point (electrode by time by frequency) of the two conditions, a simple dependent-samples t test was performed (giving uncorrected p values). All adjacent data points exceeding a preset significance level (5%) were grouped into clusters. For each cluster, the sum of the t statistics was used in the cluster-level test statistic. Subsequently, a null distribution that assumes no difference between conditions was created. This distribution was obtained by randomly assigning the conditions in the subjects for a total of 1000 times and calculating the largest cluster-level statistic for each randomization. Finally, the actually observed cluster-level test statistics were compared against the null distribution, and clusters falling in the highest or lowest 2.5th percentile were considered significant (using the correcttail option to correct p values for doing a two-sided test).

For Experiment 2, the analysis was identical to the analysis for Experiment 1 but performed in a shorter 200–1000 msec latency window after onset of each noun phrase to avoid data associated with the disambiguation (average distance between onset of the noun phrase and onset of the disambiguating word was 1083 msec).

For Experiment 3, the analysis was identical to the analysis for Experiments 1 and 2 but performed in a 200–1000 msec latency window after onset of the determiner (which was also the window of analysis in Martin et al., 2012).

In Experiment 4, analysis was performed in a 200–1000 msec latency window after onset of the pronoun and was therefore identical to the analysis for Experiments 2 and 3.

The data from Experiment 4 were collected from 64 EEG channels and because the larger number of electrodes allows a more reliable estimation of underlying sources, this experiment was best suited to attempt source localization of the observed effects. We performed this source reconstruction with a beamforming approach, namely, Dynamic Imaging of Coherent Sources (Gross et al., 2001), which uses an adaptive spatial filtering procedure to localize power in the entire brain.

Because referential coherence and ambiguity elicited significant differences in gamma activity around 40 Hz and around 60–80 Hz in different time intervals, we performed two separate source reconstructions. The first analysis contained data from 400 to 600 msec after critical word (CW) onset in both conditions, therefore focusing on activity predominantly elicited by the pronoun itself. The second analysis contained data from 500 to 1000 msec after CW onset in both conditions. Both analyses involved comparison with the 500–300 msec pre-CW baseline period.

After extracting the data segments, all data were re-referenced to the average of all electrodes (common average reference), as is required for source localization purposes. The first analysis centered on 40 Hz with frequency smoothing of ±5 Hz and used a Hanning taper. The second analysis centered on 70 Hz with frequency smoothing of ±10 Hz and used discrete prolate spheroidal sequences (Slepian sequences) as tapers.

Electrodes were aligned to a volume conduction model that was made based on a template brain using the boundary element method (Oostenveld, Praamstra, Stegeman, & van Oosterom, 2001). A common spatial filter was then computed at 40 Hz or at 70 Hz for the pre-CW period and the post-CW period of both conditions together. The spatial filter was subsequently projected to all trials. Power values were calculated on an equidistant, three-dimensional template grid with a 5-mm resolution. Trials were then averaged in the pre-CW period and in the post-CW period separately for each condition. Per condition, the power increase in the post-CW period relative to the pre-CW was computed in the following way: (post-CW − pre-CW)/pre-CW. After this, the grand averages were computed across subjects, and the difference between referential coherence and ambiguity was interpolated on the template brain for visualization.

For the statistical analysis of the source reconstruction, we computed one-sided dependent sample t statistics to compare the power values of the trial-averaged participant data of referential coherence and ambiguity at each of the 15,711 source points within the three-dimensional grid of the template brain. Clusters were constructed from source locations with significant t values of which neighboring locations also showed significant t values. Then, the sum of the t values in each cluster was calculated, and we selected the cluster with the largest sum of the t values. For localizing the spatial coordinates of the significant areas, the t values of the significant, clustered source points and zeros at all other points were interpolated to the template brain. We identified brain areas using a template atlas (Tzourio-Mazoyer et al., 2002).

In Experiment 1, we observed larger gamma-band power in the referentially coherent condition compared with the referentially ambiguous condition (p < .01), most prominent between 70 and 85 Hz and distributed from left frontal to left centroparietal electrodes (see Figure 1). No differences between the conditions were observed in the 2–30 Hz frequency band or in the more specific 4–7 Hz theta frequency band.

In Experiment 2, we also observed larger gamma-band power in the referentially coherent condition compared with the referentially ambiguous condition (p = .05), most prominent between around 80 Hz and distributed from frontal to centroparietal electrodes. No differences between the conditions were observed in the 2–30 Hz frequency band or in the more specific 4–7 Hz theta frequency band.

In Experiment 3, we observed larger gamma-band power in the referentially coherent condition compared with the referentially problematic condition, most prominent between 75 and 85 Hz and distributed from left frontal to left centroparietal electrodes. Although this difference was only marginally significant (p < .1) using an unrestricted two-tailed cluster permutation test, the difference was fully significant when taking into account the directionality of the effect expected based on Experiments 1 and 2 (i.e., a one-tailed test) or when restricting the analysis to a narrower gamma band (60–80 Hz) based on the results of Experiments 1 and 2. As in Experiment 1, no differences between the conditions were observed in the 2–30 Hz frequency band or in the more specific 4–7 Hz theta frequency band.

In Experiment 4, we observed larger gamma-band power in the referentially coherent condition compared with the ambiguous condition, most prominent between 60 and 80 Hz between approximately 500 and 1000 msec (p < .05), but also around 40 Hz in a second cluster between approximately 400 and 600 msec (p < .05). The bottom graphs in Figure 2 shows the visualization plots of the source localization results in the 40-Hz range (bottom left graphs) and in the 70–90 Hz range (bottom right graphs). The gamma increase for referential coherence compared with ambiguity around 40 Hz was localized to left posterior parietal cortex (encompassing the superior parietal lobe and precuneus). The gamma increase around 70 Hz was generated by left and right frontotemporal regions (encompassing inferior frontal lobe, inferior temporal lobe, and anterior temporal lobe), although the difference was only statistically robust in the left hemisphere.

On the basis of the observed gamma-band effect around 40 Hz in Experiment 4, we performed an additional frequency/time-of-interest analysis on the first three experiments. This analysis was a cluster permutation test, as described earlier, restricted to average power values between 35 and 45 Hz in the 400–600 msec time window. Referential coherence was associated with significantly higher gamma power than referential ambiguity in Experiment 1 (p < .05) and Experiment 2 (p = .05), but not in Experiment 3. Thus, although the unrestricted cluster permutation tests only revealed differences between coherence and ambiguity around 40 Hz in the Experiment 4, the more restricted follow-up tests revealed similar differences in Experiments 1 and 2 (Figure 1).

As in the other experiments, no differences between the conditions were observed in the 2–30 Hz frequency band or in the more specific 4–7 Hz theta frequency band.

We conducted oscillatory analyses of four EEG experiments on reference resolution during language comprehension. We investigated the increased coupling of functional neural systems associated with referentially coherent expressions compared with referentially problematic expressions and the neural sources supporting these changes. Despite varying in modality, language, and type of referential expression, all experiments showed larger gamma-band power for coherence compared with ambiguity. In Experiments 1, 2, and 4, a gamma-band increase was observed around 40 Hz around 400–600 msec after anaphor onset and also at a higher-frequency range (60–80 Hz) around 500–1000 msec. Beamformer analysis in high-density EEG Experiment 4 localized the increase around 40 Hz to left posterior parietal cortex and the increase in the 60–80 Hz range to left inferior frontotemporal cortex. We argue that the observed gamma-band power increases reflect successful reference resolution, encompassing the reinstatement of antecedents by the brain's recognition memory network and integration of antecedent information into the sentence representation by the frontotemporal language network.

In the sections below, we first discuss our results in comparison with those of previous oscillatory studies on sentence- and discourse-level language comprehension, and we subsequently integrate our findings and previous results from patient and neuroimaging studies into a corticohippocampal theory of the neurobiology of reference. The corticohippocampal theory of reference is a first attempt to bridge three seemingly disparate areas of research, namely the cognitive/psycholinguistic study of reference, the neurobiology of recognition memory, and the neurobiology of language. We discuss the predictions that follow from this theory and outline its major challenges for future research.

Our study is the first to investigate the neural oscillatory signature of successful reference (but see Van Berkum et al., 2004, for unpublished data) and generated a relatively robust pattern of results across four different experiments. Referentially coherent expressions led to increased gamma-band activity compared with referentially problematic expressions, and in three of the four studies this activity occurred both at 40 Hz around 500 msec after onset and at 60–80 Hz around 500–1000 msec after onset. In conjunction with the source localization results, we interpret these patterns to reflect the successful reactivation of antecedent information and integration into the unfolding sentence representation. For referentially problematic expressions, no antecedent is reactivated to a sufficient degree, and therefore, at least initially, no unique antecedent is successfully integrated into the unfolding representation. We thus assign two different functional interpretations for the 40 Hz and 60–80 Hz activity, although we acknowledge that the processes as reflected in these activity bands must work in close cooperation, and it is an open question whether activity in these frequency bands is truly separate.

We note that our study is not the first to find gamma-band activity in these frequency ranges modulated by a sentence-level linguistic manipulation. Several studies have reported increased gamma activity for semantically coherent and predictable sentences compared with semantically anomalous or unexpected sentences (e.g., Rommers et al., 2013; Wang et al., 2012), and current debate centers on whether such patterns reflect effects of semantic unification/integration or effects of preactivation/prediction (Lewis & Bastiaansen, 2015; Lewis et al., 2015; Wang et al., 2012). We cannot logically rule out an explanation of our own findings in terms of prediction, as the observed effects may reflect the processing consequences of a successful prediction of a referential expression (see Bögels, Barr, Garrod, & Kessler, 2015, for a potential demonstration to this effect in a mentalizing task). However, we think that this explanation does not sit well with the absence of N400 ERP modulation in the four experiments that are analyzed here. In addition, our source localization results do not include the temporal lobe regions that show rapid effects of semantic predictability (e.g., Lau & Nguyen, 2015; Lau, Gramfort, Hämäläinen, & Kuperberg, 2013), which typically show less activity to predictable words than to unpredictable words. For these reasons, an account of our results in terms of prediction is not particularly compelling.

Instead, we think that our 80-Hz gamma findings are compatible with an interpretation in terms of ongoing sentence-level semantic unification operations, taking place predominantly in left inferior frontal cortex (e.g., Hagoort & Indefrey 2014). Such unification operations may proceed more fully and smoothly with a coherent anaphor than with an ambiguous or insufficient one. This interpretation is consistent with other reports of gamma activity observed when participants are able to form meaningful interpretations of presented sentences. For example, Peña and Melloni (2012) observed increased gamma power in the 50–75 Hz range when participants listened to normally presented sentences in their own language, not to sentences played backword or sentences in a language they did not speak. Similarly, Fedorenko et al. (2016) observed gamma power increases, albeit in a much wider and higher 70–170 Hz range, for sentences compared with word lists or to grammatical nonword strings (see also Bastiaansen & Hagoort, 2015). In addition, we note the similarity of our 80-Hz findings in the left inferior frontal cortex to results obtained with fMRI (Nieuwland, Petersson, et al., 2007,Figure 1E). In that previous study, coherent pronouns elicited stronger BOLD responses than ambiguous pronouns, particularly in left and right inferior frontal brain regions but with some extension into the anterior temporal lobe. We thus take these previous results and the current results as convergent evidence for the role of left inferior frontal regions in successful sentence-level unification operations.

Whereas activity in the higher gamma range may reflect successful sentence-level unification operations, we associate the observed activity around 40 Hz with memory retrieval processes (antecedent reactivation). This interpretation is based solely on our localization of this activity to posterior parietal cortex, a region strongly associated with old/new effects and thought to govern memory-to-attention (e.g., Cabeza, Ciaramelli, Olson, & Moscovitch, 2008; Wagner et al., 2005). Recognition memory effects do not seem to be specifically associated with activity in this part of the gamma frequency band (e.g., Gonzalez et al., 2015). An important caveat to our conclusion about the observed 40-Hz gamma activity is that this effect was relatively weak in Experiments 1 and 2 and was not observed at all in Experiment 3. These differences could have something to do with the specific ambiguity manipulation in each experiment. In Experiment 4, where the 40-Hz effect was most pronounced, the ambiguous condition was a “mismatch condition” such that no matching antecedent was available for reference resolution. This may constitute a stronger comparison than the ambiguous conditions in Experiments 1 and 2, which contained two antecedents for a given anaphor. In Experiment 3, the ambiguous condition was arguably also a “mismatch condition,” but this agreement-mismatch manipulation may have been relatively weak as it involved a very long distance dependency and multiple discourse entities intervened between antecedent and anaphor (“distractors”), which could have impacted retrieval operations. In the ERP analysis, the agreement-mismatch did not elicit the typical P600 effect seen to grammatical violations, but an NRef effect, which is the reason we deem this ungrammatical expression to be referentially problematic rather than a strong syntactic anomaly. However, it was also the case that Experiment 3 had the lowest numbers of trials per condition and a low number of participants, so this experiment was perhaps not sufficiently powered to detect an effect at 40 Hz. The effect at the higher gamma range in this experiment was also weakest of the four experiments. A dedicated and well-powered experiment is needed to replicate the current pattern of 40-Hz results and perhaps to test the specific effect of distractors on oscillatory activity associated with anaphor resolution.

Against our predictions, we did not observe theta power increases for referentially coherent sentences in any of our experiments. This could mean that theta activity during language comprehension is only involved in retrieving word-elicited semantic information from long-term memory (e.g., Bastiaansen, van der Linden, ter Keurs, Dijkstra, & Hagoort, 2005), rather than being involved in reactivating antecedent information from a representation of the recent discourse. If this is the case, we may expect to see modulation of theta activity in a comparison of old/new referents that can be resolved based on semantic information (e.g., a repeated noun phrase anaphor compared with a noun phrase that introduces a new referent), but perhaps not when comparing pronouns with little semantic content or when comparing noun phrases with identical semantic content, as was done in the current studies.

Our results are compatible with a novel neurobiological account that we dub the corticohippocampal theory of reference. This coarse-grained theory combines the cognitive architecture and processing principles from cue-based retrieval and psycholinguistic theories of anaphora (e.g., Martin, 2016; McElree, 2006; Gerrig & McKoon, 1998; Gernsbacher, 1989; Sanford & Garrod, 1989; Dell et al., 1983), with extant neurobiological theories of recognition memory (e.g., Eichenbaum, Yonelinas, & Ranganath, 2007; Aggleton & Brown, 2006; Kahn, Davachi, & Wagner, 2004; Rugg & Yonelinas, 2003) and of language comprehension (e.g., Friederici & Singer, 2015; Fedorenko & Thompson-Schill, 2014; Hagoort & Indefrey, 2014; Bornkessel-Schlesewsky & Schlesewsky, 2013; Hagoort, 2013; Friederici, 2012; Price, 2012). Its core claim is that anaphor comprehension draws on the interaction between the recognition memory network (medial-temporal lobe, including the hippocampus, and posterior parietal cortex) and the canonical frontal-temporal network, with the former being primarily responsible for reinstatement of the antecedent information and the latter being primarily responsible for integrating antecedent information into the unfolding sentence representation. We do not wish to equate anaphor resolution and recognition in terms of the computations required (and we note the obvious task differences between anaphor resolution and recognition memory paradigms), but rather, we posit that anaphor resolution recruits mnemonic subroutines that distinguish between old and new information (which also support probe recognition).

The currently available neurobiological and neuropsychological evidence in support of this theory is (1) observed BOLD increases in left hippocampus for pronouns that uniquely match an antecedent compared with pronouns that match no antecedents (Nieuwland, Petersson, et al., 2007), (2) impairments in reference processing in patients with episodic memory dysfunction (hippocampal amnesia, Kurczek et al., 2013; Alzheimer's disease, Almor et al., 1999), and (3) the current findings of increased oscillatory activity in posterior parietal cortex and inferior frontal/temporal cortex for coherent reference.

The corticohippocampal theory of reference harbors at least the following predictions for future research. First, referential manipulations that compare old/new information (e.g., anaphoric expressions compared with expressions that introduce a new discourse referent) elicit stronger activity difference in the recognition network than manipulations of the ease with which an anaphor is understood (e.g., a distance or distractor manipulation). Second, activity in the recognition network is of shorter duration and may even take place before activity in the language network, reflecting the processing phases of antecedent reactivation and sentence-level integration (e.g., Garrod & Terras, 2000; Sanford & Garrod, 1989). Third, coherent reference should evoke stronger connectivity within the recognition network (e.g., between hippocampus and posterior parietal cortex) and between the recognition network and the language network. This connectivity could manifest itself in connectivity measures of BOLD fMRI and possibly in the coupling of oscillatory activity (cross-frequency coupling). Fourth, reference to things that are perceptually available (e.g., visually present objects) do not engage the recognition network but, instead, may require multimodal integration through cooperation between the language network and brain regions involved in perceptual processing (see Martin, 2016, for a sensory integration-based process model).

Beyond these predictions, there are also several important challenges to the corticohippocampal theory of reference. We acknowledge that recognition memory is supported by multiple subregions of the medial-temporal lobe, not just the hippocampus. Therefore, one challenge is to determine the respective contributions of the medial-temporal lobe and posterior parietal cortex and also of the various structures within these regions (e.g., Staresina, Cooper, & Henson, 2013; Staresina, Fell, Dunn, Axmacher, & Henson, 2013; Staresina, Fell, Do Lam, Axmacher, & Henson, 2012). This is, of course, just as much a challenge to neurobiological theories of recognition memory as it is to the corticohippocampal theory of reference. Within the medial-temporal lobe memory system, however, the hippocampus might play a particularly important role during anaphor resolution. Medial-temporal lobe subdivisions are associated with distinct roles for reactivating previously encoded items or the context in which they appeared (e.g., Backus, Bosch, Ekman, Grabovetsky, & Doeller, 2016; Staresina & Davachi, 2009; for a review, see Eichenbaum et al., 2007; Davachi, 2006). Parahippocampal cortex is thought to retrieve context information, perirhinal cortex is thought to retrieve item information, and the hippocampus is thought to bind together item and context information (e.g., the relative position or time of an item) and also binds different item-features (e.g., color and form) across time or relational binding (Davachi, 2006). Thus, an interesting possibility is that the hippocampus is initially involved in binding operations (e.g., Staresina & Davachi, 2009) that link the anaphor to antecedent information (for a possible role of hippocampal binding in online language processes, see Duff & Brown-Schmidt, 2012). If this is true, we predict to see increased hippocampal involvement when anaphor and antecedent are nonidentical and must be resolved based on shared semantic features (e.g., murderer–criminal), compared with repeated noun phrase anaphora (murderer–murderer).

Another challenge, in the frequency domain, is whether there is evidence for involvement of the well-established hippocampal theta rhythm and perhaps its coupling with gamma-band activity (e.g., Canolty & Knight, 2010; Colgin et al., 2009; Jensen & Colgin, 2007; Canolty et al., 2006). If antecedents are represented in the frontotemporal language network and parietal/medial temporal lobes activity indexes the reactivation of the antecedent, this makes the prediction that parietal/medial temporal lobes signals should drive left inferior frontal gyrus signals in cross-frequency coupling. In the current study, we did not observe any hippocampal theta activity changes as a function of referential coherence. However, we note that hippocampal theta may simply not be reliably detectable in 64-channel surface EEG data (e.g., Da Silva, 2013) and is better targeted with MEG (Backus et al., 2016; Tesche & Karhu, 2000) or intracranial recordings (Piai et al., 2016).

We report time–frequency analysis of four EEG experiments to reveal the increased coupling of functional neural systems associated with referentially coherent expressions compared with referentially problematic expressions. Despite varying in modality, language, and type of referential expression, all experiments showed larger gamma-band power for coherence compared with ambiguity or insufficient reference. We localized this increase in Experiment 4 to posterior parietal cortex around 400–600 msec after anaphor onset and to frontal-temporal cortex around 500–1000 msec. The current findings can be synthesized with previous results from patient and neuroimaging studies such that, together, the evidence suggests a corticohippocampal neurobiological theory of reference. This nascent corticohippocampal theory posits that anaphor comprehension draws on the interaction between the recognition memory network (medial-temporal lobe and posterior parietal cortex) and the canonical frontotemporal language network, with the former being primarily responsible for reinstatement of the antecedent information and the latter being primarily responsible for integrating antecedent information into the unfolding sentence representation. This neurobiological implementation would in turn offer support for a process model that incorporates the basic representational and mechanistic principles from the cue-based retrieval framework for language processing (e.g., Martin, 2016; Lewis et al., 2006; McElree, 2006; McElree et al., 2003). Taken together, these are the first steps toward a fully articulated, mechanistic neurobiological theory of language that includes reference.

We gratefully thank Lin Wang for making her data analysis scripts available to us, and we thank Marcel Bastiaansen and Lilla Magyari for helpful suggestions on data analysis. A. E. M. was supported by ESRC Future Research Leaders fellowship and grant ES/K009095/1.

Reprint requests should be sent to Mante S. Nieuwland, MPI for Psycholinguistics, Nijmegen, the Netherlands, or via e-mail: mante.nieuwland@mpi.nl.