Abstract

Grapheme–color synesthesia is a heritable trait where graphemes (“2”) elicit the concurrent perception of specific colors (red). Researchers have questioned whether synesthetic experiences are meaningful or simply arbitrary associations and whether these associations are perceptual or conceptual. To address these fundamental questions, ERPs were recorded as 12 synesthetes read statements such as “The Coca-Cola logo is white and 2,” in which the final grapheme induced a color that was either contextually congruous (red) or incongruous (“…white and 7,” for a synesthetes who experienced 7 as green). Grapheme congruity was found to modulate the amplitude of the N1, P2, N300, and N400 components in synesthetes, suggesting that synesthesia impacts perceptual as well as conceptual aspects of processing. To evaluate whether observed ERP effects required the experience of colored graphemes versus knowledge of grapheme–color pairings, we ran three separate groups of controls on a similar task. Controls trained to a synesthete's associations elicited N400 modulation, indicating that knowledge of grapheme–color mappings was sufficient to modulate this component. Controls trained to synesthetic associations and given explicit visualization instructions elicited both N300 and N400 modulations. Lastly, untrained controls who viewed physically colored graphemes (“2” printed in red) elicited N1 and N400 modulations. The N1 grapheme congruity effect began earlier in synesthetes than colored grapheme controls but had similar scalp topography. Data suggest that, in synesthetes, achromatic graphemes engage similar visual processing networks as colored graphemes in nonsynesthetes and are in keeping with models of synesthesia that posit early feed-forward connections between form and color processing areas in extrastriate cortex. The P2 modulation was unique to the synesthetes and may reflect neural activity that underlies the conscious experience of the synesthetic induction.

INTRODUCTION

Synesthesia is a neurological condition in which stimulation in one cognitive or perceptual stream results in additional experiences more typically associated with stimulation of a different domain (Hubbard, 2007). An extreme example can be found in S, the subject of the classic case study by Luria. Presented with a 50-Hz tone, S simultaneously heard the tone, experienced the taste of a strong beet soup, and saw a brown stripe against a dark background surrounded by a red rim (Luria, 1969). A more common variety of this condition is grapheme–color synesthesia, in which printed numbers and letters induce particular colors (for a review, see Ramachandran & Brang, 2008). For example, 5 might always appear to be indigo, whereas 7 might appear to be green. In grapheme–color synesthesia then, graphemes (such as the digit 5 or the letter “A”) are the inducers, and the experience of color (i.e., indigo) is the concurrent.

Synesthetic associations are idiosyncratic, differing from synesthete to synesthete; however, within an individual each synesthetic association is highly consistent over time (Simner & Logie, 2007; Baron-Cohen, Burt, Smith-Laittan, Harrison, & Bolton, 1996; Cytowic, 1989). Thus, although different synesthetes typically experience the number 7 as different colors, an individual synesthete will associate 7 with (say) the exact same shade of green throughout their life span. In fact, qualifying characteristics of synesthesia require the association to be (a) stable over time (a 7 experienced as green should always be green), (b) involuntary and automatic (experiences do not require effort), and (c) both memorable and affect-laden (a number printed in the “incorrect” color may make a synesthete uncomfortable). As even nonsynesthetes experience synesthesia-like associations to some degree (Ward, Moore, Thompson-Lake, Salih, & Beck, 2008), uncovering the cognitive and neural basis of synesthesia has potential implications for the mechanisms underlying the categorization of sensory experience and the integration of information from different neural processing streams.

One proposed explanation of synesthesia is that it is a heritable condition that results from an excess of neural connections (Ramachandran & Hubbard, 2001). According to the cross-activation model, a genetic factor in synesthetes decreases neural pruning between interconnected areas leading to an abnormal increase in connectivity between brain areas that underlie the processing of inducers and concurrent sensations (Ramachandran & Hubbard, 2003). In grapheme–color synesthesia, for example, incomplete neural pruning during development results in increased connectivity between color processing area V4 and the nearby visual word form area in the fusiform gyrus (Ramachandran & Hubbard, 2001, 2003). Consequently, when graphemes activate the visual word form area they also excite neurons in V4 giving rise to the concurrent sensation of color. Alternatively, the disinhibited feedback proposal provides that synesthetic concurrents arise because of disinhibited feedback from higher level visual areas in pathways common to synesthetes and nonsynesthetes alike (Grossenbacher & Lovelace, 2001). A hybrid, the reentrant processing model posits hyperconnectivity between form and color processing areas in the fusiform (as in the cross-activation model), but, as in the disinhibited feedback model also suggests that synesthetic colors require neural activity from higher level areas in the temporal lobe (e.g., anterior inferior temporal and posterior inferior temporal) to feedback to V4 (Smilek, Dixon, Cudahy, & Merikle, 2001).

Although grapheme–color synesthesia has received a good deal of attention of late (e.g., the special issue of Cortex 2006 on synesthesia), researchers continue to debate some fundamental issues regarding the cognitive properties of synesthetic associations and their underlying neural substrate (for a review, see Hubbard, 2007). Cognitively, researchers have questioned whether the connection between inducer and concurrent is meaningful and connects to other information in the semantic memory system or is simply an arbitrary, isolated association (Brang, Edwards, Ramachandran, & Coulson, 2008); whether synesthetic associations are perceptual or conceptual (Ramachandran & Hubbard, 2001); and whether synesthetic connections operate in only one direction (Cohen Kadosh, Cohen Kadosh, & Henik, 2007; Cohen Kadosh et al., 2005; Knoch, Gianotti, Mohr, & Brugger, 2005). Relatedly, researchers have tested whether synesthetic colors result from feed-forward activation (Brang et al., 2010; Hubbard, Arman, Ramachandran, & Boynton, 2005), feedback from higher level visual areas (Grossenbacher & Lovelace, 2001), or a combination of feed-forward and feedback connections (Smilek et al., 2001). Moreover, synesthesia researchers continue to dispute whether connectivity differences between synesthetes and neurotypicals are fundamentally anatomical (Weiss & Fink, 2008; Rouw & Scholte, 2007), reflecting incomplete pruning during development, or physiological because of the lack of inhibitory processes (Grossenbacher & Lovelace, 2001).

Focusing on the cognitive issue of whether concurrent color sensations in grapheme–color synesthesia are available to conceptual processes, Brang et al. (2008) recorded event-related brain potentials as eight grapheme–color synesthetes read sentence fragments that suggested a particular color as a sentence-final word. For example, “The Coca Cola logo is white and” suggests “red” as the final word (or at least it did so to 75/75 participants in our normative sentence completion study). Each sentence ended either with a “congruous” grapheme that induced the contextually appropriate color (e.g., “2,” which for synesthete LS induced red) or an “incongruous” grapheme that induced a contextually inappropriate color (where “7” induced green). Relative to the incongruous graphemes, congruous graphemes elicited larger amplitude N1, smaller P2, and reduced N400 components in synesthetes but not age-matched naive control subjects.

Because reduced N400 for congruous relative to incongruous sentence completions is conventionally understood as resulting from contextual integration processes (Van Berkum, Hagoort, & Brown, 1999; Holcomb, 1993), the N400 effect in synesthetes was argued to support the automaticity of the color-induction process in grapheme–color synesthesia and to suggest that the concurrent color sensation is treated by the brain as a meaningful stimulus. However, as is common practice in synesthesia research, the control group tested by Brang et al. (2008) was naive to the synesthetes' associations—and those participants were surely bewildered as to why they were reading sentences ending in isolated letters and numbers.

This raises the question of whether ERP effects reported by Brang et al. (2008) reflect the synesthetes' altered experience of graphemes as colored objects or whether they merely reflect their knowledge of the correspondences between particular graphemes and their assigned colors. Indeed, previous research has demonstrated that training nonsynesthetic individuals on synesthetic associations can produce markedly similar behavioral performance between synesthetes and controls on the synesthetic Stroop task (Meier & Rothen, 2009). Thus, the relative import of synesthetes' altered perceptual experience versus conceptual knowledge of the grapheme–color mappings is at issue in the N400 context effect, P2 effect, and N1 effect reported by Brang et al.

The Present Study

The goal of the present study was, first, to partially replicate contextual priming effects in grapheme–color synesthetes' ERPs described in Brang et al. (2008) and, second, to elucidate their functional significance via the testing of three different groups of neurotypical controls. As in Brang et al., we collected data from grapheme–color synesthetes as they read sentences (e.g., “The Coca Cola logo is white and”) that ended in color words, color patches, or achromatic graphemes (see Figure 1). Three groups of neurotypical controls were also tested on variations of the original paradigm: colored grapheme controls, trained controls, and explicit visualization controls. The colored grapheme controls were tested to identify ERP effects related to the perceptual experience of viewing colored graphemes. Consequently, each member of this group of naive nonsynesthetes viewed sentences ending in color words, color patches, or graphemes colored to mimic the perceptual experience of one of our synesthetes. The trained controls were included to test whether any of the contextual priming effects observed in synesthetes could be attributed to conceptual knowledge of the mappings between graphemes and their concurrent colors. These neurotypical controls were first trained on one grapheme–color synesthete's associations and then had ERPs recorded as they viewed the same sentences as their synesthete.

Figure 1. 

Example trial illustrating the display sequence. Depending on the condition, the final display ended with a congruous/incongruous color word, color patch, or grapheme.

Figure 1. 

Example trial illustrating the display sequence. Depending on the condition, the final display ended with a congruous/incongruous color word, color patch, or grapheme.

In addition to the conceptual knowledge of grapheme–color pairings, synesthetes' greater familiarity with these mappings may have affected the way they performed this task. That is, as soon as they realized the upcoming color ending, they may have begun to anticipate the associated congruous grapheme. To test whether ERP effects observed in the synesthetes resulted from anticipation of the form of the expected grapheme (viz., its shape), explicit visualization controls were trained on a synesthete's grapheme–color associations and given an explicit visualization task. These participants were told that as soon as they realized which grapheme should appear at the end of the sentence (i.e., 2 at the end of “The Coca Cola logo is white and”), they should explicitly visualize that grapheme on the monitor in front of them.

The present study was aimed to test for group differences in the cognitive and neural processes invoked by our task and to assess the time course of any such differences between synesthetes and neurotypical controls. In particular, we hoped to explore whether processing differences between the groups would arise during a timeframe associated with perceptual versus conceptual aspects of processing. ERPs were employed because they are a direct measure neural activity with a high degree of temporal resolution.

The inclusion of the three control groups was intended to determine which of the previously reported ERP context effects (N1, P2, and N400) result from synesthetes' knowledge of the mappings between graphemes and colors and which result from the subjective experience of seeing colored graphemes. The observation of ERP effects in the trained controls would show that the mere knowledge of the conceptual mappings of synesthetic associations is sufficient to elicit previously reported ERP effects in synesthetes. Of the ERP components listed above, the N400 is most likely to be affected by knowledge of the grapheme–color mappings given that it has been linked to the access of information in semantic memory (Kutas & Federmeier, 2000). Earlier, perceptual effects such as the N1 and P2 might be driven by the way that synesthetes approach the task. For example, if synesthetes anticipate the shape of the expected grapheme, we might expect to see these effects in the explicit visualization controls. Alternatively, if perceptual experience is required for the elicitation of these ERP effects, we would expect to see N1 and P2 effects only in the colored grapheme controls.

METHODS

Participants

Data were collected from four grapheme–color synesthetes and combined with those from the eight synesthetes reported in Brang et al. (2008). Grand-average ERP data from synesthetes thus includes a total of 12 synesthetic participants. Thirty-six nonsynesthetic controls also participated, none of whom participated in the previous study. All participants were fluent English speakers, had normal or corrected-to-normal vision, and none had any history of psychiatric or neurological disorder. The 12 synesthetes ranged in age from 18 to 30 years (mean age = 19.7 years, SD = 1.5 years) and included 10 women. The 12 colored grapheme controls ranged in age from 19 to 24 years (mean age = 20.7 years, SD = 1.6 years) and included 9 women. The 12 trained controls' age ranged from 19 to 28 years (mean age = 21.6 years, SD = 2.8 years) and included 4 women. The 12 explicit visualization controls' age ranged from 19 to 29 years (mean age = 21.2 years, SD = 2.8 years) and included 11 women.

Handedness was assessed via the Edinburgh Inventory (Oldfield, 1971), which yields a laterality quotient ranging from +1 (strongly right-handed) to −1. The majority of our participants were right-handed: synesthetes, 10 right, 2 left, average laterality quotient of +0.40; colored graphemes controls, 11 right, 1 left, average laterality quotient of +0.69; trained controls, 10 right, 2 left, average laterality quotient of +0.44; explicit visualization controls, 11 right, 1 left, average laterality quotient of +0.55. Synesthetes were recruited via fliers posted on the UCSD campus as well as similar ads on the Web. Synesthesia was confirmed by means of consistency matching as well as RT testing for color congruency, standardized by Eagleman, Kagan, Nelson, Sagaram, and Sarma (2007). All participants gave signed informed consent before the experiment and participated either for cash or in fulfillment of a course requirement. None of the control participants reported any known forms of synesthesia.

Materials and Procedure

Synesthetic participants, colored grapheme controls, and trained controls read 80 sentences that ended in text, 80 in color patches, and 80 in graphemes. Multiple stimulus lists were employed so that although no individual participant saw the same sentence twice, across participants each sentence ended an equal number of times with a color term, a color patch, or a grapheme. To control for surprise effects due to the introduction of nonlinguistic stimuli into sentence contexts, sentence-ending forms were presented in a blocked format. Blocks were pseudorandomized for each subject, such that two blocks of the same type were not presented sequentially. Block order was counterbalanced between subjects. Half of the sentences in each block ended as expected (congruous completions) and half with an unexpected color word, color patch, or a grapheme that induced an unexpected color (incongruous completions). Explicit visualization controls read 240 sentences that ended in graphemes (half ended congruously, half incongruously).

General Procedure

All participants were seated in a comfortable chair at a distance of 37 in. from the monitor in a sound-attenuated room. Sentences were visually presented one word at a time at the rate of two words per second. Unless noted otherwise, all words and graphemes were presented in black against a gray background. Each sentence-final word, color patch, or grapheme was presented for 200 msec followed by 1300 msec of blank screen (Figure 1). Participants were instructed to silently read experimental materials and to refrain from eye movements from the time the sentence-final stimulus appeared until the fixation cross appeared on the screen. Congruity of sentence-final words was established in a separate normative study.

Synesthetes

The stimulus material and the experimental design were identical to those used in Brang et al. (2008). Stimuli consisted of 240 sentences ending in one of three forms: text, color patch, or grapheme. In the grapheme condition, sentences ended with an achromatic grapheme known to induce either a congruous concurrent color (“2” for a synesthetes for whom 2 appears red) or an incongruous one (“7” for a synesthetes for whom 7 appears green). Sentence-final graphemes were adapted for each of our 12 synesthetes to match 11 possible color terms (red, orange, yellow, green, blue, purple, black, brown, white, gray, and pink) when possible. However, as some synesthetes did not experience grapheme–color associations for each of these color terms, the number of associations varied slightly between individuals (range = 9–11 grapheme–color associations, average = 10).

Colored Grapheme Controls

The experimental materials and procedure were identical to those applied to the synesthetes, with the exception of sentence-final stimuli in the grapheme condition. Whereas synesthetes viewed achromatic (black) graphemes, colored grapheme controls viewed graphemes written in a colored font matched to the subjective experience of a particular synesthete. Each member of this control group was yoked to a different synesthetic participant.

Trained Controls

Each participant in this group was yoked to a different synesthete and taught that synesthete's mappings between graphemes and colors via the use of a computerized training program. Each trained control thus learned a distinct set of 10 grapheme–color associations, in which each of the 10 graphemes corresponded to one of the 10 colors used in the study: red, orange, yellow, green, blue, purple, black, brown, white, and pink.

On a given trial, the training program displayed one achromatic grapheme in the center of the screen, surrounded by 10 colored blocks (location of each color was randomized on each trial). Subjects proceeded to click one of the colors in a guess-and-check fashion and received feedback on whether their grapheme–color pairing was correct. Participants were not allowed to advance to the EEG portion of the study until they correctly identified the associated color with each of the 10 graphemes twice in a row without any errors. Training took approximately 15 minutes.

During the EEG recording session, each participant in the trained control group viewed the exact same stimuli as had their yoked synesthete. Experimenters quizzed them verbally about the color of each grapheme immediately before beginning the recording session and during a break at the session's midpoint.

Explicit Visualization Controls

Differing from the synesthetes and other two control groups, explicit visualization controls viewed a grapheme at the end of all 240 sentences (120 with a grapheme associated with the contextually congruous color and 120 with an incongruous color); that is, there were no text or color blocks. As with trained controls, explicit visualization controls began with a brief training session in which they learned the grapheme–color associations of a particular synesthete. Using the computer program described above, training on pairings between letter/number and color proceeded via a process of guess and check followed by feedback. Participants were not allowed to advance to the EEG portion of the study until they correctly identified the associated color with each of 10 graphemes twice in a row without any errors.

For the ERP study, these participants were asked to anticipate the sentence-final item and to visualize it on the screen. That is, participants were told that as soon as they realized a 2 would appear at the end of “The Coca Cola logo is white and,” they were to imagine a 2 on the monitor in front of them. As in the trained control group, participants were quizzed verbally about the color of each grapheme immediately before beginning the recording session and during a break at the session's midpoint.

Electroencephalographic and Electrooculographic Recording

Participants' EEG was monitored with a commercial electrode cap with 29 scalp sites arranged according to the International 10–20 system (Ebner, Sciarnetta, Epstein, & Nuwer, 1999). Horizontal eye movements were measured with a bipolar derivation of electrodes placed at the outer canthi. Vertical eye movements and blinks were monitored with an electrode under the right eye. All electrodes were referenced on-line to the left mastoid, and all impedances were maintained below 5 kΩ. The EEG at a band pass of 0.01 and 100 Hz was amplified with SA Instruments 32-channel bioamplifiers (Stony Brook, NY), digitized at 250 Hz, and stored on a computer hard disk for later averaging.

Analysis of ERPs

ERPs were time locked to the onset of the sentence-final stimuli, and signals were averaged with a time window of −100 to 920 msec around an event. The 100-msec period preceding the onset of the target item served as the baseline. Epochs containing artifacts were rejected before averaging. Rejection values for each artifact were modified on a subject-by-subject basis, yet rejection values were applied uniformly throughout a given subject. Epochs were rejected for thresholds exceeding on average 292 μV for blinks, 337 μV for eye movements, 493 μV for amplifier drift, or 15 msec for blocking. Unless noted otherwise, analysis involved mean amplitude measurements of each participant's ERPs elicited between 150 and 200 msec post-sentence-final stimulus onset (intended to capture the N1 component), 150 and 250 msec post-sentence-final stimulus onset (intended to capture the P2 component), between 250 and 350 msec post-sentence-final stimulus onset (intended to capture the N300 component), and between 350 and 450 msec post-sentence-final stimulus onset (intended to capture the N400 component). Measurements were subjected to repeated measures ANOVA, with the within subject factors of congruity (congruous/incongruous) and scalp distribution factors as follows: N1 analysis included posterior electrodes (TP7, CP3, CPz, CP4, TP8, T5, P3, Pz, P4, TP7, O1, Oz, O2), and P2 and N400 analyses included all electrode sites (FP1, FPz, FP2, F7, F3, Fz, F4, F8, FT7, FC3, FCz, FC4, FT8, C3, Cz, C4, TP7, CP3, CPz, CP4, TP8, T5, P3, Pz, P4, TP7, O1, Oz, O2). Although the original degrees of freedom have been maintained for clarity, where appropriate, p values were subjected to Greenhouse–Geisser correction (Greenhouse & Geisser, 1959).

RESULTS

Text Condition

The final word of sentences in the text condition elicited ERPs typical of visually presented words: N1, P2, and N400, observed in synesthetes, trained controls, and colored grapheme controls. ERPs to sentences ending in color words can be seen in Figure 2. As predicted, the congruity manipulation reliably affected the amplitude of the N400 because congruous sentence completions elicited less negative N400 than did incongruous ones, Congruity, F(1, 33) = 30.46, p < .0001. Neither the amplitude nor the distribution of the congruity effect varied as a function of group, Group × Congruity, F(2, 33) = 0.11, p = .90; Group × Congruity × Electrode, F(56, 924) = 1.74, p = .11.

Figure 2. 

(Left) Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a color word. N400 amplitude between 350 and 450 msec was larger for incongruous compared with congruous words. (Right) Spline-interpolated topographic maps show the scalp distribution of the N400 congruity effect. Congruity effects were derived by subtracting the waveform elicited by congruous graphemes from incongruous graphemes. Results are shown separately for synesthetes (n = 12), controls trained to a particular synesthete's associations (n = 12), and controls who viewed physically colored graphemes (n = 12). Controls in the explicit visualization condition did not read sentences ending in color words.

Figure 2. 

(Left) Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a color word. N400 amplitude between 350 and 450 msec was larger for incongruous compared with congruous words. (Right) Spline-interpolated topographic maps show the scalp distribution of the N400 congruity effect. Congruity effects were derived by subtracting the waveform elicited by congruous graphemes from incongruous graphemes. Results are shown separately for synesthetes (n = 12), controls trained to a particular synesthete's associations (n = 12), and controls who viewed physically colored graphemes (n = 12). Controls in the explicit visualization condition did not read sentences ending in color words.

Follow-up analyses in each group revealed a similar-sized congruity effect in synesthetes (2.04 μV), F(1, 11) = 14.39, p < .01, colored graphemes (naive controls; 1.75 μV), F(1, 11) = 6.89, p < .05, and trained controls (2.14 μV), F(1, 11) = 10.86, p < .01; Congruity × Electrode, F(28, 308) = 3.23, p < .05 (see Figure 2).

Color Patch Condition

Rectangular color patches elicited ERPs that were similar to the color words in the text condition, including N1, P2, and N400 components. ERPs to congruous and incongruous color patches in each group are shown in Figure 3. As in the text condition, color patches elicited less negative N400 when they served as congruous than incongruous sentence completions, Congruity, F(1, 33) = 14.74, p < .001. Moreover, the color patch congruity effect did not differ as a function of group, Group × Congruity, F(2, 33) = 0.60, p = .55; Group × Congruity × Electrode, F(56, 924) = 0.82, p = .59. N400 congruity effects were reliable in synesthetes, F(1, 11) = 6.36, p < .05, and colored graphemes (naive controls), F(1, 11) = 6.82, p < .05, and trended toward significance in trained controls, F(1, 11) = 3.28, p = .10.

Figure 3. 

(Left) Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a color patch. N400 amplitude between 350 and 450 msec was larger for incongruous compared with congruous words. (Right) Spline-interpolated topographic maps show the scalp distribution of the N400 Congruity effect. Congruity effects were derived by subtracting the waveform elicited by congruous graphemes from incongruous graphemes. Results are shown separately for synesthetes (n = 12), controls trained to a particular synesthete's associations (n = 12), and controls who viewed physically colored graphemes (n = 12). Controls in the explicit visualization condition did not read sentences ending in color patches.

Figure 3. 

(Left) Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a color patch. N400 amplitude between 350 and 450 msec was larger for incongruous compared with congruous words. (Right) Spline-interpolated topographic maps show the scalp distribution of the N400 Congruity effect. Congruity effects were derived by subtracting the waveform elicited by congruous graphemes from incongruous graphemes. Results are shown separately for synesthetes (n = 12), controls trained to a particular synesthete's associations (n = 12), and controls who viewed physically colored graphemes (n = 12). Controls in the explicit visualization condition did not read sentences ending in color patches.

Grapheme Condition

Synesthetes' ERPs to sentence-final graphemes are shown in Figure 4. As in the text and color patch conditions, graphemes elicited N1 and P2 potentials. However, whereas text and color patches elicited a monophasic (single peaked) negativity in the latency window of the N400, sentence-final graphemes elicited a biphasic negativity with a peak at 320 msec—presumed to be the N300 component—and a second peak at 380 msec representing the classic N400 component.

Figure 4. 

Synesthete's grand-average ERPs at all electrode sites to congruous (black line) and incongruous (gray line) sentences ending in a grapheme.

Figure 4. 

Synesthete's grand-average ERPs at all electrode sites to congruous (black line) and incongruous (gray line) sentences ending in a grapheme.

N1

The N1 elicited by graphemes is shown for each group in Figure 5. Omnibus analysis revealed a main effect of Congruity, F(1, 44) = 4.45, p < .05, as well as a Group × Congruity interaction, F(3, 44) = 3.30, p < .05. Preplanned analyses of the congruity effect in each group revealed a reliable congruity effect only in synesthetes, as the N1 component elicited by congruous graphemes was 1.57 μV more negative than that elicited by incongruous graphemes (see Table 1). This pattern of N1 congruity effects was apparent in 10 of our 12 synesthetes. A similar pattern was observed in 9 of the 12 naive controls who viewed colored graphemes, resulting in a nonsignificant trend in the colored graphemes control group (see Table 1). No congruity effects were found either in trained controls or in trained controls given the explicit visualization task (see Table 1).

Figure 5. 

The N1 effect observed in synesthetes (n = 12) and who viewed physically colored graphemes (n = 12) but not seen in controls trained to a particular synesthete's associations (n = 12) or trained controls given an explicit visualization task (n = 12). Grand-average ERPs at posterior electrodes O1 and O2 to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. N1 amplitude between 170 and 190 msec was larger (more negative) for congruous compared with incongruous graphemes. Shaded region encapsulates measured window.

Figure 5. 

The N1 effect observed in synesthetes (n = 12) and who viewed physically colored graphemes (n = 12) but not seen in controls trained to a particular synesthete's associations (n = 12) or trained controls given an explicit visualization task (n = 12). Grand-average ERPs at posterior electrodes O1 and O2 to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. N1 amplitude between 170 and 190 msec was larger (more negative) for congruous compared with incongruous graphemes. Shaded region encapsulates measured window.

Table 1. 

Grapheme Condition


Synesthetes
Controls (Colored Graphemes)
Controls (Trained)
Controls (Explicit Visualization)
N1 
Amplitude difference −1.57 μV −0.81 μV −0.01 μV 0.38 μV 
Congruity F(1, 11) = 6.41, p < .05 F(1, 11) = 3.35, p = .09 F(1, 11) = 0.00, p = .98 F(1, 11) = 1.85, = .20 
Congruity × Electrode F(12, 132) = 1.01, p = .39 F(12, 132) = 2.03, p = .12 F(12, 132) = 1.30, p = .29 F(12, 132) = 0.93, p = .44 
 
P2 
Amplitude difference −1.41 μV 0.24 μV −0.06 μV 0.57 μV 
Congruity F(1, 11) = 5.59, p < .05 F(1, 11) = 0.68, p = .43 F(1, 11) = 0.02, p = .90 F(1, 11) = 6.78, p < .05 
Congruity × Electrode F(28, 308) = 1.02 p = .39 F(28, 308) = 2.10 p = .13 F(28, 308) = 0.69 p = .52 F(28, 308) = 1.13 p = .34 
 
N400 
Amplitude difference 1.95 μV 1.40 μV 1.24 μV 2.06 μV 
Congruity F(1, 11) = 6.88, p < .05 F(1, 11) = 7.02, p < .05 F(1, 11) = 4.48, p = .06 F(1, 11) = 32.74, p < .001 
Congruity × Electrode F(28, 308) = 5.02 p < .01 F(28, 308) = 0.82 p = .50 F(28, 308) = 3.74 p < .05 F(28, 308) = 4.03 p < .05 

Synesthetes
Controls (Colored Graphemes)
Controls (Trained)
Controls (Explicit Visualization)
N1 
Amplitude difference −1.57 μV −0.81 μV −0.01 μV 0.38 μV 
Congruity F(1, 11) = 6.41, p < .05 F(1, 11) = 3.35, p = .09 F(1, 11) = 0.00, p = .98 F(1, 11) = 1.85, = .20 
Congruity × Electrode F(12, 132) = 1.01, p = .39 F(12, 132) = 2.03, p = .12 F(12, 132) = 1.30, p = .29 F(12, 132) = 0.93, p = .44 
 
P2 
Amplitude difference −1.41 μV 0.24 μV −0.06 μV 0.57 μV 
Congruity F(1, 11) = 5.59, p < .05 F(1, 11) = 0.68, p = .43 F(1, 11) = 0.02, p = .90 F(1, 11) = 6.78, p < .05 
Congruity × Electrode F(28, 308) = 1.02 p = .39 F(28, 308) = 2.10 p = .13 F(28, 308) = 0.69 p = .52 F(28, 308) = 1.13 p = .34 
 
N400 
Amplitude difference 1.95 μV 1.40 μV 1.24 μV 2.06 μV 
Congruity F(1, 11) = 6.88, p < .05 F(1, 11) = 7.02, p < .05 F(1, 11) = 4.48, p = .06 F(1, 11) = 32.74, p < .001 
Congruity × Electrode F(28, 308) = 5.02 p < .01 F(28, 308) = 0.82 p = .50 F(28, 308) = 3.74 p < .05 F(28, 308) = 4.03 p < .05 

P2

The P2 elicited by congruous and incongruous graphemes is shown for each group in Figure 6. Figure 6 suggests that incongruous graphemes elicited more positive P2 than congruous ones in synesthetes but in none of the control groups. This was borne out by a reliable Group × Congruity interaction, F(3, 44) = 4.10, p < .05, as well as the results of preplanned analyses of each group. Grapheme congruity reliably affected P2 amplitude in the synesthetes but in none of the control groups (Table 1).

Figure 6. 

The P2 observed in synesthetes (n = 12) but not controls trained to a particular synesthete's associations (n = 12), trained controls given an explicit visualization task (n = 12), or controls who viewed physically colored graphemes (n = 12). Grand-average ERPs at anterior electrodes FC3 and FC4 to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. P2 amplitude between 150 and 250 msec was larger (more positive) for incongruous compared with congruous words. Shaded region encapsulates measured window.

Figure 6. 

The P2 observed in synesthetes (n = 12) but not controls trained to a particular synesthete's associations (n = 12), trained controls given an explicit visualization task (n = 12), or controls who viewed physically colored graphemes (n = 12). Grand-average ERPs at anterior electrodes FC3 and FC4 to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. P2 amplitude between 150 and 250 msec was larger (more positive) for incongruous compared with congruous words. Shaded region encapsulates measured window.

N300

Consistent with other researchers who report both N300 and N400 (McPherson & Holcomb, 1999; Holcomb & McPherson, 1994), incongruous stimuli elicited more negative ERPs than congruous, with the earlier peak being largest over anterior sites, and the later peak largest over posterior sites (see Figure 4). Figure 7 shows that the biphasic N300/N400 complex was evident in synesthetes and in the controls given an explicit visualization task. To confirm the presence of the N300/N400 in synesthetes and explicit visualization controls we conducted a post hoc analysis of the mean amplitude of ERPs comparing between-participants factor group (synesthetes, trained controls, explicit visualization controls, and colored number controls) and within-participant factors latency window (250–350 and 350–450 msec), congruity (congruous, incongruous), and electrode (12 medial electrodes). This analysis revealed a significant interaction of group, latency window, congruity, and electrode, F(33, 484) = 2.16, p < .05, suggesting the distribution patterns of the N300 and N400 did indeed vary as a function of group.

Figure 7. 

The N300 and N400 observed in synesthetes (n = 12) and trained controls given an explicit visualization task (n = 12) and only the N400 seen in controls trained to a particular synesthete's associations (n = 12) and controls who viewed physically colored graphemes (n = 12). Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. N300 and N400 amplitudes between 250–350 and 350–450 msec, respectively, were larger (more negative) for incongruous compared with congruous graphemes.

Figure 7. 

The N300 and N400 observed in synesthetes (n = 12) and trained controls given an explicit visualization task (n = 12) and only the N400 seen in controls trained to a particular synesthete's associations (n = 12) and controls who viewed physically colored graphemes (n = 12). Grand-average ERPs at posterior electrode CPz to congruous (black line) and incongruous (gray line) sentences ending in a grapheme. N300 and N400 amplitudes between 250–350 and 350–450 msec, respectively, were larger (more negative) for incongruous compared with congruous graphemes.

N400

Omnibus analysis of ERPs in the N400 window revealed a main effect of Congruity, F(1, 44) = 33.89, p < .001, as congruous graphemes elicited less negative N400 than did incongruous ones. However, this congruity effect did not vary as a function of group, Group × Congruity, F(3, 44) = 0.50, p = .69. Results of preplanned analyses of the N400 congruity effect in each group are shown in Table 1. Synesthetes, naive controls who viewed colored graphemes, and trained controls given the explicit visualization task all showed robust congruity effects; trained controls showed a smaller difference that did quite not reach significance (see Table 1).

DISCUSSION

To test whether synesthetic associations in grapheme–color synesthesia are treated by the brain as arbitrary associations versus potentially meaningful connections, scalp-recorded ERPs were recorded as 12 grapheme–color synesthetes read sentences that evoked the concept of a particular color. Sentences ended either with a color word, a color patch, or a grapheme that induced a color. For example, “The Coca Cola logo is white and” ended with either a congruous target (such as the word “red,” a red color patch, or with a grapheme such as 2 known to elicit the concurrent sensation of red) or an incongruous target (such as the word “green,” a green color patch, or with a grapheme such as 7 known to elicit the concurrent sensation of green).

The congruity manipulation in the text and color patch conditions resulted in the well-known contextual priming effect on the N400 component in the ERP. N400 contextual priming effects were also observed in the grapheme condition, indicating that synesthetic associations are available for integration with linguistically activated concepts. Moreover, the congruity manipulation affected the amplitude of the N1, P2, and N300 components elicited by graphemes.

Three neurotypical control groups were tested in a similar paradigm to elucidate the functional significance of these ERP effects. For example, to identify effects attributable to conceptual knowledge of the correspondences between graphemes and their concurrent colors, trained controls were taught these mappings and subsequently read sentences ending in achromatic graphemes. To identify effects attributable to task strategy, explicit visualization controls were taught grapheme–color mappings and asked to anticipate the appearance of the expected grapheme at the end of the sentence. Finally, to identify ERP effects attributable to the experience of colored graphemes, colored grapheme controls read sentences ending with colored graphemes.

Grapheme congruity modulated the N400 in synesthetes and all three control groups (Figure 3). Congruity modulation of the N300 was observed in synesthetes and explicit visualization controls (Figure 7). Congruity effects on the P2 were observed only in synesthetes (Figure 6). Finally, congruity modulation of the N1 was observed in synesthetes and colored grapheme controls (Figure 5).

N400

Contextual priming effects on the N400 component were thus the most consistent finding in the study, as the amplitude of this component was reduced by congruous sentence contexts for color words in synesthetes and in both the naïve and trained neurotypical control groups (Figure 2). The amplitude of the N400 elicited by color patches was also reduced in congruous relative to incongruous sentence contexts in synesthetes as well as for neurotypical controls (see Figure 3). Most importantly, similar N400 contextual priming effects were found for achromatic graphemes in synesthetes, for trained controls doing an explicit visualization task, and for colored graphemes viewed by naive control participants (see Figure 7).

These data suggest that the perceptual experience of colored graphemes is sufficient but not necessary for the elicitation of grapheme priming effects in this paradigm. Given that similar grapheme congruity effects on the N400 were observed in synesthetes and trained controls, effects in the synesthetes presumably stem from knowledge of the correspondences between particular letters or numbers and their corresponding colors. This knowledge could arise from the experience of graphemes as colored objects (as in the synesthetes and the naive controls who viewed colored graphemes) or from a short training period on the mappings.

Beyond synesthesia, these data speak to the flexibility of multimodal meaning-integration processes as conceptual expectations about color displayed a remarkably similar impact on ERPs 350–450 msec postonset, irrespective of whether the eliciting event was a color word, a color patch, a colored grapheme, or a grapheme recently associated with a color. The rapid integration of color information with participants' representation of sentence meaning may reflect the import of combining information from different modalities in face-to-face communication and in keeping with recent evidence that the brain simultaneously integrates information from spoken words and cospeech gestures (Ozyurek, Willems, Kita, & Hagoort, 2007; Wu & Coulson, 2007a, 2007b).

Another possibility is that perceived colors are rapidly integrated into sentences about color because the neural representation of color words recruits brain regions important for color perception, as in the cell assembly model proposed by Pulvermuller (2001). According to this model, word meanings are established via associative learning mechanisms linking information about word forms and the things they represent. Subsequently, word forms activate a distributed cortical network that includes associated sensorimotor activations. The model is supported by data showing that the comprehension of words related to olfactory experience (e.g., garlic) involves the activation of olfactory cortex (Gonzalez et al., 2006) and that understanding action verbs involves somatotopic activation of motor and premotor cortex (Pulvermuller, Shytrov, & Ilmoniemi, 2005).

The cell assembly model thus suggests that any neural activity correlated with the occurrence of “2” will be incorporated into its meaning and become available for integration with other activated conceptual information. Although the neural activity underlying the (for example) redness of a synesthete's 2 is irrelevant to its numeric referent, it nonetheless functions as part of its meaning. As a result, contextual congruity of the normally irrelevant concurrent had a facilitative effect on the meaning integration processes indexed by the N400 component.

N300

In view of the similarities between our four groups' ERPs to contextual priming of color words and color patches, the differences in their ERPs to graphemes subjected to the very same contextual priming paradigm are all the more remarkable. For example, the N300/N400 complex was evident in synesthetes and in the explicit visualization controls but not in trained controls without the visualization task or in untrained controls viewing colored graphemes (see Figure 7).

More typically reported in studies of picture priming (e.g., Hamm, Johnson, & Kirk, 2002; McPherson & Holcomb, 1999), this complex consists of an anteriorly distributed negative peak (N300) evident 250–350 msec poststimulus onset, thought to reflect object recognition, followed by a more broadly distributed negativity, the N400, thought to reflect more general, multimodal semantic processing (for a review, see Wu & Coulson, 2007a). The N300 is more negative for pictures of unidentifiable objects than for identifiable objects, and like the N400, its amplitude is reduced by the prior presentation of a picture of a related object (McPherson & Holcomb, 1999). N300 amplitude has also been shown to be reduced by contextual congruity, evoked by either linguistic (Federmeier & Kutas, 2002) or pictorial (West & Holcomb, 2002) contexts.

Although not typically elicited by graphemes, the observation of N300 effects in the present study is not particularly surprising given that graphemes are especially complex objects for synesthetes, eliciting color and even texture from form. Perhaps more puzzling is our failure to observe N300 effects in ERPs elicited by color patches (see Figure 3) or in ERPs elicited by colored graphemes (see Figure 7). These findings suggest that color per se is not sufficient to engage the object recognition processes that underlie the N300. Rather, N300 elicitation in controls was confined to participants asked to explicitly visualize expected graphemes, suggesting this ERP effect was related either to the anticipation of particular graphemes in sentence-final position or to the instruction to engage in mental imagery (or both). The latter suggestion is consistent with the report that synesthetes exhibit superior visual imagery abilities than do nonsynesthetic controls (Barnett & Newell, 2008).

P2

Congruity effects, revealed by an enhanced positivity on the anterior P2, were observed in synesthetes but neither of the control groups (see Figure 6). By all appearances, this effect is unique to synesthetes' processing of graphemes in sentence contexts. Moreover, although anterior P2 effects have occasionally been reported for words in sentence contexts, it is typically contextually congruous (expected) words that elicit a larger positivity than incongruous words (e.g., Federmeier, Mai, & Kutas, 2005). In contrast, the anterior P2 in the synesthetes was more positive for incongruous than congruous graphemes (see also Brang et al., 2008).

Citing Luck and Hillyard's (1994) suggestion that the functional significance of the anterior P2 is in multidimensional feature detection, Brang et al. (2008) interpreted the observed anterior P2 effect in grapheme–color synesthetes as indexing the binding of form information with synesthetically induced color presumed to underlie the synesthete's conscious percept of an achromatic 2 as a red 2. In keeping with this suggestion, the time course of the observed P2 effect is consistent with electrophysiological investigations of feature binding. Schoenfeld et al. (2003) report a positive ERP deflection beginning 230–240 msec postonset for attended dots in a moving dot display when the dot had a task-irrelevant color. Schoenfeld et al. localized the source of this effect to ventral V4 and suggested it was important for the feature binding process underlying the perceptual unity of attended objects.

Anterior positive components in this time window have been also been identified with inhibitory processes, such as the suppression of irrelevant visual features (Guillem, Bicu, & Debruille, 2001) and the inhibition of auditory distractors (Melara, Rao, & Tong, 2002). Bles, Alink, and Jansma (2007) have shown that the anterior P2 elicited in word recognition paradigms is sensitive to the need to suppress salient alternative lexical candidates, with larger P2 being associated with greater inhibitory demands. Synesthetes' generation of a larger P2 for contextually incongruous graphemes may thus reflect inhibitory processes needed to suppress or limit the engagement of synesthetically induced colors.

If the anterior P2 indexes inhibition, it appears that the inhibition of synesthetically induced colors imposes a greater demand than the inhibition of colors that are perceptually perceived. If the anterior P2 indexes the demands of binding of form and color information, it suggests that synesthetically induced colors are more negatively impacted than are colors that are perceptually perceived. In fact, these two explanations are not mutually exclusive and both point to an important role for contextual expectations in the processes underlying the induction of synesthetic colors.

Irrespective of the precise characterization of the processes underlying the P2, results of the present study indicate that they were not engaged by expectations about grapheme form, as suggested by their absence in the explicit visualization group. Nor were they engaged by expectations about grapheme color, as suggested by their absence in the colored grapheme group. Indeed, they were not engaged by color apprehended via normal perceptual processes, as suggested by their absence in the color patch condition in controls and synesthetes alike. The implication is that synesthetic induction of color from an achromatic grapheme involves slightly different cognitive and neural processes than does the perception of stimulus color information. In keeping with this suggestion, synesthetes report that the colors induced by graphemes often have an intangible quality that sets them apart from other colors, including the presence of unique hues (e.g., a “blueness” that exists only for 32), textures (a “metallic green”), and appearances, such as subtle tinges or halos emanating from the graphemes.

N1

N1 effects were not observed in any group in sentences ending with color words (see Figure 2) or with color patches (see Figure 3). In the grapheme condition, N1 effects were seen only in synesthetes who viewed achromatic graphemes and neurotypical controls who viewed colored graphemes (see Figure 5), further supporting the notion that synesthetically colored graphemes are processed similarly to physically colored graphemes in nonsynesthetic subjects. Importantly, trained controls given an explicit visualization task showed no modulation of the N1 component, suggesting that the expectation of grapheme form was not sufficient to modulate early components of the ERP. Of course, given that both the trained and the explicit visualization controls underwent a relatively short period of training on the color–grapheme correspondences, we cannot rule out the possibility that N1 effects might have been observed in controls who engaged in a more extensive training regime.

As the N1 is thought to index visual discrimination processes (Vogel & Luck, 2000), its elicitation in our contextual priming paradigm suggests that the conceptual expectations about color engendered by the sentence context facilitated visual processing of the appropriately colored grapheme. Findings in the present study thus replicate earlier work which suggests that synesthetically induced colors can influence the perception of digits (Smilek et al., 2001).

To test whether the N1 effect in synesthetes differed in size or scalp distribution from the N1 effect observed in controls who viewed physically colored graphemes, we conducted a post hoc analysis of the mean amplitude of ERPs measured 150–200 msec poststimulus onset, with between-participants factor Group (synesthetes and colored grapheme controls) and within-participants factors Congruity and Electrode site. This analysis revealed a main effect of Congruity, F(1, 22) = 9.76. p < .01, but critically no interaction between Group and Congruity, F(1, 22) = 0.98, p = .33, nor between Group, Congruity, and Electrode sites, F(12, 264) = 0.50, p = .69. Results of this analysis are thus consistent with the conclusion that the N1 grapheme congruity effect had similar neural sources in synesthetes who viewed achromatic graphemes and controls who viewed colored graphemes. However, null results of this sort must always be interpreted with caution (see Urbach & Kutas, 2003). Moreover, this particular analysis was post hoc and did not involve high-density recording best suited for addressing claims of this sort.

Nonetheless, if taken at face value, the similar N1 response in these two groups suggests achromatic graphemes engage similar visual processing networks, as do colored graphemes in nonsynesthetes. This finding is thus consistent with fMRI results that argue for synesthetes' activation of color processing networks in response to achromatic graphemes (Sperling, Prvulovic, Linden, Singer, & Stirn, 2006; Hubbard et al., 2005).

The strength of electrophysiological measures used in the present study, however, lies not in their spatial precision but rather in their temporal resolution. Although our analysis revealed no evidence for group differences in the topography of the N1 component, we did find evidence for group differences in the timing of the brain response. As the amplitude of the N1 itself was measured between 150 and 200 msec poststimulus onset, we assessed N1 onset by measuring the mean amplitude of ERPs elicited in the 50-msec interval before this window (i.e., 100–150 msec poststimulus onset), with between-participants factor group (synesthetes and colored grapheme controls) and within-participants factors congruity and electrode site. This analysis revealed an interaction between group and congruity, F(1, 22) = 6.05, p < .05, indicating that our experimental manipulation of color expectations had an earlier impact on the synesthetes' brain response than on that of controls who viewed colored graphemes. Indeed, generalized differences in synesthetes' visual processing mechanisms have been argued to occur as early as between 65 and 85 msec (Barnett et al., 2008), providing a basis for these findings of early synesthetic engagement.

Generalized Waveform Differences

In addition to notable differences between groups with respect to congruity manipulations, synesthetes also showed strikingly more positive ERP waves compared with each of the control groups in text, color, and grapheme conditions. Although the interpretation of standing waveform differences between groups must be considered with caution, this generalized positivity in synesthetes has now been shown in other ERP studies on synesthesia by different research groups, using different synesthetes, and critically with different tasks (Beeli, Esslen, & Jancke, 2008; Schiltz et al., 1999). Compared with trained controls and controls who viewed physically colored numbers, synesthetes showed significantly more positive waveforms starting between 200 and 250 msec, notably encompassing the window of the P2, which we have argued reflects processing unique to synesthetes, F(2, 33) = 3.47, p < .05 (Figure 8).

Figure 8. 

Overall positivity differences observed in synesthetes (n = 12; black line) compared with controls who viewed physically colored graphemes (n = 12; gray line) and controls trained to a particular synesthete's associations (n = 12; gray dotted line). Grand-average ERPs at electrodes T5, CPz, and T6 to sentences ending in a grapheme (top) or a color patch (bottom) regardless of congruity. In grapheme and color patch conditions, notice the more positive waveform elicited by synesthetes along the central electrode.

Figure 8. 

Overall positivity differences observed in synesthetes (n = 12; black line) compared with controls who viewed physically colored graphemes (n = 12; gray line) and controls trained to a particular synesthete's associations (n = 12; gray dotted line). Grand-average ERPs at electrodes T5, CPz, and T6 to sentences ending in a grapheme (top) or a color patch (bottom) regardless of congruity. In grapheme and color patch conditions, notice the more positive waveform elicited by synesthetes along the central electrode.

These standing differences in the waveforms may reflect structural differences in the brains of synesthetes and neurotypical controls, such as greater neural connectivity among synesthetes. Besides producing the enhanced sensory experiences associated with synesthesia, greater connectivity could also serve to enhance the amplitude of ERP components such as the P2 that reflect coordinated activity across multiple distributed neural generators. Analogous explanations have previously been suggested to explain the larger amplitude P3 response observed in left- relative to right-handers; namely, larger corpora callosa in left-handers affords greater interhemispheric transmission needed for the generation of the P3 (Polich & Hoffman, 1998). Diffusion tensor imaging has shown greater connectivity in synesthetes than nonsynesthetes, and moreover, the degree of connectivity has been shown to correlate with the “depth” of synesthesia (Rouw & Scholte, 2007). More recently, Weiss and Fink (2008) have shown that synesthetes have more gray matter (measured by volume) than nonsynesthetes, specifically in parietal cortices, which they suggest reflects the hyperbinding of the synesthetic concurrent. To date, few investigators have considered whether generalized differences between the neural organization of synesthetes and nonsynesthetes reflect any phenotypical traits besides the synesthetic experience (Brang, Williams, & Ramachandran, under review). The question of whether altered connectivity presumed to underlie synesthesia has implications for other aspects of cognitive or perceptual processing is thus an issue deserving of further study.

Implications for Models of Synesthesia

As for whether the associations in grapheme–color synesthesia reflect conceptual or perceptual connections, results of the present study suggest that both sorts of connections are important. N400 grapheme priming effects were observed in synesthetes as well as both groups of trained controls, suggesting they reflect conceptual knowledge of the relationship between particular graphemes and particular colors. N300 grapheme priming effects were observed both in synesthetes and in the explicit visualization controls, suggesting that these effects index processes of visual imagery. Besides the synesthetes, N1 grapheme priming effects were observed only in the control group who viewed colored graphemes. These data strongly suggest a perceptual connection between inducers and concurrents in synesthesia and indicate that the integration of color and form information in synesthesia recruits the neural substrate of form-color integration in neurotypicals individuals. Moreover, the fact that the onset of N1 effects was earlier in synesthetes who viewed achromatic graphemes than it was in controls who viewed colored graphemes is consistent with models of synesthesia that posits early feed-forward connections between form and color processing areas in extrastriate cortex.

Previous research has shown that top–down factors (which in our paradigm were manipulated via the meaning of sentential stimuli) can affect feature based attention, such that stimuli presented in the attended color receive enhanced processing, even when they are in an unattended location (Zhang & Luck, 2008; Giesbrecht, Woldorff, Song, & Mangun, 2003). As no group showed N1 modulation in response to congruous/incongruous color patches, it can be inferred that the observed contextual priming effects required the focused attention needed to recognize a grapheme. The posterior N1 component observed in the present study has previously been localized to sources in dorsal extrastriate cortex of the middle occipital gyrus (Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002). Moreover, attentional modulation of this ERP component has been argued to arise from activity in the ventral occipital region of V4v and adjacent regions of the fusiform gyrus, including V4/V8 (Di Russo, Martinez, & Hillyard, 2003). Activity in this time frame likely reflects a combination of both feed-forward and feedback connections (Martínez et al., 2001). Our results of early activation are consistent with the reentrant model of synesthesia, such that higher cognitive streams can affect lower level visual processing.

Results of the present study also have implications for the issue of whether connections in synesthesia are bidirectional. Subjectively, synesthetes report that their associations are unidirectional. That is, although 7s consistently induce the experience of green, green does not induce the appearance of a 7. But although putative connections between color and grapheme processing areas do not affect conscious experience, careful testing has indicated that the visual experience of colors can affect numerical cognition in synesthetes (Cohen Kadosh et al., 2005).

The report by Brang et al. (2008) that the expectation of a particular color term affected the processing of graphemes also appears consistent with bidirectional connections in synesthesia. However, our observation of similar effects on the N400 in nonsynesthetes suggests that the increased neural connectivity in synesthesia is not necessary for expectations about color to affect semantic processing of graphemes. Likewise, our finding of N300 effects in the nonsynesthetes who performed an explicit visualization task and the N1 effects in controls who viewed colored graphemes further limit the notion that observed effects in synesthetes reflect bidirectional connections unique to grapheme–color synesthesia. This is perhaps unsurprising considering the degree of interconnectivity in the primate visual system (Callaway, 1998; Felleman & Van Essen, 1991) and the importance of reentrant connections for attentional modulation of visual processing (Lamme & Roelfsema, 2000; Martínez et al., 1999). Nonetheless, observed amplitude modulation of the P2—present in the synesthetes but neither in naive controls tested by Brang et al. nor by any of the control groups tested in the present study—may reflect bidirectional connections between brain regions important for grapheme recognition and those that mediate knowledge of colors.

In sum, graphemes that elicit synesthetic colors were shown to modulate the ERP as early as 100 msec poststimulus onset, affecting the amplitude of the N1, P2, N300, and N400 components. Although N1, N300, and N400 effects were all observed in neurotypical control groups, the P2 modulation was unique to the synesthetes and may reflect neural activity that underlies that conscious experience of the synesthetic induction, including the subjective qualia of this phenomenon. These data argue against an account of synesthetic mappings reflecting isolated associations, pointing instead to connections at both the conceptual and the perceptual levels. Furthermore, this set of studies highlights the need for synesthesia researchers to use appropriate control groups in their experiments, as some previously reported effects on grapheme processing in synesthetes were attributable to knowledge of the inducer-concurrent mappings and task-specific behaviors.

Reprint requests should be sent to David Brang, University of California, San Diego, 9500 Gilman Dr. 0109, La Jolla, CA 92093, or via e-mail: dbrang@ucsd.edu.

REFERENCES

Barnett
,
K. J.
,
Foxe
,
K. J.
,
Molholm
,
S.
,
Kelly
,
S. P.
,
Shalgi
,
S.
,
Mitchell
,
K. J.
,
et al
(
2008
).
Differences in early sensory-perceptual processing in synesthesia: A visual evoked potential study.
Neuroimage
,
43
,
605
613
.
Barnett
,
K. J.
, &
Newell
,
F. N.
(
2008
).
Synaesthesia is associated with enhanced, self-rated visual imagery.
Consciousness and Cognition
,
17
,
1032
1039
.
Baron-Cohen
,
S.
,
Burt
,
L.
,
Smith-Laittan
,
F.
,
Harrison
,
J.
, &
Bolton
,
P.
(
1996
).
Synaesthesia: Prevalence and familiality.
Perception
,
25
,
1073
1079
.
Beeli
,
G.
,
Esslen
,
M.
, &
Jancke
,
L.
(
2008
).
Time course of neural activity correlated with colored-hearing synesthesia.
Cerebral Cortex
,
18
,
379
.
Bles
,
M.
,
Alink
,
A.
, &
Jansma
,
B. M.
(
2007
).
Neural aspects of cohort-size reduction during visual gating.
Brain Research
,
1150
,
143
154
.
Brang
,
D.
,
Edwards
,
L.
,
Ramachandran
,
V. S.
, &
Coulson
,
S.
(
2008
).
Is the sky 2? contextual priming in grapheme–color synaesthesia.
Psychological Science
,
19
,
421
428
.
Brang
,
D.
,
Hubbard
,
E. M.
,
Coulson
,
S.
,
Huang
,
M. X.
,
Song
,
T.
, &
Ramachandran
,
V. S.
(
2010
).
Magnetoencepalography reveals early activation of V4 in grapheme color synesthesia.
Neuroimage
,
53
,
268
274
.
Brang
,
D.
,
Williams
,
L. E.
, &
Ramachandran
,
V. S.
(
under review
).
Grapheme-color synesthetes show enhanced crossmodal processing between auditory and visual modalities
.
Callaway
,
E. M.
(
1998
).
Local circuits in primary visual cortex of the macaque monkey.
Annual Review of Neuroscience
,
21
,
47
74
.
Cohen Kadosh
,
R.
,
Cohen Kadosh
,
K.
, &
Henik
,
A.
(
2007
).
The neuronal correlate of bi-directional synaesthesia: A combined ERP and fMRI study.
Journal of Cognitive Neuroscience
,
19
,
2050
2059
.
Cohen Kadosh
,
R.
,
Sagiv
,
N.
,
Linden
,
D. E. J.
,
Robertson
,
L. C.
,
Elinger
,
G.
, &
Henik
,
A.
(
2005
).
When blue is larger than red: Colors influence numerical cognition in synesthesia.
Journal of Cognitive Neuroscience
,
17
,
1766
1773
.
Cytowic
,
R. E.
(
1989
).
Synesthesia and mapping of subjective sensory dimensions.
Neurology
,
39
,
849
850
.
Di Russo
,
F.
,
Martinez
,
A.
, &
Hillyard
,
S. A.
(
2003
).
Source analysis of event-related cortical activity during visuo-spatial attention.
Cerebral Cortex
,
13
,
486
499
.
Di Russo
,
F.
,
Martínez
,
A.
,
Sereno
,
M. I.
,
Pitzalis
,
S.
, &
Hillyard
,
S. A.
(
2002
).
Cortical sources of the early components of the visual evoked potential.
Human Brain Mapping
,
15
,
95
111
.
Eagleman
,
D. M.
,
Kagan
,
A. D.
,
Nelson
,
S. S.
,
Sagaram
,
D.
, &
Sarma
,
A. K.
(
2007
).
A standardized test battery for the study of synesthesia.
Journal of Neuroscience Methods
,
159
,
139
145
.
Ebner
,
A.
,
Sciarnetta
,
G.
,
Epstein
,
C.
, &
Nuwer
,
M.
(
1999
).
EEG instrumentation. recommendations for the practice of clinical neurophysiology: Guidelines of the international federation of clinical physiology.
Electroencephalography and Clinical Neurophysiology
,
52
,
7
10
.
Federmeier
,
K. D.
, &
Kutas
,
M.
(
2002
).
Picture the difference: Electrophysiological investigations of picture processing in the two cerebral hemispheres.
Neuropsychologia
,
40
,
730
747
.
Federmeier
,
K. D.
,
Mai
,
H.
, &
Kutas
,
M.
(
2005
).
Both sides get the point: Hemispheric sensitivities to sentential constraint.
Memory & Cognition
,
33
,
871
886
.
Felleman
,
D. J.
, &
Van Essen
,
D. C.
(
1991
).
Distributed hierarchical processing in the primate cerebral cortex.
Cerebral Cortex
,
1
,
1
47
.
Giesbrecht
,
B.
,
Woldorff
,
M. G.
,
Song
,
A. W.
, &
Mangun
,
G. R.
(
2003
).
Neural mechanisms of top–down control during spatial and feature attention.
Neuroimage
,
19
,
496
512
.
Gonzalez
,
J.
,
Barros-Loscertales
,
A.
,
Pulvermuller
,
F.
,
Meseguer
,
V.
,
Sanjuan
,
A.
,
Belloch
,
V.
,
et al
(
2006
).
Reading cinnamon activates olfactory brain regions.
Neuroimage
,
32
,
906
912
.
Greenhouse
,
S. W.
, &
Geisser
,
S.
(
1959
).
On methods in the analysis of profile data.
Psychometrika
,
24
,
95
112
.
Grossenbacher
,
P. G.
, &
Lovelace
,
C. T.
(
2001
).
Mechanisms of synesthesia: Cognitive and physiological constraints.
Trends in Cognitive Sciences
,
5
,
36
41
.
Guillem
,
F.
,
Bicu
,
M.
, &
Debruille
,
J. B.
(
2001
).
Dissociating memory processes involved in direct and indirect tests with ERPs to unfamiliar faces.
Cognitive Brain Research
,
11
,
113
125
.
Hamm
,
J. P.
,
Johnson
,
B. W.
, &
Kirk
,
I. J.
(
2002
).
Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts.
Clinical Neurophysiology
,
113
,
1339
1350
.
Holcomb
,
P. J.
(
1993
).
Semantic priming and stimulus degradation: Implications for the role of the N400 in language processing.
Psychophysiology
,
30
,
47
61
.
Holcomb
,
P.
, &
McPherson
,
W.
(
1994
).
Event-related brain potentials reflect semantic priming in an object decision task.
Brain and Cognition (Print)
,
24
,
259
276
.
Hubbard
,
E. M.
(
2007
).
Neurophysiology of synesthesia.
Current Psychiatry Reports
,
9
,
193
199
.
Hubbard
,
E. M.
,
Arman
,
A. C.
,
Ramachandran
,
V. S.
, &
Boynton
,
G. M.
(
2005
).
Individual differences among grapheme–color synesthetes: Brain–behavior correlations.
Neuron
,
45
,
975
985
.
Knoch
,
D.
,
Gianotti
,
L. R. R.
,
Mohr
,
C.
, &
Brugger
,
P.
(
2005
).
Synesthesia: When colors count.
Cognitive Brain Research
,
25
,
372
374
.
Kutas
,
M.
, &
Federmeier
,
K. D.
(
2000
).
Electrophysiology reveals semantic memory use in language comprehension.
Trends in Cognitive Sciences
,
4
,
463
470
.
Lamme
,
V. A. F.
, &
Roelfsema
,
P. R.
(
2000
).
The distinct modes of vision offered by feedforward and recurrent processing.
Trends in Neurosciences
,
23
,
571
579
.
Luck
,
S. J.
, &
Hillyard
,
S. A.
(
1994
).
Electrophysiological correlates of feature analysis during visual search.
Psychophysiology
,
31
,
291
308
.
Luria
,
A. M.
(
1969
).
The mind of a mnemonist.
London
:
Jonathan Cape
.
Martínez
,
A.
,
Anllo-Vento
,
L.
,
Sereno
,
M.
,
Frank
,
L.
,
Buxton
,
R.
,
Dubowitz
,
D.
,
et al
(
1999
).
Involvement of striate and extrastriate visual cortical areas in spatial attention.
Nature Neuroscience
,
2
,
364
369
.
Martínez
,
A.
,
DiRusso
,
F.
,
Anllo-Vento
,
L.
,
Sereno
,
M. I.
,
Buxton
,
R. B.
, &
Hillyard
,
S. A.
(
2001
).
Putting spatial attention on the map: Timing and localization of stimulus selection processes in striate and extrastriate visual areas.
Vision Research
,
41
,
1437
1457
.
McPherson
,
W. B.
, &
Holcomb
,
P. J.
(
1999
).
An electrophysiological investigation of semantic priming with pictures of real objects.
Psychophysiology
,
36
,
53
65
.
Meier
,
N.
, &
Rothen
,
N.
(
2009
).
Training grapheme–colour associations produces a synaesthetic Stroop effect, but not a conditioned synaesthetic response.
Neuropsychologia
,
47
,
1208
1211
.
Melara
,
R. D.
,
Rao
,
A.
, &
Tong
,
Y.
(
2002
).
The duality of selection: Excitatory and inhibitory processes in auditory selective attention.
Journal of Experimental Psychology: Human Perception and Performance
,
28
,
279
306
.
Oldfield
,
R. C.
(
1971
).
The assessment and analysis of handedness: The Edinburgh Inventory.
Neuropsychologia
,
9
,
97
113
.
Ozyurek
,
A.
,
Willems
,
R. M.
,
Kita
,
S.
, &
Hagoort
,
P.
(
2007
).
On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials.
Journal of Cognitive Neuroscience
,
19
,
605
.
Polich
,
J.
, &
Hoffman
,
L. D.
(
1998
).
P300 and handedness: On the possible contribution of corpus callosal size to ERPs.
Psychophysiology
,
35
,
497
507
.
Pulvermuller
,
F.
(
2001
).
Brain reflections of words and their meaning.
Trends in Cognitive Sciences
,
5
,
517
524
.
Pulvermuller
,
F.
,
Shytrov
,
Y.
, &
Ilmoniemi
,
R.
(
2005
).
Brain signatures of meaning access in action word recognition.
Journal of Cognitive Neuroscience
,
17
,
884
892
.
Ramachandran
,
V.
, &
Hubbard
,
E.
(
2001
).
Psychophysical investigations into the neural basis of synaesthesia.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
268
,
979
983
.
Ramachandran
,
V. S.
, &
Brang
,
D.
(
2008
).
Synesthesia.
Scholarpedia
,
3
,
3981
.
Ramachandran
,
V. S.
, &
Hubbard
,
E. M.
(
2003
).
The phenomenology of synaesthesia.
Journal of Consciousness Studies
,
10
,
49
57
.
Rouw
,
R.
, &
Scholte
,
H. S.
(
2007
).
Increased structural connectivity in grapheme–color synesthesia.
Nature Neuroscience
,
10
,
792
797
.
Schiltz
,
K.
,
Trocha
,
K.
,
Wieringa
,
B. M.
,
Emrich
,
H. M.
,
Johannes
,
S.
, &
Munte
,
T. F.
(
1999
).
Neurophysiological aspects of synesthetic experience.
Journal of Neuropsychiatry and Clinical Neurosciences
,
11
,
58
65
.
Schoenfeld
,
M. A.
,
Tempelmann
,
C.
,
Martinez
,
A.
,
Hopf
,
J. M.
,
Sattler
,
C.
,
Heinze
,
H. J.
,
et al
(
2003
).
Dynamics of feature binding during object-selective attention.
Proceedings of the National Academy of Sciences, U.S.A.
,
100
,
11806
11811
.
Simner
,
J.
, &
Logie
,
R. H.
(
2007
).
Synaesthetic consistency spans decades in a lexical-gustatory synaesthete.
Neurocase
,
13
,
358
365
.
Smilek
,
D.
,
Dixon
,
M. J.
,
Cudahy
,
C.
, &
Merikle
,
P. M.
(
2001
).
Synaesthetic photisms influence visual perception.
Journal of Cognitive Neuroscience
,
13
,
930
936
.
Sperling
,
J. M.
,
Prvulovic
,
D.
,
Linden
,
D. E. J.
,
Singer
,
W.
, &
Stirn
,
A.
(
2006
).
Physiology/imaging neuronal correlates of colour-graphemic synaesthesia: A fMRI study.
Cortex
,
42
,
295
303
.
Urbach
,
T. P.
, &
Kutas
,
M.
(
2003
).
The intractability of scaling scalp distributions to infer neuroelectric sources.
Psychophysiology
,
39
,
791
808
.
Van Berkum
,
J. J. A.
,
Hagoort
,
P.
, &
Brown
,
C. M.
(
1999
).
Semantic integration in sentences and discourse: Evidence from the N400.
Journal of Cognitive Neuroscience
,
11
,
657
671
.
Vogel
,
E. K.
, &
Luck
,
S. J.
(
2000
).
The visual N1 component as an index of a discrimination process.
Psychophysiology
,
37
,
190
203
.
Ward
,
J.
,
Moore
,
S.
,
Thompson-Lake
,
D.
,
Salih
,
S.
, &
Beck
,
B.
(
2008
).
The aesthetic appeal of auditory—Visual synaesthetic perceptions in people without synaesthesia.
Perception
,
13
,
1285
1297
.
Weiss
,
P. H.
, &
Fink
,
G. R.
(
2008
).
Grapheme–colour synaesthetes show increased grey matter volumes of parietal and fusiform cortex.
Brain
,
132
,
65
70
.
West
,
W. C.
, &
Holcomb
,
P. J.
(
2002
).
Event-related potentials during discourse-level semantic integration of complex pictures.
Cognitive Brain Research
,
13
,
363
375
.
Wu
,
Y. C.
, &
Coulson
,
S.
(
2007a
).
How iconic gestures enhance communication: An ERP study.
Brain and Language
,
101
,
234
245
.
Wu
,
Y. C.
, &
Coulson
,
S.
(
2007b
).
Iconic gestures prime related concepts: An ERP study.
Psychonomic Bulletin & Review
,
14
,
57
63
.
Zhang
,
W.
, &
Luck
,
S. J.
(
2008
).
Feature-based attention modulates feedforward visual processing.
Nature Neuroscience
,
12
,
24
25
.