Neural correlates of word recognition are commonly studied with (rapid) serial visual presentation (RSVP), a condition that eliminates three fundamental properties of natural reading: parafoveal preprocessing, saccade execution, and the fast changes in attentional processing load occurring from fixation to fixation. We combined eye-tracking and EEG to systematically investigate the impact of all three factors on brain-electric activity during reading. Participants read lists of words either actively with eye movements (eliciting fixation-related potentials) or maintained fixation while the text moved passively through foveal vision at a matched pace (RSVP-with-flankers paradigm, eliciting ERPs). The preview of the upcoming word was manipulated by changing the number of parafoveally visible letters. Processing load was varied by presenting words of varying lexical frequency. We found that all three factors have strong interactive effects on the brain's responses to words: Once a word was fixated, occipitotemporal N1 amplitude decreased monotonically with the amount of parafoveal information available during the preceding fixation; hence, the N1 component was markedly attenuated under reading conditions with preview. Importantly, this preview effect was substantially larger during active reading (with saccades) than during passive RSVP with flankers, suggesting that the execution of eye movements facilitates word recognition by increasing parafoveal preprocessing. Lastly, we found that the N1 component elicited by a word also reflects the lexical processing load imposed by the previously inspected word. Together, these results demonstrate that, under more natural conditions, words are recognized in a spatiotemporally distributed and interdependent manner across multiple eye fixations, a process that is mediated by active motor behavior.
Research using ERPs has greatly advanced our understanding of visual word recognition (e.g., Kutas & Federmeier, 2011; Kutas, Van Petten, & Kluender, 2006) but has also given little consideration to key properties of the visual and oculomotor system during natural reading. Typical procedures in neurolinguistic research require participants to maintain constant eye fixation while words are presented one by one in foveal vision (central 1–2° of the visual field) at a fixed pace. Although this (rapid) serial visual presentation (RSVP) procedure has the advantage that it reduces eye movement-related artifacts and temporal overlap between the brain responses elicited by successive words, it also differs in at least three ways from how words are recognized during normal, active reading.
First, during everyday reading, readers do not only see isolated words but also take up visually degraded information from upcoming words in the parafoveal region (eccentricity of 2°–5°) of their visual field (Rayner, 1975). Second, to bring words of interest into high-acuity foveal vision, readers actively prepare and execute between three and four saccadic eye movements per second (Schotter, Angele, & Rayner, 2012). Finally, because the resulting fixation pauses last on average only 200–250 msec, readers need to reallocate their limited attentional resources dynamically between foveal and parafoveal words, and this can cause the processing of neighboring words to interact (Henderson & Ferreira, 1990).
These fundamental properties of the active reading process raise the question whether and how the neural dynamics of word recognition differ from those established with passive stimulation techniques (Kliegl, Dambacher, Dimigen, Jacobs, & Sommer, 2012). In this study, we systematically investigated the effects of all three factors—parafoveal preprocessing, saccade execution, and attentional load—and demonstrate that, in combination, they have a significant impact on the word recognition process and its established neural correlates in the EEG. By comparing active reading with RSVP-based stimulation, we also aimed to narrow the existing gap between eye movement and ERP research on reading (Sereno & Rayner, 2003). In the following, we provide some background on each of the three factors investigated.
Parafoveal Vision Plays an Important Role in Reading
A central finding of the past 40 years of eye-tracking research is the preview benefit in fixation durations. If a word had been visible in the parafovea during preceding fixations, foveal fixations on this word are 20–50 msec shorter than when the same word had been parafoveally masked and is only uncovered when fixated. This effect is usually studied with the gaze-contingent boundary technique (Rayner, 1975), where a target word in a text (called word n in the following) is masked until the participant's eye crosses an invisible boundary located left of the word. During the saccade from the preboundary word (n − 1) toward word n, while visual sensitivity is temporarily reduced (Slattery, Angele, & Rayner, 2011; Matin, 1974), the mask is exchanged with the correct word.
To determine the level at which preview facilitates word recognition, researchers have manipulated the visual, orthographic, phonological, and semantic relation between preview mask and target. Although the exact results depend on the language and writing system used, there is broad consensus that readers extract partial orthographic and phonological information from the parafovea (e.g., Schotter et al., 2012). These trans-saccadic priming effects are largely independent of the similarity between preview and target not only in terms of meaning (e.g., song usually does not prime tune; Rayner, Balota, & Pollatsek, 1986; but see Hohenstein, Laubrock, & Kliegl, 2010; Yan, Richter, Shu, & Kliegl, 2009) but also in terms of low-level features (e.g., TABLE primes table; Rayner, McConkie, & Zola, 1980; McConkie & Zola, 1979), suggesting that readers retrieve more abstract information (letter codes and orthographic codes especially from word-initial letters) from the upcoming word, facilitating its subsequent recognition after the saccade.
Preview and the Brain's Response to Words
Importantly, the findings summarized above imply that, under natural circumstances, the neurocognitive system identifies words in a manner that is spatially and temporally distributed across separate eye fixations. Because traditional RSVP procedures do not include multiple words at once in the visual field, this key property of natural word recognition is not captured. Recently, two approaches have been proposed to study the electrophysiological effects of parafoveal preprocessing. The first is to record fixation-related potentials (FRPs) by coregistering eye movements and EEG during natural reading and aligning the EEG signal to fixation onsets (Metzner, von der Malsburg, Vasishth, & Roesler, 2015; Henderson, Luke, Schmidt, & Richards, 2013; Dimigen, Kliegl, & Sommer, 2012; Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl, 2011; Kretzschmar, Bornkessel-Schlesewsky, & Schlesewsky, 2009; Simola, Holmqvist, & Lindgren, 2009; Hutzler et al., 2007; Baccino & Manunta, 2005). An alternative proposal is to adapt RSVP stimulation by placing parafoveal flanker words at each side of the centrally presented word (RSVP-with-flankers paradigm; Li, Niefind, Wang, Sommer, & Dimigen, 2015; Barber, van der Meij, & Kutas, 2013; Barber, Donamayor, Kutas, & Munte, 2010; see also Pernet, Uusvuori, & Salmelin, 2007).
Recently, Dimigen et al. (2012) recorded FRPs in natural reading to study preview effects in lists of German nouns. In each list, one target word n was either shown in the parafovea during preceding fixations (valid preview) or masked by another word of the same length (invalid preview). Following the direct fixation of word n, FRPs differed after valid as compared with invalid previews: Between 200 and280 msec, following the peak of the N1 component, correctly previewed targets elicited a significantly smaller occipitotemporal response than those previously masked (“preview positivity”). In addition, the N400 component, peaking between 300 and 500 msec after fixation onset tended to be smaller after valid previews. Only two RSVP-with-flankers studies used designs that allowed investigating benefits of valid parafoveal information. Whereas Barber et al. (2013) observed no main effect of preview on ERPs, Li and colleagues (2015) replicated the attenuated N1 component for correctly previewed Chinese characters, followed by a later N400 effect (as in Dimigen et al., 2012).
In ERP research, the N1 has been linked to acquired visual expertise for specific domains, most notably faces (Bentin, Allison, Puce, Perez, & McCarthy, 1996). In addition, the left-lateralized portion of the N1 elicited by text strings (Bentin, Mouchetant-Rostaing, Giard, Echallier, & Pernier, 1999) has been taken to reflect sublexical (Kutas et al., 2006) or possibly also early lexical processing (Hauk & Pulvermüller, 2004; Sereno, Rayner, & Posner, 1998), and there is converging evidence from various methods (intracranial and scalp-recorded EEG, MEG, and fMRI) linking the left-hemispheric N1 to orthographic and sometimes also phonological processing (Maurer, Blau, Yoncheva, & McCandliss, 2010; Maurer, Brem, Bucher, & Brandeis, 2005; Tarkiainen, Helenius, Hansen, Cornelissen, & Salmelin, 1999; Nobre, Allison, & McCarthy, 1994). Masked repetition priming effects in a slightly later time window (N250) have also been interpreted as reflecting the mapping from orthography to phonology (e.g., Holcomb & Grainger, 2007; see Grainger & Holcomb, 2009, for a review).
Together, these results suggest an overlap between the factors modulating the N1 and the orthographic/phonological variables known to influence the preview benefit in behavior. Because the N1 in ERPs peaks within the duration of a typical reading fixation (Sereno & Rayner, 2003), it presents a plausible neural correlate of preview-related changes in oculomotor behavior. Specifically, N1 amplitude might be a direct index of the amount of sublexical information extracted parafoveally during preceding fixations. To test this hypothesis, we combined EEG, eye-tracking, and the boundary technique to manipulate the parafoveal information available during each fixation in a parametric fashion. Once the reader fixates word n, we expected to replicate the classic preview benefit in behavior, graded by the number of parafoveally previewed letters. Moreover, we expected an EEG effect corresponding to this oculomotor effect in two regards. First, it should occur early enough during a fixation to influence motor planning for the next saccade (estimated to take around 150 msec; Rayner, Slowiaczek, Clifton, & Bertera, 1983). Second, the N1 should decrease as a function of the amount of information acquired during previous fixations.
The Role of Saccadic Eye Movements
Although cognition during natural reading is tightly coupled to oculomotor actions, the impact of eye movements on the word recognition process has rarely been investigated and the only two electrophysiological studies on this topic have yielded discrepant results: Marton, Szirtes, and Breuer (1985) showed their participants single words that were either female (80%) or male names (20%), with the task to detect the less frequent male names (which therefore elicited a much larger P300 component). In one condition, the word was presented in the screen center, whereas in the other, the word was presented at 24° in the periphery, meaning that participants first had to execute a large saccade towards it. Interestingly, when measured relative to saccade offset, Marton et al. found that both RT and the latency of the P300 were significantly shorter than in the condition with passive foveal stimulation, suggesting that saccades facilitate word recognition.
Temereanca et al. (2012) recently reached a different conclusion by presenting words in two conditions with similar retinal inputs. In one condition, participants executed alternating saccades between two screen positions, whereas in the other, words moved passively into foveal vision. Temereanca et al. found that early visual brain responses to words (measured with MEG) were briefly attenuated after saccades, but not after external stimulus motion. They concluded that word recognition is transiently impaired after saccades because saccadic suppression (Ross, Morrone, Goldberg, & Burr, 2001; Matin, 1974) outlasts the eye movement and delays early stages of word identification. Although Temereanca and colleagues found differences between reading conditions only in the MEG, but not in RTs, these results imply that saccades are actually detrimental for word recognition.
Crucially, however, in both studies, effects of eye movements were investigated in the absence of useful parafoveal information. Because saccades are preceded by a shift of visuospatial attention toward the saccade target (Rolfs & Carrasco, 2012; Deubel & Schneider, 1996; Hoffmann & Subramaniam, 1995), active motor behavior is likely to alter the attentional dynamics of reading by enhancing the uptake of not-yet-fixated information. In the current experiment, we investigated the impact of saccades on word recognition by comparing a reading situation with and without eye movements, with the retinal inputs kept highly similar in both conditions. We expected that parafoveal information is used more efficiently during active, saccadic vision, leading to an even stronger modulation of occipitotemporal brain responses by preview.
Processing Demands Vary Rapidly during Reading
As a final factor, we addressed the rapid changes in cognitive processing load that occur from fixation to fixation during natural reading. With RSVP, words are usually presented at stimulus SOAs between 400 and 1000 msec (Dambacher et al., 2012), a situation that is fundamentally different in natural reading, where retinal inputs change every 200–250 msec, and the reader's attention needs to be quickly reallocated to different words within the perceptual span. Indeed, the foveal load hypothesis (Henderson & Ferreira, 1990) holds that the processing of neighboring words in a text is not independent, because a difficult stimulus in foveal vision requires more attention to be processed than an easy one, leaving fewer resources to process extrafoveal information. Conversely, if the fixated word is easy to process, the reader can shift attention to the parafoveal word sooner (Reichle, 2011) or distribute visuospatial attention more widely (Schad & Engbert, 2012). In support of this hypothesis, eye-tracking studies have found that the difficulty of a preboundary word n − 1 (usually manipulated by varying its frequency of occurrence) affects the size of the preview benefit measured on the postboundary word n (White, Rayner, & Liversedge, 2005; Schroyens, Vitu, Brysbaert, & d'Ydewalle, 1999; Kennison & Clifton, 1995; Henderson & Ferreira, 1990). Specifically, the benefit obtained from a correct preview tends to be larger if the previous word was easy to process.
In EEG, effects of foveal load have been investigated in the context of perceptual discrimination tasks, with several authors reporting that the visual evoked potentials (P1 and/or N1) elicited by parafoveally presented probe stimuli are smaller if the load induced by a concurrent discrimination task in the fovea is high (Fu et al., 2008; Handy, Soltani, & Mangun, 2001; for a review, see Lavie, 2005). Although these studies provide first evidence for an influence of foveal load on electrophysiological responses, this relationship has not yet been investigated in the context of reading. Specifically, we are not aware of any study that has investigated whether the ERP response to a currently inspected word also depends on the lexical processing load induced by word n − 1, that is, the preceding word in the text.
If words are presented at a long SOA without parafoveal information—as in most RSVP studies—attention never needs to be split or shifted between adjacent words and such interactive effects of load will be absent. However, under more natural, active reading conditions, we hypothesized that due to the reader's limited attentional resources—and the resulting changes in the efficiency of extrafoveal processing—preview-sensitive brain responses like the N1 should reflect not only properties of the currently inspected word but also those of the word previously encountered in the text, causing interactions between the neural processing of adjacent words.
In summary, reading differs from established laboratory procedures in at least three respects: parafoveal preprocessing, active saccade execution, and the load-dependent allocation of visuospatial attention. As outlined above, these basic properties of reading have rarely been considered in neurophysiological studies on visual word recognition and never in combination. Understanding their influence on scalp-recorded potentials is important not only to evaluate the ecological validity of RSVP-based paradigms but also to integrate the findings of EEG and eye movement research on reading (Sereno & Rayner, 2003).
To investigate their individual and combined impacts, we manipulated all three factors orthogonally in a within-subject design. Eye movements and EEG were simultaneously recorded while participants read short lists of German nouns with the task to detect animal names (see Figure 1).1 At any given moment, the fixated word n was fully visible; however, the parafoveal information available about this word during the preceding fixation on word n − 1 was manipulated in a parametric fashion: Readers were shown either no, one, two, three, or all letters of the upcoming word, with each remaining letter replaced by the character “x.” In active reading blocks, participants read the lists from left to right with eye movements. In passive reading blocks (RSVP-with-flankers paradigm), participants maintained central fixation while the list moved word by word through their field of vision at a speed closely matching the average duration of fixations in the active condition. Finally, foveal processing load imposed by the preceding word in the list (n − 1) was manipulated by varying the lexical frequency of the words within the list, which were either easy or difficult to process (i.e., of high vs. low lexical frequency). Combining these factors resulted in a 5 (amount of previewed letters) × 2 (active vs. passive reading) × 2 (low vs. high load) design.
Twenty-four young adults (ages 18–34 years, M = 24.8 years; 10 women) participated after providing written informed consent. All participants were right-handed (laterality quotient: M = 98.0, SD = 6.5; Oldfield, 1971) native German speakers, with normal uncorrected vision (decimal acuity: M = 1.29, SD = 0.33; Bach, 1996). They received course credits or € 16–24 for participation. Procedures were approved by the local ethics committee (application 2015-20).
Participants were instructed to silently read lists of five nouns each and, once finished with reading, to indicate whether the list contained the name of an animal or not. This semantic decision (e.g., Grainger, Kiyonaga, & Holcomb, 2006) requires that participants read for understanding. In the following, lists containing an animal (20%) are referred to as target lists and lists without an animal (80%) as regular lists. Only data from regular lists were analyzed.
Words were drawn from a pool of 1000 German nouns ranging in length between four and six letters (M = 5.1, SD = 0.8). For each noun, we retrieved word form and lemma frequency (i.e., inflectional word-stem frequency) and as additional covariates word length and number of syllables from a 197-million words sample of the extended DWDS corpus (Geyken, 2007). Additional word properties were retrieved from the DLEX database (dlexdb.de; Heister et al., 2011). Words were assigned to three frequency classes according to the natural logarithm of their word form and lemma frequencies. Four hundred words were categorized as low frequency (both word form and lemma log frequencies < 1 per million words; mean word form frequency = 0.3 ± 1.0, mean lemma frequency = −0.2 ± 0.8), 400 words were of high frequency (word form and lemma frequencies > 2; mean word form frequency = 3.5 ± 1.1, mean lemma frequency = 3.6 ± 1.1), and 200 words were of medium frequency (both word form and lemma frequencies > 1 and <2; mean word form frequency = 1.7 ± 0.8, mean lemma frequency = 1.5 ± 0.3). Words in the three frequency classes were matched for number of syllables and length so that four-, five-, and six-letter words occurred equally often in each class.
From these words, we created 200 regular word lists, with each list containing five nouns of varying frequencies. Whereas the first four positions of each list were filled with high- and low-frequency words (in random sequence), the last position was always occupied by a medium-frequency word, because it was not important to manipulate foveal load at this position (because there was no next word). Orthographic overlap between words within each list was minimized by precluding repetitions of word-initial or word-final letter bigrams. For use in the second half of the experiment, we randomly reshuffled the 1000 nouns into 200 new lists, that is, words were repeated once in the second half of the experiment. Possible influences of word repetition were controlled by including repetition as a covariate in the statistical analyses (see control models in the section on EEG Statistics below).
In addition, we created 100 target lists that were only needed for the animal task. Target lists were constructed from a further 400 German nouns of varying frequency (mean word form frequency = 1.2 ± 1.5, mean lemma frequency = 1.6 ± 1.3) as well as 100 animal names (mean word form frequency = 0.03 ± 1.2, mean lemma frequency = −0.4 ± 1.1; listed in the online supplement of Dimigen et al., 2012). Unlike regular lists, words for target lists were not repeated. Words in target lists were also four to six letters long (M = 5.1 ± 0.8). Animal names were embedded with equal probability at all five list positions. Apart from containing an animal, target lists were indistinguishable from regular lists shown during the experiment.
Three factors were manipulated within participants: parafoveal preview, reading paradigm, and foveal load. Parafoveal preview had five levels: During fixations on the preceding word in the list (always termed word n − 1, regardless of its absolute list position), either no letter, the first letter, the first two letters, the first three letters, or all letters of the upcoming word were presented, while the remaining letters were masked with the letter “x.” For example, while in parafoveal vision, the upcoming word “Magen” (stomach) was shown as “Xxxxx,” “Mxxxx,” “Maxxx,” or “Magxx” or “Magen,” respectively. Henceforth we will refer to these preview levels by the number of visible letters (i.e., no, 1, 2, 3, full). For each word in the list, the preview level was randomly assigned (e.g., the five words in a given trial might be presented at the levels: “full–2–no–full–1”; see also Figure 1), but across the experiment each preview manipulation occurred equiprobably at all list positions.
The second factor—reading paradigm—had two levels and was manipulated blockwise: In the active reading condition, participants read the lists at their own pace with eye movements. In the passive reading condition, participants maintained central fixation while the list moved word by word from right to left through foveal vision at a speed matched to the average first fixation duration (FFD) during active reading (as estimated from Dimigen et al., 2012). The two conditions are illustrated in Figure 1.
The third factor—foveal load—had two levels and varied randomly for the first four words of each list. As in previous behavioral studies (White et al., 2005; Schroyens et al., 1999; Kennison & Clifton, 1995; Henderson & Ferreira, 1990), load was operationalized as the lexical frequency of word n − 1 and manipulated for the first four words in each list. High- and low-frequent words were considered as inducing low and high cognitive load, respectively. Note that we only analyze words 2–5 in the list, because the first word does not have a predecessor word n − 1.
Across all trials of a given participant, the factors preview, reading mode, and word frequency were fully balanced. In addition, all three factors were balanced over list position, that is, each combination of factor levels was shown equally often at each list position (except for the fifth word, for which frequency was always medium).
Stimulation and Procedure
Recordings were conducted in an electrically shielded and dimly lit cabin. Stimuli were presented on a 22-in. monitor (Iiyama Vision Master Pro 514, resolution 1024 × 760 pixel, vertical refresh 160 Hz, Hoofddorp, The Netherlands) with Presentation software (version 14.9; Neurobehavioral Systems, Inc., Albany, CA). Before the experiment, participants performed a 10-min calibration task in which they produced saccades in all four cardinal directions while the EEG was recorded. The resulting stereotypical eye movement artifacts were used offline for ocular artifact correction (cf. Dimigen et al., 2011, 2012). Participants were then instructed for the experiment.
Words were presented in black monospaced Courier font on a white background and separated by one-character blank spaces. At the viewing distance of 60 cm, one character extended 0.43° horizontally. Depending on word length, the visual angle between the center of one word and the left edge of the succeeding word (n + 1) ranged from 1.34° to 1.79° (M = 1.59° ± 0.17°). As required by German orthography, the first letter of each noun was capitalized.
In the active reading condition, each trial started with the presentation of a small black point near the left edge of the screen (see Figure 1). Once a precise fixation was registered (fixation check), the full list of words appeared on the center line of the screen (with the first letter of the first word located 9.4° to the right of the fixation point) together with an additional small point near the right edge of the screen. Participants moved their eyes to read the word list from left to right. As soon as their eyes crossed one of the invisible vertical boundaries located in the middle of each blank space between two adjacent words, the previously masked word to the right of that boundary was uncovered. Thus, directly fixated words were always fully visible. The mean delay between the first recorded horizontal eye position sample beyond the boundary and the update of the stimulus was 10.7 msec (SDbetween-trials = 2.8), meaning that the vast majority of display changes (90%) occurred during the boundary-crossing saccade (mean saccade duration = 32.4 ± 4.1 msec), while the reader is functionally blind (Matin, 1974). After reading the last word, participants looked at the point near the right edge of the screen for at least 800 msec. This triggered the presentation of the response screen, asking whether the list had contained an animal. Participants responded “yes” or “no”
In the passive reading condition, each trial started with a fixation check on a small black point in the screen center (see Figure 1). Afterwards, the point disappeared and the full list of five words was shown on the midline of the screen, with its horizontal position adjusted so that the center of the first word was positioned at the point of fixation. The word list remained on the screen for 300 msec, was then replaced by a blank white screen for 30 msec, and reappeared, but now with the center of the second word positioned at fixation and so on. Thus, the list moved word by word through the central visual field with a (constant) SOA of 330 msec. Apart from the fixated word in the center and the word immediately to its right, all remaining words in the list were replaced by x-masks (as in the active condition). To help maintain fixation, two small arrows were presented above and below the screen center throughout the trial (cf. Li et al., 2015). After presentation of the fifth word, the list was again replaced by the central fixation point for 800 msec and the response screen appeared. The rest of the trial was the same as in active reading.
Each participant completed 500 trials that were split into four blocks (one block per reading condition in each half of the experiment), each containing 125 lists (100 regular and 25 target lists). Regular lists and target lists were presented randomly intermixed. Before each block, participants completed six practice trials to adapt to the reading mode required in the following block. The same lists were presented in the same order to groups of four participants, but with the block order counterbalanced (i.e., A-P-A-P, P-A-P-A, A-P-P-A-, P-A-A-P, where A and P stand for active and passive reading). Hence, across participants, the same word lists were read under active and passive conditions. For each new group of four participants, words were randomly reshuffled into new lists.
Eye Movement Recording
Eye movements were recorded binocularly with a video-based eye tracker (iView-X Hi-Speed 1250, SMI GmbH, Boston, MA) at a sampling rate of 500 Hz. Head position was stabilized by the chin and forehead rests of the eye-tracker. Tracking quality was maintained by running standard 9-point recalibrations every 25 trials or whenever the participant's gaze deviated >0.45° from the fixation point shown at trial onset (fixation check).
Electrophysiological data were recorded from 46 Ag/AgCl electrodes. Forty scalp electrodes were mounted in an elastic cap (Easycap, Eaton, OH) at standard 10–10 system positions. Additionally, six electrodes were positioned on the left and right mastoid (A1 and A2) as well as the outer canthus and infraorbital ridge of each eye. A1 served as reference electrode and an additional electrode at FCz served as ground. The EEG was recorded using Brainamp DC amplifiers (Brain Products GmbH, München, Germany) at a sampling rate of 500 Hz and a band-pass from 0.016 to 250 Hz. Impedances were kept below 5 kΩ. Offline, the EEG was low-pass filtered with a cutoff at 40 Hz and subsequently high-pass filtered at 0.2 Hz using the eegfiltnew function of EEGLAB (version 12.0; Delorme & Makeig, 2004) with default settings. Finally, the EEG was re-referenced against the mean of all electrodes (average reference).
In both reading conditions, we discarded trials containing blinks, missing data in the eye-track, less than three fixations, a vertical gaze deviation of >3.6°, or an incorrect manual response. Furthermore, in the passive reading condition, any deviation of horizontal gaze > 1.3° from the screen center (i.e., beyond the edges of the current word plus the blank space) led to the rejection of the rest of the trial data from that word onwards (Li et al., 2015). Additionally, we removed a total of 36 stimulus-locked epochs with nonocular artifacts in the EEG (see criteria below).
In the active reading condition, saccades and fixations were detected with the binocular algorithm by Engbert and Mergenthaler (2006) with the adaptive velocity threshold set to 5 SDs. Saccades smaller than one character space were considered part of the fixation. Fixation positions in-between words, but on the left or right side of the boundary, were considered part of the fixation on the left or right word, respectively. We removed any data from “bad” fixations, as defined by the following criteria: (1) The display change (i.e., the start of the display cycle with the updated word list presented on the monitor) occurred >20 msec after saccade offset (n = 1883), (2) the left and right eye fixated different words (n = 632), (3) fixation duration was <50 msec (n = 14) or >1000 msec (n = 42), or (4) the corresponding EEG segment contained nonocular artifacts (n = 109, see below).
After rejections, 15,206 epochs (79%) from the active and 16,464 epochs (86%) from the passive reading condition entered final analyses.
Eye Movement Analysis
The data from the active reading condition (with eye movements) was used to analyze oculomotor behavior during reading. As the primary dependent variable for the analysis of first-pass fixation behavior, we used FFD. FFD is the duration of the initial fixation on a word, regardless of the number of subsequent refixations. We chose this measure because it corresponds directly to the fixation-aligned EEG epochs and can be related to FRPs in a more straightforward manner than measures that sum up the duration of multiple fixations (e.g., gaze duration). For statistical testing, FFDs were log-transformed to ensure a normal distribution.
Event- and Fixation-related EEG Analysis
Corneoretinal artifacts from reading saccades were corrected using the surrogate variant of the Multiple Source Eye Correction method (Berg & Scherg, 1994) with identical settings as in Dimigen et al. (2012). Please refer to Dimigen et al. (2011, 2012) for procedural details and a detailed evaluation of the algorithm and its performance on natural reading data. Following ocular correction, segments for ERP and FRP analysis were cut from −200 to +800 msec, relative to stimulus or fixation onsets, respectively; the mean voltages during the 100 msec before stimulus/fixation onsets were subtracted as baselines. To eliminate nonocular EEG artifacts (e.g., voltage drifts, high-amplitude muscle activity), epochs with a peak-to-peak voltage difference > 120 μV in any channel were discarded (1%). To obtain ERPs/FRPs, segments were averaged first within and then across participants.
Eye Movement Statistics
Statistical analyses were carried out with linear mixed effect models (LMMs) using the lmer function of the lme4 package (Version 1.1-7; Bates, Maechler, & Bolker, 2013) in R (v3.1.1; R Core Team, 2013). In the context of reading research, LMMs provide three major advantages over traditional ANOVAs: First, they are able to deal with imbalanced datasets with differing amounts of observations in each design cell without losing statistical power (Pinheiro & Bates, 2000). For example, data loss due to the rejection of trials with bad eye-tracking data is typically not equally distributed over design cells and participants. Second, LMMs allow using participants and items (words) as crossed random effects with one model, whereas ANOVAs require separate F1 and F2 tests for participants and words (Forster & Dickinson, 1976). Third, in LMMs, continuous variables, such as word frequency, do not need to be dichotomized into discrete levels. Such factorization loses information from continuous predictors, conceals nonlinear relationships with the dependent variable, and decreases statistical power (Baayen, 2004; MacCallum, Zhang, Preacher, & Rucker, 2002; Cohen, 1983).
To analyze oculomotor behavior, we modeled (log) FFDs as a function of the predictors preview (a five-level factor) and load. The predictor load was operationalized as the word form frequency of the preceding word in the list. For LMM analysis, load was log-transformed, centered, and entered as a continuous variable. The predictor preview was specified with a treatment contrast, that is, all levels of preview were compared with the baseline masking condition without any visible letters (no preview). The contrast between this baseline and the condition with a full preview corresponds to the classic identity preview benefit effect (Rayner, 1975), which was of central interest here. In the following, we will report all significant level contrasts for the main effect preview. For interactions with the factor load, we focus on the classic contrast for the identity preview benefit (no vs. full).
Following recommendations by Barr, Levy, Scheepers, and Tily (2013), models were run with a maximum random effects structure. If a model did not converge, it was reduced by first removing the random effect correlations and then by successively removing the random effects explaining the least variance until the maximal converging model was identified. The maximal converging FFD model contained random slopes for all fixed effects over participants and words, respectively. No correlations between random effects were included.
For fixed effects, we report regression weights b, standard errors SE, and t values. Because of the lack of an exact estimate of degrees of freedom, obtaining p values in LMMs is not straightforward. However, for large datasets like ours (e.g., Risse & Kliegl, 2014), t values exceeding |1.96| are commonly interpreted as significant because the t distribution approximates the normal distribution for large samples (cf. Baayen, Davidson, & Bates, 2008).
To relate our findings to the preview effects previously reported in FRPs (Dimigen et al., 2012) and ERPs (Li et al., 2015), the spatiotemporal ROI for EEG analysis was defined a priori based on these studies. Primary dependent variable for EEG analysis was the average ERP/FRP amplitude between 200 and 280 msec at left temporal-occipital electrode PO9. Because the falling flank of the N1 component (i.e., its return to baseline after the peak) falls into this window, we refer to EEG amplitude in this late N1 time window simply as “N1 amplitude” in the following, but it is important to note that our window only captures the later parts of the N1 component in most participants. The same LMM statistics as for FFDs were used for ERP/FRP analysis, except that the factor paradigm (2 levels, sum contrast) was included as additional predictor. If interactions between preview and load with paradigm were found, post hoc tests were conducted by rerunning the model with all predictors nested within paradigm.2
To describe the time course and duration of occipitotemporal preview effects, we ran an additional post hoc analysis for the main contrast of the preview effect (no vs. full) with running t tests. These tests were conducted at PO9 in consecutive 10-msec time windows after fixation/stimulus onset (from −100 to 600 msec) and the resulting p values were Bonferroni-corrected.
Task Performance and Reading Behavior
Participants correctly answered the animal question after M = 94.4% (SD = 2.4%) of the trials and there was no difference in accuracy between the active (M = 94.4% ± 2.2%) and passive reading condition (M = 94.3% ± 2.9%; p = .70).
During active reading, participants needed on average 2.4 sec to read the list and typically fixated all of the words—only 2.6% of the words were skipped. The average FFD was 309 msec and therefore not significantly different from the duration of stimulus presentations (set to a fixed value of 300 msec) in the passive condition, t(23) = .150, p = .882. Furthermore, the duration of saccades (32 msec) in the active condition closely matched that of the interword blank intervals (30 msec) in the passive condition. Thus, overall reading speed was closely matched in active and passive reading. Please also note that the overall duration of FFDs in our task was longer than during natural sentence reading (Rayner, 1998), which is expected due to the absence of contextual predictability and short function words in word lists.
The eye-tracking data were also used to compare the relative viewing positions within the fixated words during active and passive reading. This comparison is relevant because, if viewing positions were shifted further to the right in one of the paradigms, this fact alone could lead to differences in the amount of parafoveal processing. However, relative within-word viewing positions (defined as fixated_letter/[word_length + 1]) did not differ between paradigms. In both FRP and ERP blocks, words were typically fixated slightly left of the word center (active reading: M = 0.489 ± 0.064; passive reading: M = 0.492 ± 0.042; t(23) = −0.0255, p = .80). The inclusion of viewing position as an additional predictor in our control LMMs also did not change the pattern of results.
Eye Movement Results
Regarding fixation behavior, results were in accord with our hypotheses: Both preview and load modulated fixation durations in the active reading condition and interacted with each other in their influence (see Table 1). With regard to preview, we replicated the expected benefit: On average, fixations on fully visible words were 26 msec shorter than those on completely masked words. For the preview levels in-between, FFD tended to decrease as a function of the number of letters visible in parafoveal vision, that is, the more letters of a word had been previewed during the preceding fixation, the shorter were the subsequent fixations on that word. In the LMM analysis, this was confirmed by a significant main effect of Preview on FFDs, revealing highly significant differences between the preview levels full preview (b = −0.080, SE = 0.011, t = −7.06) and 3 (b = −0.022, SE = 0.009, t = −2.43) as compared against the baseline condition without useful preview (no preview).
|Preview Level during Last Fixation .||First-fixation Duration (in msec) .|
|Overall .||Foveal Load during Last Fixation .|
|Low .||High .|
|No preview||326 (39)||326 (40)||326 (40)|
|1 letter||329 (40)||321 (39)||337 (44)|
|2 letters||321 (41)||316 (42)||327 (42)|
|3 letters||319 (39)||315 (37)||322 (43)|
|Full preview||300 (36)||295 (38)||304 (38)|
|Preview benefit (full preview – no preview)||−26 msec||−31 msec||−22 msec|
|Preview Level during Last Fixation .||First-fixation Duration (in msec) .|
|Overall .||Foveal Load during Last Fixation .|
|Low .||High .|
|No preview||326 (39)||326 (40)||326 (40)|
|1 letter||329 (40)||321 (39)||337 (44)|
|2 letters||321 (41)||316 (42)||327 (42)|
|3 letters||319 (39)||315 (37)||322 (43)|
|Full preview||300 (36)||295 (38)||304 (38)|
|Preview benefit (full preview – no preview)||−26 msec||−31 msec||−22 msec|
Values are provided for words 2–5 in the list. Overall mean FFD, including the first word, was 309 msec.
A handful of eye-tracking studies have reported stronger preview benefits if the previous word n − 1 was relatively easy to process (Henderson & Ferreira, 1990); however, this effect was not always consistently found in early fixation measures (Schroyens et al., 1999). Here, we clearly replicate both a main effect of Load as well as a robust interaction between Load and Preview. On average, fixations on word n were longer by 9 msec when the word n − 1 had been difficult rather than easy to process (main effect of Load on FFD on word n; b = −0.006, SE = 0.002, t = −3.19). Furthermore, as Table 1 shows, the preview benefit measured on word n was larger after easy words (M = 31 msec) than after difficult words (M = 14 msec; significant interaction between Preview benefit [no vs. full] and Foveal load; b = −0.011, SE = 0.004, t = −2.61). No effect of Previous load was observed in the no preview subcondition (see Table 1) in which preprocessing was not possible because the upcoming word was fully masked. This pattern suggests that the observed “spillover” of processing difficulty from word n − 1 into fixations on word n is not a general consequence of more difficult processing but specifically caused by load-dependent changes in the extent of parafoveal preprocessing during the fixation of word n − 1.
EEG Results: Preview
Figures 2 and 3 present the corresponding electrophysiological data and now also includes data from the passive reading condition requiring steady fixation. In both reading conditions, absolute brain responses to words were dominated by the typical P1–N1 complex, with the visually evoked P1 peaking around 90 msec after stimulus/fixation onset at occipital electrodes, followed by an N1 with a bilateral temporal-occipital distribution.
Importantly, brain responses to words were strongly influenced by the amount of useful parafoveal information available during the preceding fixation on word n − 1: EEG amplitude during the falling flank of the N1 (average amplitude between 200 and 280 msec at PO9) was substantially larger when word n had been completely masked during the preceding fixation than when it had been completely visible (Figure 2). In fact, EEG amplitude in this windows reflected the number of previewed letters in a perfectly ordinal manner: Across the five preview levels—and aggregated across both reading conditions—N1 amplitude decreased monotonically from no preview (M = −2.22 μV) to the one-, two-, and three-letter preview conditions (−1.96, −1.47, and −1.27 μV, respectively) and was finally much attenuated after full preview (M = −0.34 μV), mirroring the pattern in oculomotor behavior. LMMs confirmed this robust preview effect: All levels differed significantly from the baseline condition with no preview: 1 (t = 2.141, b = 0.254, SE = 0.119), 2 (t = 6.591, b = 0.713, SE = 0.108), 3 (t = 7.001, b = 0.907, SE = 0.130), and full (t = 10.740, b = 1.801, SE = 0.168). Thus, amplitude in the late N1 window reflects the amount of parafoveally preprocessed information at the level of letters.
EEG Results: Active versus Passive Reading
Figure 2 compares preview effects in FRPs and ERPs. A first important finding is that the influence of preview on the N1 was seen both during active reading and in the passive RSVP-with-flankers paradigm. Figure 2A shows the scalp-electric distribution of the preview effect in each reading condition both as raw differences and with the topographies normalized by the strength of the effect (amplitudes at each electrode were normalized by dividing them by the standard deviation across the scalp; McCarthy & Wood, 1985). Importantly, normalization shows that the N1 preview effect is qualitatively similar in active and passive reading.
However, the size of the preview effect differed strongly between conditions. In FRPs recorded during active reading, mean N1 amplitude was reduced by 2.94 μV after full preview (M = 0.31 μV) relative to the baseline level without useful preview (M = −2.63 μV); in contrast, this difference was only 0.82 μV in ERPs (full preview M = −0.98 μV; no preview M = −1.8 μV). The LMM confirmed that each of the preview contrasts interacted with reading paradigm: 1 (t = 3.397, b = 0.354, SE = 0.104), 2 (t = 5.608, b = 0.584, SE = 0.104), 3 (t = 5.954, b = 0.622, SE = 0.104), and full preview (t = 9.188. b = 0.962, SE = 0.105). In fact, post hoc tests showed that all contrasts of the factor preview were significant in FRPs, whereas in ERPs only the maximal contrast (no vs. full preview) reached significance (Figure 2E).
A post hoc analysis of the time course of the preview effect at PO9, shown in Figure 2C, suggests that it was not only much stronger, but also more long-lasting in FRPs (significant from 160 to 300 msec) than in ERPs (significant only from 230 to 250 msec). In summary, these results show that, although preview effects are replicable in the EEG irrespective of the experimental paradigm, they are much larger during active, left-to-right reading.
We also performed two control analyses (Figure 3C). First, for the case of FRPs, it is important to ensure that condition differences in the wave shape are not simply a trivial reflection of the preview benefit in fixation durations, that is, the result of systematically changing temporal overlap with the brain potentials evoked by the following fixation on word n + 1 (cf. Kamienkowski, Ison, Quiroga, & Sigman, 2012; Dimigen et al., 2011). We therefore repeated our FRP analysis on a subset of n = 10,575 fixations with a minimum duration of 280 msec. In this sample, the interval up to 280 msec is largely free of overlapping potentials from the next saccade and fixation. As shown in the right panel of Figure 3C, the pattern of N1 results in this control analysis was the same as in the analysis including all fixations.
Second, to underline that our N1 results are unrelated to different visual stimulation caused by showing preview masks with a varying number of x-letters, we analyzed the effect of different masking conditions for word n on the FRPs time-locked to the first fixation of the preboundary word n − 1. As can be seen in the left panel of Figure 3C and as expected, the amount of masked letters in the parafovea had no effect on the potentials elicited by fixations on word n − 1. This also means that the ongoing brain activity that overlaps in time with the following fixation on word n was the same in all preview conditions.
EEG Results: Load
Finally, we found that EEG responses to visual words were also influenced by the processing load during the fixation of the previous word in the list. Amplitude in the late N1 window was generally more negative following an easy rather than a difficult word n − 1, which manifested as a main effect of Load (t = −4.867, b = −0.084, SE = 0.17). Furthermore, this load effect was larger in FRPs than in ERPs as reflected in a significant interaction of load and paradigm (t = 2.491, b = 0.043, SE = 0.017). Please note, however, that because load was manipulated via the lexical frequency of word n − 1, a main effect of Load is expected in this paradigm and most likely reflects an overlapping late N400 frequency effect (Dambacher & Kliegl, 2007; see also Van Petten & Kutas, 1990) elicited by word n − 1. Because this late frequency effect overlaps with the analysis epoch for word n and because it does so to slightly different degrees in ERPs (less latency jitter) and FRPs (more latency jitter), any main effect of Load (and its interaction with paradigm) on word n must be interpreted with caution.
Importantly, however, and in accordance with the oculomotor results, the load effect also interacted with preview: As shown in Figure 4, preview had a significantly larger impact on the N1 if the preceding word n − 1 was easy rather than difficult to process (interaction of Load × Preview; t = 3.313, b = 0.179, SE = 0.054). This interaction was independent of reading mode, that is, the three-way interaction of Preview, Paradigm, and Load did not reach significance (all ts < |1.96|). However, there was at least a numeric tendency for a larger influence of load in FRPs than ERPs (size of the Load × Preview interaction in FRPs = 0.85 μV and in ERPs = 0.36 μV; see Figure 4B). Importantly, these results suggests that, under natural reading conditions with parafoveal information, the N1 elicited by a fixated word n reflects also the difficulty of the previously encountered word n − 1, because this load determines when or how much attention can be allocated to the upcoming word.
Passive stimulation paradigms based on the serial visual presentation of words are popular in ERP research but are unable to capture important properties of the natural reading process. Here we investigated three basic aspects of everyday reading with brain-electric responses to visual words: parafoveal preprocessing, eye movement execution, and foveal processing difficulty during past fixations. Participants read short lists of German nouns with the task to detect occasional animal names. In the active reading condition, they moved their eyes freely over the text with saccades; in passive reading blocks, they fixated on a location in the screen center while the same visual input was presented using the RSVP-with-flankers paradigm. In both conditions, the average reading rate was closely matched, upcoming words were visible to various degrees, and processing load was manipulated by showing words of different frequencies. As discussed in the following, all three factors—and their interactions—had a considerable impact on scalp-recordable correlates of word recognition.
Parafoveal Preprocessing Modulates the Brain-electric Response to Words
During everyday reading, the processing of a word does not begin in foveal vision. Instead, readers usually obtain some visually degraded information about soon-to-be fixated words, and this preprocessing eases word recognition once the word is looked at. Here, we further explored the brain-electric signature of this preview effect, first observed by Dimigen et al. (2012) in FRPs and recently replicated in stimulus-locked ERPs (Li et al., 2015). Over temporal-occipital areas, a valid preview as compared with a partially or fully masked preview modulated the wave shape between 160 and 300 msec, a time range corresponding to the falling flank of the N1 component. As visible in Figure 3A, this change in ERP and FRP morphology was visible at various electrode sites (e.g., Cz) but was most pronounced at inferior–posterior electrodes of the left hemisphere—and therefore over areas implicated in early stages of orthographic processing (McCandliss, Cohen, & Dehaene, 2003). As hypothesized, the N1 component was not just considerably smaller after valid previews, but EEG amplitude in this window reflected the number of previewed letters in a graded, strictly monotonic fashion (see Figure 2B and E).
The time course and left temporal-occipital maximum of the effect fit to previous N1/N170 findings (e.g., Maurer, Brandeis, & McCandliss, 2005; Bentin et al., 1999). In alphabetic writing systems, eye-tracking studies have found the preview benefit to depend on the orthographic overlap between preview and target, rather than similarity in terms of low-level features or meaning (Schotter et al., 2012). On the basis of this evidence, the most parsimonious explanation for the N1 attenuation is that it reflects a form of partial, trans-saccadic repetition priming that activates abstract orthographic and phonological representations (Dimigen et al., 2012).
We can rule out several low-level explanations for the N1 effects reported here. First, Dimigen et al. (2012) discuss the possibility that the preview positivity in FRPs could be an artifact of the visual transient generated by the perisaccadic unmasking of the target in the invalid preview condition, whereas no such change happens in the fully valid condition. Our results argue against such an explanation because we also found N1 differences between partial masking conditions which all involved a display change. More importantly, qualitatively similar N1 effects were also observed in the RSVP-with-flankers paradigm, where there is an intense display change in all conditions, as the entire set of words moves to a new screen position. Second, a control analysis on a subset of long fixations confirmed that our EEG results were unaffected by overlapping potentials from the next fixation (see Figure 3C). Third, given that it has been shown that N1 effects are similar for participants who did and did not notice display changes during the experiment (Dimigen et al., 2012), it is unlikely that the N1 effects reported here are related to differences in the participants' awareness of the preview manipulation.
Please note, however, that we cannot rule out the possibility that the N1 effect reflects not only preview benefits but also costs due to the violation of implicit expectations in the boundary paradigm. The predictive coding framework (Friston, 2005) holds that humans constantly generate predictions about likely sensory inputs, whereas the actual sensory input serves as an error signal to allocate attention to unpredicted events. The N1 modulations found here show at least some superficial resemblance to the visual MMN (Stefanics, Kremlacek, & Czigler, 2014), which has been interpreted as such a mismatch signal (Rao & Ballard, 1999). Thus, it is conceivable that in the classic boundary paradigm, expectations before the saccade may be (unconsciously) violated after the saccade because the stimulus changes in the conditions with an invalid preview. The question whether invalid previews constitute an appropriate “neutral” baseline for the measurement of preview benefits or whether they can also lead to processing costs is as old as the boundary paradigm itself (McClelland & O'Regan, 1981; Rayner & Slowiaczek, 1981) and this issue has received increased interest very recently (Hutzler et al., 2013; Kliegl, Hohenstein, Yan, & McDonald, 2013). Coregistration is clearly a promising technique to address this question in more detail in the future. With regard to this study, it is important to stress that our main findings—that saccades and load systematically influence the amount of information extracted from parafoveal vision during reading—holds regardless of whether the N1 effect reflects pure preview benefit or a combination of benefits and costs.
Importantly, the preview positivity not only corresponds well with the graded effect on fixation durations; but with an onset around 160 msec, it also begins early enough to influence oculomotor behavior. With conservative testing, robust effects of preview emerged in the FRP 160–170 msec after fixation onset3—or around 160 msec before the end of the first fixation on the word, which lasted 309 msec on average (see the left part of Figure 2D). Saccade preparation in reading has been estimated to take 150–175 msec (Rayner et al., 1983). Thus, based on the temporal contingencies observed here, the preview positivity is a plausible correlate of the facilitatory brain processes that eventually lead to the preview benefit in behavior.
Please note that, in our present analyses, we did not include the N400 component, because the influences of preview on the N400 reported by Dimigen et al. (2012) and Li et al. (2015) occur at a latency that is too late to affect first fixation behavior. However, we did replicate an N400 attenuation by valid previews in the present data set, and this result also explains the late condition difference that can be seen at mid-central electrode Cz between 400 and 500 msec (compare Figure 3A) as well as three significant t tests at electrode PO9 between 460 and 480 msec (see FRP statistics in Figure 2C, left).
Brain-electric Effects of Preview Depend on Active Saccade Execution
The two existing studies on the brain-electric effects of saccade execution during reading (Temereanca et al., 2012; Marton et al., 1985) arrived at opposite conclusions and neither of them investigated the effect in the presence of useful parafoveal information. Here, we were able to directly compare the effect of eye movements being present or absent. Importantly, the effect of preview was substantially larger in active reading with eye movements. The preview positivity has been reported in active as well as passive reading before, but prior studies were too different to allow for direct comparisons. The difference observed here can be explained by attention mechanisms in a straightforward manner. In reading, saccades usually target the next word, which is currently in the parafovea. A large body of evidence demonstrates a facilitating effect of saccade planning on stimulus processing. Saccades are preceded by an obligatory shift of visuospatial attention toward the saccade target (Rolfs & Carrasco, 2012; Deubel & Schneider, 1996; Hoffmann & Subramaniam, 1995) and preparing an eye movement toward a stimulus has been shown to increase the strength of its neural representation in visual cortex (e.g., Mazer & Gallant, 2003; Sheinberg & Logothetis, 2001; Chelazzi, Miller, Duncan, & Desimone, 1993), to increase the perceived contrast of the stimulus (Rolfs & Carrasco, 2012), and to mitigate the detrimental effects of visual crowding (Pelli, 2008) on extrafoveal object recognition (e.g., Harrison, Mattingley, & Remington, 2013). Although central saccadic suppression may outlast the saccade and therefore inhibit early visual processing of the postsaccadic input (as suggested by Temereanca et al., 2012), such costs are probably more than compensated by increased parafoveal preprocessing at the saccade goal during natural vision.
Because there is no need to plan saccades in the RSVP-with-flankers paradigm, an attentional reorientation toward the parafovea is unnecessary if not counterproductive in this condition—after all, participants are explicitly instructed to keep their gaze in the center—and engaging in this secondary fixation may in itself demand attentional resources. Moreover, word stimuli in the RSVP-with-flankers paradigm move leftwards across different presentations of the same word array (i.e., each word first appears on the right and then moves step by step to the left). Because the direction of this apparent motion is opposite to the direction of normal reading, it may bias the reader's attention toward the left in a reflexive attempt to follow the stimulus, similar to the optokinetic nystagmus. This bias might persist even if only a single flanker word is presented on each side of the foveal word (e.g., Li et al., 2015; Barber et al., 2010, 2013).
Regardless of the exact underlying mechanism—enhanced preprocessing with saccades or also compromised preprocessing with passive stimulation paradigms—our results show that, compared with passive vision, saccade behavior changes the deployment of attention. Crucially, this attention difference mainly affects reading performance through a change in the depth of parafoveal preprocessing. The important implication is that, to fully capture influences of preview (Li et al., 2015; Barber et al., 2013; Pernet et al., 2007) or saccades (Temereanca et al., 2012; Marton et al., 1985), both factors need to be included in the experimental design. In contrast, the RSVP-with-flankers paradigm appears to underestimate the size of preview effects.
Difficulty of the Last Word Influences the Current Neural Response
The foveal load hypothesis in reading (Henderson & Ferreira, 1990) holds that the difficulty of a fixated word dictates the attentional resources required for its processing. Consequently, if foveal stimulus processing is easy, surplus resources can be deployed to preprocess parafoveal information, and this speeds up word recognition once the next word is directly fixated. Although such interactions between load and preview have been shown in eye-tracking studies (White et al., 2005; Schroyens et al., 1999; Kennison & Clifton, 1995; Henderson & Ferreira, 1990), the findings have not always been robust and consistent. Specifically, in some studies interactions were constrained to a subset of fixations in which the last preboundary fixation was nearby (Kennison & Clifton, 1995), to a subset of participants who remained completely unaware of the display changes (White et al., 2005), or to late fixation time measures such as gaze duration (Schroyens et al., 1999).
In this study, the interaction manifested robustly during the initial fixation on the word and—for the first time—also even earlier in the electrophysiological response to the word. As predicted by the foveal load hypothesis, the overall preview benefit was larger if processing load was low (31 msec benefit) rather than high (22 msec benefit) during the preceding fixation. Similarly, the N1 effect—the preview positivity—was significantly stronger after easy than difficult words. In contrast to the main effect of Preview on the N1, which was much stronger during active than passive reading, the interaction between load and preview was not qualified by the reading paradigm. It is possible that this absence of a three-way interaction is simply an issue of statistical power, because the numerical trends go in the expected direction (a larger modulation of the preview effect by load in active reading).
Although our results show that past processing load affects early stages of word recognition, they do not provide answers about the exact attentional mechanism underlying this interaction. Computational models of eye guidance in reading can account for such an effect in two ways: Serial models that assume that readers allocate attention to only one word at a time in a strictly serial fashion (Reichle, 2011; Reichle, Pollatsek, & Rayner, 2006) predict that, if the current word is easy to process, attention will move to the parafoveal word sooner, leaving more time to preprocess it covertly before the saccade toward that word is programmed and executed. In contrast, parallel models which assume that readers process multiple words simultaneously within a distributed attentional gradient (i.e., within a “zoom lens” of varying size; Schad & Engbert, 2012; see also Reilly & Radach, 2006) posit that an easy foveal word will cause the reader to spread attention more broadly around the fixated location, allowing for stronger concurrent preprocessing of the next word. Future studies using the coregistration method may help to resolve which of these accounts is correct. Irrespective of the exact underlying mechanism, our results provide evidence from three independent methods—behavior, ERPs, and FRPs—in support of the foveal load hypothesis. Any realistic model of visual word recognition needs to appreciate that, under natural conditions, the speed and efficiency of word recognition also depend on the previously encountered word.
During natural reading, the recognition of words extends across multiple fixations, and this process is facilitated by the execution of eye movements. Also, because several words are visible during each fixation and their processing draws on limited attentional resources, word recognition depends on the difficulty of the previously fixated word. It is therefore more interactive than suggested by traditional ERP recordings using RSVP. Taken together, the three properties of natural reading investigated here—preview, saccades, and load—have a substantial impact on the neural response to words and should be considered if one wishes to draw conclusions about normal reading from studies of visual word recognition. As demonstrated here, the coregistration of eye movements and EEG is a suitable approach to elucidate these processes.
This work was supported by a grant (So177/19-2) from the German Research Foundation (DFG) within Research Group 868. We would like to thank Linda Gerresheim and Niels Krause for their help with data collection and Michael Dambacher for cocreating the word materials used in this study.
Reprint requests should be sent to Benthe Kornrumpf or Olaf Dimigen, Institut für Psychologie, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099 Berlin, Germany, or via e-mail: firstname.lastname@example.org, email@example.com.
A main advantage of the list reading paradigm is that brain responses are obtained for every word in every list. This allows us to isolate specific neural correlates of the natural reading process with a high signal-to-noise ratio. Of course, the task itself is still relatively artificial compared with the reading of sentences or connected paragraphs of text.
To control for additional covariates often studied in reading research, we also ran two more complex models for the eye movement and the EEG data. As additional predictors, these LMMs also included word length, (log) lexical frequency, list position (1–5), and landing position of the currently fixated word, as well as launch site (distance of last fixation from the beginning of the current word), latency of the display change relative to fixation onset (in msec; always 0 in ERPs), and a binary variable coding whether the word was repeated or not (first/second half of experiment). All covariates were centered and the repetition factor was contrast coded using a sum contrast. Because the pattern of effects was unchanged in these models, we only report results from the simple models here. The fact that words were repeated once in the second half of the experiment also had no influences on the pattern of significant effects.
Please note that in FRPs there was also a pattern of small but seemingly systematic preview effects shortly after fixation onset (30–70 msec; see Figure 2B), which were not significant after Bonferroni correction. It is unclear whether this pattern reflects extremely early influences of preview or whether it is related to the perisaccadic display changes. This is an interesting issue for future research with a priori hypotheses about this latency.
These authors contributed equally to this study.