Abstract

Sensory processing is strongly influenced by prior expectations. Valid expectations have been shown to lead to improvements in perception as well as in the quality of sensory representations in primary visual cortex. However, very little is known about the neural correlates of the expectations themselves. Previous studies have demonstrated increased activity in sensory cortex following the omission of an expected stimulus, yet it is unclear whether this increased activity constitutes a general surprise signal or rather has representational content. One intriguing possibility is that top–down expectation leads to the formation of a template of the expected stimulus in visual cortex, which can then be compared with subsequent bottom–up input. To test this hypothesis, we used fMRI to noninvasively measure neural activity patterns in early visual cortex of human participants during expected but omitted visual stimuli. Our results show that prior expectation of a specific visual stimulus evokes a feature-specific pattern of activity in the primary visual cortex (V1) similar to that evoked by the corresponding actual stimulus. These results are in line with the notion that prior expectation triggers the formation of specific stimulus templates to efficiently process expected sensory inputs.

INTRODUCTION

Prior expectations affect sensory processing at the earliest stages of the cortical hierarchy (Kok, Rahnev, Jehee, Lau, & De Lange, 2012; Todorovic & De Lange, 2012; Arnal, Wyart, & Giraud, 2011; Todorovic, Van Ede, Maris, & De Lange, 2011; Alink, Schwiedrzik, Kohler, Singer, & Muckli, 2010; Den Ouden, Friston, Daw, McIntosh, & Stephan, 2009; Summerfield, Trittschuh, Monti, Mesulam, & Egner, 2008). Valid expectations have been shown to lead to improvements in perception (Bar, 2004) as well as in the quality (Kok, Jehee, & De Lange, 2012) and content (Kok, Brouwer, Van Gerven, & De Lange, 2013) of sensory representations in early visual cortex. These findings are in line with a framework in which perception is cast as a process of inference, wherein bottom–up stimulus inputs and prior knowledge jointly determine the contents of perception (Fiser, Berkes, Orbán, & Lengyel, 2010; Yuille & Kersten, 2006; Helmholtz, 1867). In this framework, prior knowledge is expected to affect sensory processing not only when this is required for optimal task performance (Ma, 2012), but even when stimuli are fully task irrelevant (Yaron, Hershenhoren, & Nelken, 2012; Alink et al., 2010; Den Ouden et al., 2009). However, despite numerous demonstrations of prior expectations affecting stimulus processing, very little is known about the neural correlates of the expectations themselves.

Previous studies have demonstrated increased activity in sensory cortex following the omission of an expected stimulus compared with when a stimulus was not expected (SanMiguel, Widmann, Bendixen, Trujillo-Barreto, & Schröger, 2013; Kok, Rahnev, et al., 2012; Todorovic et al., 2011; Wacongne et al., 2011; Den Ouden et al., 2009). Although this has sometimes been interpreted as a “pure expectation“ signal, uncontaminated by bottom–up signals, it is unclear whether this increased activity constitutes a general surprise signal or has representational content. For example, early visual cortex shows an overall activity increase when an auditory target is selected (Swallow, Makovski, & Jiang, 2012), showing that temporal selection of behaviorally relevant items can enhance sensory activity in other modalities in a rather unspecific manner. It is also possible that the omission of an expected event draws spatial attention but does not have any feature specificity. However, another intriguing possibility is that top–down expectation leads to a template of the expected stimulus being set up in visual cortex, which can then be compared with subsequent bottom–up input (Friston, 2005; Rao & Ballard, 1999; Mumford, 1992). In line with this, feature-specific top–down effects on visual cortex have been shown as a consequence of preparatory attention (Stokes, Thompson, Nobre, & Duncan, 2009), mental imagery (Albers, Kok, Toni, Dijkerman, & De Lange, 2013; Lee, Kravitz, & Baker, 2012; Stokes, Thompson, Cusack, & Duncan, 2009), and working memory (Harrison & Tong, 2009; Serences, Ester, Vogel, & Awh, 2009).

To test this hypothesis, we used fMRI to investigate whether the neural activity in early visual cortex elicited by the expectation of a particular stimulus is specific to the feature that is being expected. Such feature specificity would suggest that prior expectation leads to the generation of a stimulus template in visual cortex.

METHODS

Participants

Twenty-six healthy right-handed individuals (17 women, mean age = 22 years, SD = 2 years) with normal or corrected-to-normal vision gave written informed consent to participate in this study in accordance with the institutional guidelines of the local ethics committee (CMO region Arnhem-Nijmegen, The Netherlands). Data from three participants were excluded because of excessive (>5 mm) head movements.

Stimuli

Grayscale luminance-defined sinusoidal Gabor grating stimuli were generated using MATLAB (MathWorks, Natick, MA) and the Psychophysics Toolbox (Brainard, 1997). In the behavioral session, stimuli were presented on a Samsung Syncmaster 940BF screen (1024 × 786 screen resolution, 60 Hz refresh rate). In the fMRI session, they were projected on a rear projection screen using a luminance-calibrated SONY VPL FX40 projector (1024 × 768 resolution, 60 Hz). A set of two gratings (20% Michelson contrast) was displayed in succession in an annulus (outer diameter: 15° of visual angle, inner diameter: 3°), surrounding a fixation point. The two gratings differed from each other in terms of phase and spatial frequency. The first grating had random spatial phase, and the second grating was in counterphase to the first grating. The two gratings had spatial frequencies of 1.0 and 1.5 cpd, respectively; the order of which was pseudorandomized and counterbalanced over conditions. The grating stimuli were presented for 100 msec each, separated by a blank screen (500 msec).

Experimental Design

Each trial started with an auditory cue (200 msec), consisting of either a low-frequency (450 Hz) or high-frequency (1000 Hz) tone, which predicted the orientation of the first grating stimulus of the pair with 100% validity (45° or 135°; see Figure 1). On 75% of trials, participants were then presented with a set of gratings, the first of which had the (expected) orientation, and the second was tilted a few degrees clockwise or anticlockwise with respect to the first (Stimulus trials; Figure 1A). On these trials, participants performed an orientation discrimination task, wherein they judged whether the second grating was rotated clockwise or anticlockwise. On the remaining 25% of trials, no gratings were presented (Omission trials; Figure 1B). Therefore, on these trials, participants had an expectation of a particular visual stimulus, but no actual visual input. Participants had no task on these trials, except for holding central fixation. The intertrial interval (ITI) was jittered between 3250 and 5250 msec. A central fixation point was presented throughout the trial and ITI. All participants completed two runs, consisting of two blocks of 64 trials each, yielding 256 trials. The contingencies between cues and gratings were counterbalanced over participants.

Figure 1. 

Experimental paradigm. (A) Each trial started with an auditory cue that predicted the orientation of the subsequent grating stimulus (45° or 135°) with 100% validity. On 75% of trials, participants were then presented with a set of gratings, the first of which had the (expected) orientation and the second was tilted a few degrees clockwise or anticlockwise with respect to the first. On such trials, participants performed an orientation discrimination task, wherein they judged whether the second grating was rotated clockwise or anticlockwise with respect to the first grating. (B) In 25% of trials, no gratings were presented. On these trials, there was an expectation of a particular visual stimulus but no actual visual input. Participants had no task in these trials, except for holding central fixation. (C) Throughout the experiment, two different tones were used as cues, each one predicting one of the two possible orientations.

Figure 1. 

Experimental paradigm. (A) Each trial started with an auditory cue that predicted the orientation of the subsequent grating stimulus (45° or 135°) with 100% validity. On 75% of trials, participants were then presented with a set of gratings, the first of which had the (expected) orientation and the second was tilted a few degrees clockwise or anticlockwise with respect to the first. On such trials, participants performed an orientation discrimination task, wherein they judged whether the second grating was rotated clockwise or anticlockwise with respect to the first grating. (B) In 25% of trials, no gratings were presented. On these trials, there was an expectation of a particular visual stimulus but no actual visual input. Participants had no task in these trials, except for holding central fixation. (C) Throughout the experiment, two different tones were used as cues, each one predicting one of the two possible orientations.

After the main experiment, participants performed a functional localizer task in the magnetic resonance scanner. This consisted of flickering gratings (2 Hz), presented at 100% contrast, in blocks of 12 sec. Each block contained gratings with a fixed orientation (45° or 135°) and random spatial phase and spatial frequency (either 1.0 or 1.5 cpd). The two orientations were presented consecutively (order was pseudorandomized), followed by a 12-sec blank screen, containing only a fixation point. To ensure central fixation, participants were required to press a button whenever the white fixation dot turned black (four to eight times per 36-sec block, at unpredictable times). All participants were presented with 16 blocks, yielding ∼10 min of scanning.

The same task was used during the retinotopic mapping run, in which participants viewed a wedge, consisting of a flashing black-and-white checkerboard pattern (3 Hz), first rotating clockwise for nine cycles and then anticlockwise for another nine cycles (at a rotation speed of 24 sec/cycle).

Behavioral Session

To familiarize participants with the task and to assess whether the predictive cue had behavioral consequences for perceptual processing, the fMRI session was preceded by a behavioral experiment, which was identical to the imaging experiment except for the following changes.

In the behavioral session, expectations were violated (25% of trials) not by omitting the grating stimuli but by presenting gratings with the nonpredicted orientation (e.g., 45° when 135° was predicted). During this session, participants performed the orientation discrimination task both on expected and unexpected trials. Note that the response on the orientation discrimination task is orthogonal to the orientation expectation, thereby avoiding potential confounds with respect to response bias. During the behavioral session, the orientation difference between the two gratings was determined by an adaptive staircase procedure (Watson & Pelli, 1983), which was set to an overall correct response percentage of ∼75%. The orientation differences for expected and unexpected trials were controlled by a single, joint staircase to prevent differences in physical stimulus attributes between conditions. Furthermore, orientation differences were updated only between blocks to prevent potential differences between conditions because of fluctuations in orientation difference over the course of a block. The final staircase thresholds obtained during this behavioral session were used in the fMRI session, where orientation differences were kept fixed. Effects of Expectation (Expected vs. Unexpected) and Orientation (45° vs. 135°) on accuracy and RTs were tested using two-way repeated-measures ANOVAs.

Before the actual task, participants were given instructions and performed a short training run, consisting of two blocks of 32 trials. After this, participants completed six blocks of 64 trials, yielding a total of 384 trials. The ITI was fixed at 1700 msec.

fMRI Acquisition and Analysis

Functional images were acquired using a 1.5-T Avanto MRI system (Siemens, Erlangen, Germany) with a T2*-weighted gradient-echo EPI sequence (repetition time/echo time = 2000/40 msec, 33 transversal slices, voxel size = 3 × 3 × 3 mm, 80° flip angle). Anatomical images were acquired with a T1-weighted MP-RAGE sequence (repetition time/echo time = 2250/2.95 msec, voxel size =1 × 1 × 1 mm, 15° flip angle). SPM8 (www.fil.ion.ucl.ac.uk/spm, Wellcome Trust Centre for Neuroimaging, London, UK) was used for image preprocessing. The first four volumes of each run were discarded to allow T1 equilibration. All functional images were spatially realigned to the mean image, yielding head movement parameters that were used as nuisance regressors in the general linear model and temporally aligned to the first slice of each volume. The structural image was coregistered with the functional volumes.

Data of each participant were modeled using an event-related approach within the framework of the general linear model. Regressors representing the four different conditions (the two trial types and the two expected orientations) were constructed by convolving the (expected) onsets of the first grating in each trial (i.e., 750 msec postcue) with a canonical hemodynamic response function. The auditory cues were not modeled separately, in view of the short SOA between cues and stimuli. Instead, the modeled BOLD response was taken to be an aggregate of cue-evoked and stimulus-evoked activity (or, in the case of omission trials, cue-evoked activity only). Instruction screens at the beginning of each block were included as regressors of no interest, as were head motion parameters (Lund, Norgaard, Rostrup, Rowe, & Paulson, 2005). Finally, the data were high-pass filtered (cutoff = 128 sec) to remove low-frequency signal drifts. A similar model was constructed for the data of the localizer run, with separate regressors for the two grating orientations (45° and 135°). Additionally, to visualize the time course of the BOLD signal evoked by stimulus and omission trials, the data from the experiment were modeled using a finite impulse response approach. Here, the two trial types were each modeled by eight 2-sec time bins, covering a 16-sec post-stimulus period.

Freesurfer (surfer.nmr.mgh.harvard.edu/) was used to identify the boundaries of retinotopic areas in early visual cortex using established methods (Sereno et al., 1995). To probe the orientation specificity of the signals in visual cortex, data from the independent functional localizer run were used to split the voxels in each ROI into two populations: those responding more strongly to 45° grating stimuli and those responding more strongly to 135° grating stimuli. Specifically, for each ROI (V1, V2, and V3), we selected the 100 voxels that showed the most reliable preference for 45° (highest t value for the contrast 45°–135° gratings) and the 100 voxels that showed the most reliable preference for 135°. BOLD responses were averaged over voxels for these two populations separately. In the main experiment, we expected BOLD responses to be larger for stimuli with the orientation that voxels responded preferentially to during the functional localizer than for stimuli with the nonpreferred orientation (Kamitani & Tong, 2005; Haxby et al., 2001). Similarly, if neural responses evoked by prior expectations are feature specific, we would expect the BOLD response in voxels preferring 45° gratings to be larger for omissions of 45° stimuli than for omissions of 135° stimuli, and vice versa for voxels preferring 135° gratings. These effects were assessed by two-way repeated-measures ANOVAs, with the factors Stimulus/Omission Orientation and Voxel Orientation Preference. Feature-specific activity would be reflected by an interaction between the two factors.

Additionally, to assess the overall BOLD amplitude and its time course in these ROIs, we averaged parameter estimates from the finite impulse response model (Figure 2A) and the model incorporating the canonical hemodynamic response function (Figure 2B, bars) over the selected voxels.

Figure 2. 

BOLD signal evoked by stimuli and omissions in V1. (A) Time course of BOLD signal in V1, time locked to (expected) onset of visual stimuli. (B) Amplitude of BOLD response evoked by 45° (blue bars) and 135° (yellow bars) stimuli and omissions, separately for voxels preferring 45° and 135° oriented gratings. Error bars indicate SEM (*p < .05, ***p < .001).

Figure 2. 

BOLD signal evoked by stimuli and omissions in V1. (A) Time course of BOLD signal in V1, time locked to (expected) onset of visual stimuli. (B) Amplitude of BOLD response evoked by 45° (blue bars) and 135° (yellow bars) stimuli and omissions, separately for voxels preferring 45° and 135° oriented gratings. Error bars indicate SEM (*p < .05, ***p < .001).

Finally, to establish the retinotopic specificity of the neural response to omissions (and stimuli), we probed the BOLD response in visual cortex voxels that did not correspond to the retinotopic locations at which the gratings were presented. To this end, we selected voxels that did not show a positive BOLD response during the functional localizer. To ensure that we were probing visually responsive voxels, only voxels with a significant response to the rotating wedge stimulus (p < .01) were included. These voxels therefore responded to visual stimulation, but not at the retinotopic locations at which the gratings were presented (and expected) in the current study. For each ROI and each participant, we included all voxels that met these criteria (V1: mean = 95 voxels, SD = 33 voxels; V2: mean = 83 voxels, SD = 30 voxels; V3: mean = 52 voxels, SD = 24 voxels). As a comparison, we selected the same number of stimulus-responsive voxels for each ROI and each participant.

RESULTS

Behavioral Results

We expected a valid orientation expectation to sharpen the neural representation of the orientation of the first grating (Kok, Jehee, et al., 2012), facilitating comparison of this grating to the subsequent second (slightly tilted) grating. In this way, an improved representation of the first grating can improve performance on the orientation discrimination task. To test whether the expectation cues indeed resulted in perceptual benefits, we carried out a behavioral experiment, in which participants performed an orientation discrimination task on gratings that had either an expected or an unexpected orientation. Participants were faster on trials in which the orientation of the first grating was expected than when it was unexpected (mean RT = 612 msec vs. 622 msec, F(1, 22) = 6.2, p = .020). This indicates that a valid expectation of the orientation of the first grating led to a behavioral benefit when discriminating this grating from the second one, although the direction of rotation of the second grating (clockwise or anticlockwise) was wholly unpredictable. There was no significant difference in accuracy between the two expectation conditions (79.9% vs. 80.1%, F(1, 22) = 0.06, p = .81). There were no effects of grating orientation per se on task performance (RT: F(1, 22) = 0.2, p = .63; accuracy: F(1, 22) = 0.02, p = .90). During the behavioral session, orientation differences between the two gratings were determined by a staircase procedure (see Methods) and resulted in a final mean angle difference of 3.7° ± 0.8° (mean ± SE). These final angle differences were used for all trials during the fMRI session. Accuracy during the fMRI session was similar to that during the behavioral session (mean = 80.1%, SE =1.8%), although RTs were slightly longer (mean = 656 msec, SE = 19 msec). There was no significant difference in Task Accuracy (79.3% vs. 81.0%, t(22) = 1.24, p = .23) or RT (655 msec vs. 657 msec, t(22) = 0.42, p = .68) between the two orientations. Effects of expectation on task performance could not be probed during the fMRI session, because “unexpected” trials consisted of omitted gratings, rather than gratings with opposite orientation (Figure 1B, see Methods).

fMRI Results

To probe representational content of the BOLD signal in visual cortex (Figure 2A, B), we estimated the BOLD response evoked in voxels preferring 45° and 135° separately, on the basis of an independent localizer data set (see Methods). As expected, there was a significant interaction between stimulus orientation and voxel orientation preference in V1, F(1, 22) = 18.97, p < .001, demonstrating that neural activity in V1 was orientation specific (Kamitani & Tong, 2005). There was no overall stronger response for either of the two grating orientations, F(1, 22) = 0.39, p > .50, and no overall stronger response in either the 45° or 135° preferring voxels, F(1, 22) < 0.1, p > .50 (Figure 2B). Next, we turned to the omission trials. If the omission-induced activity represents an unspecific surprise signal, we would not expect a significant interaction between the expected orientation and voxel orientation preference. On the other hand, if the omission-induced activity represents the prior expectation of the visual stimulus, induced by the auditory cue, we would expect such an interaction between the expected orientation and voxel orientation preference. This is indeed what we found in V1, F(1, 22) = 7.28, p = .013 (Figure 2B). This indicates that expectations about upcoming visual stimuli, induced by an auditory cue, evoked patterns of activity in primary visual cortex with similar feature specificity as those evoked by actual stimuli, despite the absence of any visual stimulation. As for the stimulus trials, there were no main effects of stimulus orientation, F(1, 22) < 0.1, p > .50, and voxel orientation preference, F(1, 22) = 0.20, p > .50 (Figure 2B), ruling out an explanation in terms of nonspecific effects of arousal.

In V2, we also found a significant interaction between orientation and voxel orientation preference for both the stimulus, F(1, 22) = 11.05, p = .0031, and the omission trials, F(1, 22) = 5.03, p = .035, but neither effect was significant in V3 (stimuli: F(1, 22) = 2.46, p = .13; omissions: F(1, 22) = 1.05, p = .32).

The difference in BOLD response between voxels preferring the currently presented or expected orientation and those preferring the orthogonal orientation (see Figure 2B) was largely independent of the number of voxels selected for the analysis (Figure 3), speaking to the robustness of the effects.

Figure 3. 

Orientation-specific BOLD signal in V1 for different numbers of selected voxels. Stimuli (green) and omissions (red) evoke stronger activity in voxels that prefer the currently presented or expected orientation (solid lines) than in voxels that prefer the orthogonal orientation (dashed lines) over a wide range of selected voxels. Error bars indicate SEM.

Figure 3. 

Orientation-specific BOLD signal in V1 for different numbers of selected voxels. Stimuli (green) and omissions (red) evoke stronger activity in voxels that prefer the currently presented or expected orientation (solid lines) than in voxels that prefer the orthogonal orientation (dashed lines) over a wide range of selected voxels. Error bars indicate SEM.

Finally, we probed the retinotopic specificity of the neural response to unexpected omissions. To this end, we selected separate sets of voxels that did or did not correspond to the retinotopic locations at which the grating stimuli were presented and expected (see Methods). We found that, although there was a significant positive response to both stimuli and omissions in voxels corresponding to the retinotopic grating locations in V1 (stimuli: t(22) = 8.90, p < .001; omissions: t(22) = 5.73, p < .001), V2 (stimuli: t(22) = 10.00, p < .001; omissions: t(22) = 6.46, p < .001), and V3 (stimuli: t(22) = 9.81, p < .001; omissions: t(22) = 6.52, p < .001), there were no significant responses to either stimuli or omissions in voxels that corresponded to retinotopic locations at which gratings were not expected (and presented) in V1 (stimuli: t(22) = 1.00, p = .33; omissions: t(22) = 0.99, p = .33), V2 (stimuli: t(22) < 0.1, p < .94; omissions: t(22) = 0.62, p = .54), and V3 (stimuli: t(22) = 0.18, p = .86; omissions: t(22) = 0.76, p < .45; Figure 4). This suggests that the neural response evoked by unexpected omissions was retinotopically specific.

Figure 4. 

Retinotopic specificity of the BOLD signal in V1. Amplitude of BOLD response evoked by stimuli (green bars) and omissions (red bars), separately for voxels corresponding to the retinotopic location at which gratings are presented (and expected) during the experiment (Grating location) and voxels that correspond to other retinotopic locations (Nongrating location). Error bars indicate SEM (***p < .001).

Figure 4. 

Retinotopic specificity of the BOLD signal in V1. Amplitude of BOLD response evoked by stimuli (green bars) and omissions (red bars), separately for voxels corresponding to the retinotopic location at which gratings are presented (and expected) during the experiment (Grating location) and voxels that correspond to other retinotopic locations (Nongrating location). Error bars indicate SEM (***p < .001).

DISCUSSION

Prior expectations affect sensory processing at the earliest stages of the cortical hierarchy (Kok, Rahnev, et al., 2012; Wyart, Nobre, & Summerfield, 2012; Arnal et al., 2011; Todorovic et al., 2011; Alink et al., 2010; Arnal, Morillon, Kell, & Giraud, 2009; Den Ouden et al., 2009; Summerfield et al., 2008), yet the neural mechanisms underlying the instantiation of the expectations themselves have remained elusive. Here, we showed that prior expectation of a specific visual stimulus evokes a pattern of activity in the primary visual cortex (V1) with similar feature specificity as that evoked by the expected stimulus, even when the stimulus itself is omitted.

The separation of voxels into populations preferring 45° and 135° gratings relied on activity patterns obtained from an independent functional localizer during which participants performed a task at fixation, rendering explanations of our results in terms of eye movements or task-related components such as attention or arousal unlikely. Specifically, because the grating stimuli were task-irrelevant during the localizer run, activity patterns in early visual cortex are not confounded with potential orientation-specific differences in task performance. During the main experiment, there were no orientation-specific differences in behavioral task performance or overall BOLD amplitude evoked by the two orientations. Finally, the design of the analysis avoided potentially confounding effects of the auditory cues, because no such cues were presented during the functional localizer run.

Previous studies (Esterman & Yantis, 2010; Puri, Wojciulik, & Ranganath, 2009) have shown that manipulating the likelihood (expectation) of an upcoming stimulus category (i.e., faces vs. houses) leads to an upregulation of activity in areas of visual cortex selective for the expected category (FFA vs. PPA). Other studies have shown that preparing for a specific stimulus category (people vs. cars; Peelen & Kastner, 2011) or exemplar (letter “X” vs. “O”; Stokes, Thompson, Nobre, et al., 2009) leads to distinguishable patterns of activity in higher-order visual cortex (object-selective and shape-selective areas, respectively). These results suggest that the locus of preparatory signals is determined by the features to be attended or expected. Until now, it was unknown whether such effects could occur in the early visual cortex, given that top–down effects are generally far more prominent in higher-order visual cortex (Lee et al., 2012; Buffalo, Fries, Landman, Liang, & Desimone, 2010; Mohr, Linder, Linden, Kaiser, & Sireteanu, 2009; Johnson, Mitchell, Raye, D'Esposito, & Johnson, 2007; Kastner, De Weerd, Desimone, & Ungerleider, 1998). Indeed, with the exception of Peelen and Kastner (2011), none of the studies mentioned above uncovered preparatory signals in V1, and in Peelen and Kastner (2011), the preparatory signal in V1 was in fact anticorrelated with detection performance (of people or cars). In this study, expectation pertained to the low-level features of the stimulus: Participants knew exactly where to expect the stimulus, and the cue informed them of the likely grating orientation. These low-level features (orientation, location, size) map well onto the receptive field properties of V1 neurons, making V1 an a priori likely locus for the stimulus templates uncovered by this study. In general, it seems likely that expectation affects cortical areas whose receptive fields match the expected features. The fact that participants knew not only where but also when (750 msec postcue) stimuli would appear may have boosted the effects of expectation further, analogous to the synergistic interaction reported between temporal and spatial expectation (Morillon & Barbot, 2013; Doherty, Rao, Mesulam, & Nobre, 2005).

Given that there was no explicit baseline condition included in our study (e.g., “expected omissions”), the current data cannot distinguish whether expectation enhanced the baseline activity of V1 neurons preferring the expected orientation or inhibited the activity of neurons preferring the nonexpected orientation. However, previous studies (Kok, Rahnev, et al., 2012; Todorovic et al., 2011; Wacongne et al., 2011; Den Ouden et al., 2009) have reported increased activity in sensory cortex for unexpected omissions, compared with expected omissions, arguing for an excitatory effect of expectation in the absence of bottom–up input. Additionally, modeling work has shown that expectation amplifies the baseline response of signal-selective units (Wyart et al., 2012). Together, these results suggest that expectation evokes stimulus templates in visual cortex through top–down excitation of neurons preferring the expected orientation. Note that such a prestimulus increase in neurons representing the expected stimulus may give them an advantage in the competition with units preferring other orientations once the bottom–up stimulus input arrives, leading to a relative suppression of neurons preferring nonexpected orientations. Such a mechanism could yield representational sharpening (reduced overall amplitude, but increased stimulus information) of expected stimuli, in line with predictive coding theories (Friston, 2005; Rao & Ballard, 1999), and recently demonstrated empirically (Kok, Jehee, et al., 2012; see below for further discussion).

The mechanisms through which prior expectations evoked stimulus templates in the current study may be similar to those of feature-based attention. In previous studies, feature-based attention has been shown to modulate activity in early visual cortex in a feature-specific fashion (Kamitani & Tong, 2005), an effect that spreads across the visual field to nonstimulated regions of cortex (Serences & Boynton, 2007). However, the current study differs from these previous studies in several respects. First, in the studies by Serences and Boynton (2007) and Kamitani and Tong (2005), a stimulus was always presented on the screen. Spreading of effects of feature-based attention to the nonstimulated hemisphere (Serences & Boynton, 2007) may be mediated by callosal connections between hemispheres (Kennedy & Dehay, 1988). In other words, unlike this study, these studies cannot distinguish between top–down modulation and top–down driving of activity in V1. Second, in this study, the expectation cue (pertaining to the likely orientation of the first grating) was orthogonal to the response on the task (pertaining to the direction the second grating was tilted in). In other words, participants could have ignored the expectation cues and still perform the task successfully. Therefore, our manipulation of prior expectation is more implicit than in studies explicitly manipulating participants' task (e.g., “detect cars”). The fact that expectation resulted in behavioral improvement and detectable stimulus templates in visual cortex suggests that preparatory activity can be induced by factors more implicit than directly changing the task set. Future studies will be needed to investigate whether expectation results in the instantiation of stimulus templates even when the expected feature is fully task irrelevant, in line with findings of task-irrelevant expectation effects on stimulus-evoked activity (Alink et al., 2010; Den Ouden et al., 2009). Third, the neural response to unexpected omissions was retinotopically specific (Figure 4), whereas the effects of feature-based attention have been shown to spread across the visual field (Serences & Boynton, 2007). It should be noted, however, that the current data did not allow us to assess the orientation specificity of activity for retinotopic locations at which no gratings were presented or expected, because the functional localizer contained only gratings overlapping with the locations where stimuli were presented in the experiment. Future research may shed further light on this interesting issue.

One may wonder how the effects of expectation reported here relate to those of mental imagery. Mental imagery has been shown to modulate both perception (Winawer, Huk, & Boroditsky, 2010; Pearson, Clifford, & Tong, 2008; Finke & Schmidt, 1977) and activity in sensory cortex (Albers et al., 2013; Lee et al., 2012; Mohr et al., 2009) in a feature-specific way, and this study shows that expectation does so as well. Therefore, it seems plausible that explicit mental imagery and implicit perceptual expectations result in similar sensory templates being set up in visual cortex. An exciting avenue for further research would be to directly compare the effects of prior expectation and visual imagery on subsequent stimulus-evoked activity.

Overall, one intriguing explanation of our results is that expectation leads to a stimulus template being set up in visual cortex before the stimulus is presented, allowing subsequent input matching this template to be processed efficiently. In a recent study with a very similar design, we showed that a valid expectation (i.e., matching templates) leads to reduced overall activity but improved stimulus representation (Kok, Jehee, et al., 2012), presumably reflecting a sharpened population response. Traditional models of attention would rather predict a boosted stimulus response as a result of attention (Sylvester, Shulman, Jack, & Corbetta, 2009; Maunsell & Treue, 2006; Mangun & Hillyard, 1991), suggesting that expectation and attention may have separable effects (Kok, Rahnev, et al., 2012; Summerfield & Egner, 2009). Previous studies on predictive stimulus templates have reported effects in the medial frontal cortex, suggesting that these regions may be the source of the templates evoked in early visual cortex reported here (Bar et al., 2006; Summerfield et al., 2006). Alternatively, the patterns we observed may have been instantiated only after it had become apparent that the expected stimulus would not appear, signaling the mismatch between expected and actual outcome (i.e., prediction error; Arnal & Giraud, 2012; Den Ouden, Kok, & De Lange, 2012; Summerfield & Koechlin, 2008; Friston, 2005; Rao & Ballard, 1999). The stimulus templates uncovered in the current study are compatible with either “prediction” or “prediction error” signals, because the temporal resolution of the BOLD signal does not allow us to distinguish between pre- and poststimulus signals.

Electrophysiological studies in humans have shown increased responses to unexpected omissions compared with expected omissions in auditory cortex as early as 100–150 msec after a tone was expected (SanMiguel, Saupe, & Schröger, 2013; SanMiguel, Widmann, et al., 2013; Todorovic et al., 2011; Wacongne et al., 2011). These results cannot distinguish whether the activity is related to the prediction itself or the prediction error, because prediction error responses to actually presented stimuli have also been observed as early as 100 msec poststimulus (Todorovic & De Lange, 2012; Todorovic et al., 2011; Wacongne et al., 2011). Interestingly, neurons in the anterior temporal cortex of macaques have been shown to fire in anticipation of their preferred stimulus during paired-association tasks (Meyer & Olson, 2011; Erickson & Desimone, 1999; Sakai & Miyashita, 1991). Furthermore, electrophysiological studies in monkeys (Xu, Jiang, Poo, & Dan, 2012) and humans (De Lange, Rahnev, Donner, & Lau, 2013) suggest that expectation leads to prestimulus activity changes in early visual cortex. Together, these results hint at the possibility of forming predictive stimulus templates in sensory cortex before stimulus onset.

In summary, the current study shows that prior expectation of a specific visual stimulus evokes a feature-specific pattern of activity in the primary visual cortex that correlates positively with that evoked by the actual stimulus, possibly reflecting the formation of a stimulus template to efficiently process expected sensory inputs (Kok, Jehee, et al., 2012).

Acknowledgments

This study was supported by the Netherlands Organisation for Scientific Research (NWO VENI 451-09-001 awarded to F. P. d. L.).

Reprint requests should be sent to Peter Kok, Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands, or via e-mail: p.kok@donders.ru.nl.

REFERENCES

REFERENCES
Albers
,
A. M.
,
Kok
,
P.
,
Toni
,
I.
,
Dijkerman
,
H. C.
, &
De Lange
,
F. P.
(
2013
).
Shared representations for working memory and mental imagery in early visual cortex.
Current Biology
,
23
,
1427
1431
.
Alink
,
A.
,
Schwiedrzik
,
C. M.
,
Kohler
,
A.
,
Singer
,
W.
, &
Muckli
,
L.
(
2010
).
Stimulus predictability reduces responses in primary visual cortex.
Journal of Neuroscience
,
30
,
2960
2966
.
Arnal
,
L. H.
, &
Giraud
,
A.
(
2012
).
Cortical oscillations and sensory predictions.
Trends in Cognitive Sciences
,
16
,
390
398
.
Arnal
,
L. H.
,
Morillon
,
B.
,
Kell
,
C. A.
, &
Giraud
,
A.
(
2009
).
Dual neural routing of visual facilitation in speech processing.
Journal of Neuroscience
,
29
,
13445
13453
.
Arnal
,
L. H.
,
Wyart
,
V.
, &
Giraud
,
A.
(
2011
).
Transitions in neural oscillations reflect prediction errors generated in audiovisual speech.
Nature Neuroscience
,
14
,
797
801
.
Bar
,
M.
(
2004
).
Visual objects in context.
Nature Reviews Neuroscience
,
5
,
617
629
.
Bar
,
M.
,
Kassam
,
K. S.
,
Ghuman
,
A. S.
,
Boshyan
,
J.
,
Schmid
,
A. M.
,
Dale
,
A. M.
,
et al
(
2006
).
Top-down facilitation of visual recognition.
Proceedings of the National Academy of Sciences, U.S.A.
,
103
,
449
454
.
Brainard
,
D. H.
(
1997
).
The Psychophysics Toolbox.
Spatial Vision
,
10
,
433
436
.
Buffalo
,
E. A.
,
Fries
,
P.
,
Landman
,
R.
,
Liang
,
H.
, &
Desimone
,
R.
(
2010
).
A backward progression of attentional effects in the ventral stream.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
361
365
.
De Lange
,
F. P.
,
Rahnev
,
D. A.
,
Donner
,
T. H.
, &
Lau
,
H.
(
2013
).
Prestimulus oscillatory activity over motor cortex reflects perceptual expectations.
Journal of Neuroscience
,
33
,
1400
1410
.
Den Ouden
,
H. E. M.
,
Friston
,
K. J.
,
Daw
,
N. D.
,
McIntosh
,
A. R.
, &
Stephan
,
K. E.
(
2009
).
A dual role for prediction error in associative learning.
Cerebral Cortex
,
19
,
1175
1185
.
Den Ouden
,
H. E. M.
,
Kok
,
P.
, &
De Lange
,
F. P.
(
2012
).
How prediction errors shape perception, attention, and motivation.
Frontiers in Psychology
,
3
,
548
.
Doherty
,
J. R.
,
Rao
,
A.
,
Mesulam
,
M. M.
, &
Nobre
,
A. C.
(
2005
).
Synergistic effect of combined temporal and spatial expectations on visual attention.
Journal of Neuroscience
,
25
,
8259
8266
.
Erickson
,
C. A.
, &
Desimone
,
R.
(
1999
).
Responses of macaque perirhinal neurons during and after visual stimulus association learning.
Journal of Neuroscience
,
19
,
10404
10416
.
Esterman
,
M.
, &
Yantis
,
S.
(
2010
).
Perceptual expectation evokes category-selective cortical activity.
Cerebral Cortex
,
20
,
1245
1253
.
Finke
,
R. A.
, &
Schmidt
,
M. J.
(
1977
).
Orientation-specific color aftereffects following imagination.
Journal of Experimental Psychology: Human Perception and Performance
,
3
,
599
606
.
Fiser
,
J.
,
Berkes
,
P.
,
Orbán
,
G.
, &
Lengyel
,
M.
(
2010
).
Statistically optimal perception and learning: From behavior to neural representations.
Trends in Cognitive Sciences
,
14
,
119
130
.
Friston
,
K. J.
(
2005
).
A theory of cortical responses.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
360
,
815
836
.
Harrison
,
S. A.
, &
Tong
,
F.
(
2009
).
Decoding reveals the contents of visual working memory in early visual areas.
Nature
,
458
,
632
635
.
Haxby
,
J. V.
,
Gobbini
,
M. I.
,
Furey
,
M. L.
,
Ishai
,
A.
,
Schouten
,
J. L.
, &
Pietrini
,
P.
(
2001
).
Distributed and overlapping representations of faces and objects in ventral temporal cortex.
Science
,
293
,
2425
2430
.
Helmholtz
,
H.
(
1867
).
Handbuch der physiologischen optic.
Leipzig
:
L. Voss
.
Johnson
,
M. R.
,
Mitchell
,
K. J.
,
Raye
,
C. L.
,
D'Esposito
,
M.
, &
Johnson
,
M. K.
(
2007
).
A brief thought can modulate activity in extrastriate visual areas: Top–down effects of refreshing just-seen visual stimuli.
Neuroimage
,
37
,
290
299
.
Kamitani
,
Y.
, &
Tong
,
F.
(
2005
).
Decoding the visual and subjective contents of the human brain.
Nature Neuroscience
,
8
,
679
685
.
Kastner
,
S.
,
De Weerd
,
P.
,
Desimone
,
R.
, &
Ungerleider
,
L. G.
(
1998
).
Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI.
Science
,
282
,
108
111
.
Kennedy
,
H.
, &
Dehay
,
C.
(
1988
).
Functional implications of the anatomical organization of the callosal projections of visual areas V1 and V2 in the macaque monkey.
Behavioural Brain Research
,
29
,
225
236
.
Kok
,
P.
,
Brouwer
,
G. J.
,
Van Gerven
,
M. A. J.
, &
De Lange
,
F. P.
(
2013
).
Prior expectations bias sensory representations in visual cortex.
Journal of Neuroscience
,
33
,
16275
16284
.
Kok
,
P.
,
Jehee
,
J. F. M.
, &
De Lange
,
F. P.
(
2012
).
Less is more: Expectation sharpens representations in the primary visual cortex.
Neuron
,
75
,
265
270
.
Kok
,
P.
,
Rahnev
,
D.
,
Jehee
,
J. F. M.
,
Lau
,
H. C.
, &
De Lange
,
F. P.
(
2012
).
Attention reverses the effect of prediction silencing sensory signals.
Cerebral Cortex
,
22
,
2197
2206
.
Lee
,
S. H.
,
Kravitz
,
D. J.
, &
Baker
,
C. I.
(
2012
).
Disentangling visual imagery and perception of real-world objects.
Neuroimage
,
59
,
4064
4073
.
Lund
,
T. E.
,
Norgaard
,
M. D.
,
Rostrup
,
E.
,
Rowe
,
J. B.
, &
Paulson
,
O. B.
(
2005
).
Motion or activity: Their role in intra- and inter-subject variation in fMRI.
Neuroimage
,
26
,
960
964
.
Ma
,
W. J.
(
2012
).
Organizing probabilistic models of perception.
Trends in Cognitive Sciences
,
16
,
511
518
.
Mangun
,
G. R.
, &
Hillyard
,
S. A.
(
1991
).
Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming.
Journal of Experimental Psychology
,
17
,
1057
1074
.
Maunsell
,
J. H. R.
, &
Treue
,
S.
(
2006
).
Feature-based attention in visual cortex.
Trends in Neurosciences
,
29
,
317
322
.
Meyer
,
T.
, &
Olson
,
C. R.
(
2011
).
Statistical learning of visual transitions in monkey inferotemporal cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
19401
19406
.
Mohr
,
H. M.
,
Linder
,
N. S.
,
Linden
,
D. E. J.
,
Kaiser
,
J.
, &
Sireteanu
,
R.
(
2009
).
Orientation-specific adaptation to mentally generated lines in human visual cortex.
Neuroimage
,
47
,
384
391
.
Morillon
,
B.
, &
Barbot
,
A.
(
2013
).
Attention in the temporal domain: A phase-coding mechanism controls the gain of sensory processing.
Frontiers in Human Neuroscience
,
7
,
480
.
Mumford
,
D.
(
1992
).
On the computational architecture of the neocortex. II. The role of cortico-cortical loops.
Biological Cybernetics
,
66
,
241
251
.
Pearson
,
J.
,
Clifford
,
C.
, &
Tong
,
F.
(
2008
).
The functional impact of mental imagery on conscious perception.
Current Biology
,
18
,
982
986
.
Peelen
,
M. V.
, &
Kastner
,
S.
(
2011
).
A neural basis for real-world visual search in human occipitotemporal cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
12125
12130
.
Puri
,
A. M.
,
Wojciulik
,
E.
, &
Ranganath
,
C.
(
2009
).
Category expectation modulates baseline and stimulus-evoked activity in human inferotemporal cortex.
Brain Research
,
1301
,
89
99
.
Rao
,
R. P. N.
, &
Ballard
,
D. H.
(
1999
).
Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects.
Nature Neuroscience
,
2
,
79
87
.
Sakai
,
K.
, &
Miyashita
,
Y.
(
1991
).
Neural organization for the long-term memory of paired associates.
Nature
,
354
,
152
155
.
SanMiguel
,
I.
,
Saupe
,
K.
, &
Schröger
,
E.
(
2013
).
I know what is missing here: Electrophysiologial prediction error signals elicited by omissions of predicted “what” but not “when.”
Frontiers in Human Neuroscience
,
7
,
407
.
SanMiguel
,
I.
,
Widmann
,
A.
,
Bendixen
,
A.
,
Trujillo-Barreto
,
N.
, &
Schröger
,
E.
(
2013
).
Hearing silences: Human auditory processing relies on preactivation of sound-specific brain activity patterns.
Journal of Neuroscience
,
33
,
8633
8639
.
Serences
,
J. T.
, &
Boynton
,
G. M.
(
2007
).
Feature-based attentional modulations in the absence of direct visual stimulation.
Neuron
,
55
,
301
312
.
Serences
,
J. T.
,
Ester
,
E. F.
,
Vogel
,
E. K.
, &
Awh
,
E.
(
2009
).
Stimulus-specific delay activity in human primary visual cortex.
Psychological Science: A Journal of the American Psychological Society/APS
,
20
,
207
214
.
Sereno
,
M. I.
,
Dale
,
A. M.
,
Reppas
,
J. B.
,
Kwong
,
K. K.
,
Belliveau
,
J. W.
,
Brady
,
T. J.
,
et al
(
1995
).
Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging.
Science
,
268
,
889
893
.
Stokes
,
M.
,
Thompson
,
R.
,
Cusack
,
R.
, &
Duncan
,
J.
(
2009
).
Top–down activation of shape-specific population codes in visual cortex during mental imagery.
Journal of Neuroscience
,
29
,
1565
1572
.
Stokes
,
M.
,
Thompson
,
R.
,
Nobre
,
A. C.
, &
Duncan
,
J.
(
2009
).
Shape-specific preparatory activity mediates attention to targets in human visual cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
106
,
19569
19574
.
Summerfield
,
C.
, &
Egner
,
T.
(
2009
).
Expectation (and attention) in visual cognition.
Trends in Cognitive Sciences
,
13
,
403
409
.
Summerfield
,
C.
,
Egner
,
T.
,
Greene
,
M.
,
Koechlin
,
E.
,
Mangels
,
J.
, &
Hirsch
,
J.
(
2006
).
Predictive codes for forthcoming perception in the frontal cortex.
Science
,
314
,
1311
1314
.
Summerfield
,
C.
, &
Koechlin
,
E.
(
2008
).
A neural representation of prior information during perceptual inference.
Neuron
,
59
,
336
347
.
Summerfield
,
C.
,
Trittschuh
,
E. H.
,
Monti
,
J. M.
,
Mesulam
,
M. M.
, &
Egner
,
T.
(
2008
).
Neural repetition suppression reflects fulfilled perceptual expectations.
Nature Neuroscience
,
11
,
1004
1006
.
Swallow
,
K. M.
,
Makovski
,
T.
, &
Jiang
,
Y. V.
(
2012
).
Selection of events in time enhances activity throughout early visual cortex.
Journal of Neurophysiology
,
108
,
3239
3252
.
Sylvester
,
C. M.
,
Shulman
,
G. L.
,
Jack
,
A. I.
, &
Corbetta
,
M.
(
2009
).
Anticipatory and stimulus-evoked blood oxygenation level-dependent modulations related to spatial attention reflect a common additive signal.
Journal of Neuroscience
,
29
,
10671
10682
.
Todorovic
,
A.
, &
De Lange
,
F. P.
(
2012
).
Repetition suppression and expectation suppression are dissociable in time in early auditory evoked fields.
Journal of Neuroscience
,
32
,
13389
13395
.
Todorovic
,
A.
,
Van Ede
,
F.
,
Maris
,
E.
, &
De Lange
,
F. P.
(
2011
).
Prior expectation mediates neural adaptation to repeated sounds in the auditory cortex: An MEG study.
Journal of Neuroscience
,
31
,
9118
9123
.
Wacongne
,
C.
,
Labyt
,
E.
,
Van Wassenhove
,
V.
,
Bekinschtein
,
T.
,
Naccache
,
L.
, &
Dehaene
,
S.
(
2011
).
Evidence for a hierarchy of predictions and prediction errors in human cortex.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
20754
20759
.
Watson
,
A. B.
, &
Pelli
,
D. G.
(
1983
).
QUEST: A Bayesian adaptive psychometric method.
Perception & Psychophysics
,
33
,
113
120
.
Winawer
,
J.
,
Huk
,
A. C.
, &
Boroditsky
,
L.
(
2010
).
A motion aftereffect from visual imagery of motion.
Cognition
,
114
,
276
284
.
Wyart
,
V.
,
Nobre
,
A. C.
, &
Summerfield
,
C.
(
2012
).
Dissociable prior influences of signal probability and relevance on visual contrast sensitivity.
Proceedings of the National Academy of Sciences, U.S.A.
,
109
,
3593
3598
.
Xu
,
S.
,
Jiang
,
W.
,
Poo
,
M.
, &
Dan
,
Y.
(
2012
).
Activity recall in a visual cortical ensemble.
Nature Neuroscience
,
15
,
449
455
.
Yaron
,
A.
,
Hershenhoren
,
I.
, &
Nelken
,
I.
(
2012
).
Sensitivity to complex statistical regularities in rat auditory cortex.
Neuron
,
76
,
603
615
.
Yuille
,
A.
, &
Kersten
,
D.
(
2006
).
Vision as Bayesian inference: Analysis by synthesis?
Trends in Cognitive Sciences
,
10
,
301
308
.