Motivationally relevant stimuli benefit from strengthened sensory processing. It is unclear, however, if motivational value of positive and negative valence has similar or dissociable effects on early visual processing. Moreover, whether these perceptual effects are task-specific, stimulus-specific, or more generally feature-based is unknown. In this study, we compared the effects of positive and negative motivational value on early sensory processing using ERPs. We tested the extent to which these effects could generalize to new task contexts and to stimuli sharing common features with the motivationally significant ones. At the behavioral level, stimuli paired with positive incentives were learned faster than stimuli paired with neutral or negative outcomes. The ERP results showed that monetary loss elicited higher neural activity in V1 (at the C1 level) compared with reward, whereas the latter influenced postperceptual processing stages (P300). Importantly, the early loss-related effect generalized to new contexts and to new stimuli with common features, whereas the later reward effects did not spill over to the new context. These results suggest that acquired negative motivational salience can influence early sensory processing by means of plastic changes in feature-based processing in V1.
Reward and punishment play a fundamental role in the survival of individuals and entire species, as they help us navigate in a complex world (Berridge, 2004; Skinner, 1963; Thorndike, 1898). The drive to pursue reward and avoid punishment can profoundly bias perceptual processing, granting additional weight to stimuli characterized by enhanced motivational significance (Anderson, 2013; Sabatinelli, Lang, Keil, & Bradley, 2007; Junghöfer et al., 2006; Pessoa, Kastner, & Ungerleider, 2002). Although the exact locus of these effects has not been established yet, evidence suggests that they can take place at an early processing stage (Gilbert & Li, 2013; Schacht, Adler, Chen, Guo, & Sommer, 2012), possibly through mechanisms of increased visual salience (Hickey, Chelazzi, & Theeuwes, 2010). At the electrophysiological level, the earliest wave of cortical activation following stimulus onset corresponds to the C1, generated in the calcarine fissure (V1; Kelly, Schroeder, & Lalor, 2013; Foxe & Simpson, 2002; Jeffreys & Axford, 1972). Although motivational and emotional effects have been reported at this early processing stage (Morel, Beaucousin, Perrin, & George, 2012; Stolarova, Keil, & Moratti, 2006), effects of reward and punishment on early sensory processing in human V1 have never been compared at the electrophysiological level when presented within the same task. Exploring motivational effects on early visual processing with ERPs is complicated by the fact that stimuli with an intrinsic motivational value (e.g., images of babies or spiders) can hardly be matched along low-level properties. This caveat especially holds for early effects occurring at the level of the C1, as this early striate ERP is sensitive to low-level properties, including size, contrast, and orientation (Rauss, Schwartz, & Pourtois, 2011; Jeffreys & Axford, 1972). An elegant option to overcome this issue can be to use incentives to tag motivational value to originally neutral stimuli.
A vast literature shows that stimuli associated with positive incentives automatically gain advantage in perceptual processing under various conditions: without conscious processing of the visual stimuli and reward contingencies (Seitz, Kim, & Watanabe, 2009), when the stimuli are not salient on a mere physical level (Gottlieb, 2012) or even when deliberately attending reward-related stimulus characteristics is counterproductive (Hickey et al., 2010). Nonetheless, negative events and consequences of actions are thought to be even more powerful than positive ones in shaping various psychological processes, including perception and attention (Baumeister, Bratslavsky, Finkenauer, & Vohs, 2001; Cacioppo & Gardner, 1999); moreover, stimuli with negative motivational value are known to profoundly impact early perceptual processes (Pourtois, Schettino, & Vuilleumier, 2013; Phelps, Ling, & Carrasco, 2006). Hence, the first goal of the current study was to develop a paradigm enabling a direct comparison between early sensory processing of visual stimuli with acquired positive and negative motivational value, excluding any low-level differences between the two categories.
To achieve this goal, we used associative learning (Mitchell, De Houwer, & Lovibond, 2009; Skrandies & Jedynak, 2000; Mackintosh, 1975, 1983) to charge meaningless stimuli with a negative or positive value by systematically pairing them with a specific behavioral outcome, namely, monetary loss or reward. By counterbalancing the pairing between specific stimulus groups and specific behavioral outcomes, we carefully controlled for the contribution of low-level properties and thus were able to isolate the influence of motivation on early sensory processing.
Besides this methodological goal, our main focus was testing the generalization of effects of positive and negative motivational value to new stimuli and task contexts, providing evidence for the generalization of reward and threat associations in primary visual cortex, using the C1 as a measure of early sensory processing. One possibility is that, as soon as the incentive delivery is suspended, any sensory advantage for positive or negative stimuli would dissipate (extinction). Using a classical conditioning approach, Stolarova et al. (2006) previously found evidence for enhanced early sensory processing at the C1 level for negatively conditioned stimuli, compared with neutral ones, which was limited to the acquisition period, and absent during extinction. However, they used classical conditioning to achieve the stimulus/outcome association and provided no contrast with reward or approach-related stimuli (see also Hintze, Junghöfer, & Bruchmann, 2014). It thus remains open whether these early motivational effects that are due to a “passive” activation of the defensive motivational system can also hold for stimuli with a reward value and if acquired through associative learning. Moreover, it is possible that the acquisition of any sensory preference in V1 through active learning survives through the suspension of the reward schedule and even through a change in task demands. This would be compatible with evidence provided in studies on perceptual learning, where the sensitivity of the primary visual cortex to certain features (learned through instructions and not through incentive, though) persists for long periods of time (e.g., Zhang, Li, Song, & Yu, 2015; Bao, Yang, Rios, He, & Engel, 2010; Pourtois, Rauss, Vuilleumier, & Schwartz, 2008). We tested these two opposed hypotheses by adding a second phase after our associative learning task, comprising the same stimuli used in the learning phase, but embedded in a different task (old/new judgment task) and suspending the reward schedule.
Last, we were also interested in the mechanisms underlying sensory prioritization of motivation-related features in V1 and in a potential generalization of this acquired motivational salience to new stimuli. In this study, we use the term “prioritization” in its broader sense, that is, not strictly as a processing advantage of one stimulus over another during competition, but as describing the status of a stimulus or feature that attracts increased processing resources, even when presented in isolation, resulting in a measurable amplification in cortical responses. Presumably, the differential processing of motivationally relevant (i.e., loss or gain) and irrelevant (neutral) stimuli results from changes in visual cortex sensitivity to low-level features that carry stimulus value. If this short-term plasticity in response to a feature is the mechanism underlying sensory advantages for incentive stimuli, it could also allow generalization of the learned value to similar stimuli or to the same stimuli in new contexts. Recent studies have begun to elucidate the mechanisms of generalization in the case of fear learning, suggesting the existence of complex perceptual and emotional processes that enable the cognitive system to efficiently transfer the negative emotional value to new stimuli sharing features with the learned ones (see Onat & Büchel, 2015; Dunsmoor & Murphy, 2014). In a different context, previous ERP studies already showed feature-based perceptual learning at the C1 level in humans (Zhang et al., 2015; Bao et al., 2010; Pourtois et al., 2008). Here, we wanted to bring together these independent lines of research (on feature-based learning in V1 and on learning generalization based on fear), gaining insight into generalization of acquired motivational salience on early perceptual processing in V1. More specifically, we aimed to assess whether loss- and reward-related effects would lead to similar generalization effects.
To address our research questions (generalization across tasks and stimuli of positive and negative association effects in V1), we trained participants to associate meaningless symbols with a specific motivational value (i.e., monetary gain, loss or a neutral value) during a learning phase and measured the C1 elicited by the same stimuli right after learning, when the same task was performed (consolidation phase) as well as subsequently in a totally different task (old/new judgment). We put forward the hypothesis that both gain and loss associations could increase C1 amplitudes compared with neutral stimuli, reflecting an unspecific effect of motivational salience (see Pessoa et al., 2002; Lang, Bradley, & Cuthbert, 1997) on early sensory processing. However, because punishment has usually a stronger impact on motivation than reward (Baumeister et al., 2001; Cacioppo & Gardner, 1999), we surmised that this differential processing in V1 might be exacerbated for loss compared with reward. Moreover, concerning our first question (generalization across task contexts), we expected to observe the strongest effects while the task required stimulus categorization and incentives were delivered (consolidation phase). However, based on the literature on perceptual learning effects in V1, we still expected to observe residual increased early visual responses (C1) to incentive stimuli when moving to a new task context involving the same stimuli (old/new task). With regard to the second question (generalization of the effect to new stimuli), we predicted that because the impact of motivational salience on perceptual learning might influence feature-based processing at the C1 level (Zhang et al., 2015; Bao et al., 2010), a generalization of reward- and loss-related effects to new stimuli could be observed (Dunsmoor & Murphy, 2014), with a possible asymmetry between these two categories. To this aim, for the old/new task, we created three groups of new stimuli based on specific visual features (i.e., stimulus “families”; see Figure 1A), each of which was associated with a specific motivational value in the learning phase. This procedure allowed us to measure the extent to which the motivational value would spill over to new stimuli resembling the old (reinforced) identities.
In addition, we explored the effect of monetary reward versus loss on later ERP components, including the P300, given that earlier ERP work already showed a differential effect of motivational value at this postperceptual level (see Schacht et al., 2012; Hajcak, MacNamara, & Olvet, 2010; Olofsson, Nordin, Sequeira, & Polich, 2008; Schupp, Flaisch, Stockburger, & Junghöfer, 2006). This last research question is aimed at confirming at the electrophysiological level that our motivational manipulation was successful beyond the behavioral effects, influencing a component that is well known for its sensitivity to motivation, but also to voluntary attention allocation in neutral tasks and memory processes (Polich, 2007).
Twenty-four undergraduate students participated in this study (age: M = 22.8 years, range = 19–33 years; one participant did not report their age) after giving written informed consent. All participants had normal or corrected-to-normal vision and reported no history of psychiatric or neurological disorders.
Procedure, Stimuli, and Task
The study consisted of two parts: an associative learning task (composed by a learning and a consolidation phase) and a recognition task, modified after Schacht et al. (2012). The associative learning task was used to assign different motivational values to three categories of visual stimuli. After a 10-min break, in which participants were guided in imagining reliving a neutral experience (see Vanlessen, Rossi, De Raedt, & Pourtois, 2014), participants had to identify the stimuli used in the association task amongst new stimuli (that were never seen before) in a recognition (old/new) task.
Stimuli were selected from three perceptual groups or “families,” characterized by distinguishable visual features (see Figure 1A). In total, four stimuli per family were used in the learning task.1 For each of these 12 stimuli, two types of displays were created (see Figure 1B): The symbol was either presented alone or as a pattern that extended in the periphery of the upper visual field, a type of presentation necessary to elicit a measurable C1. In the single presentation, the symbol was presented alone, immediately above the fixation cross (1° × 1° of visual angle). The pattern consisted of 6 rows and 15 columns of the same element, filling the entire display above the horizontal meridian (35° × 13° of visual angle). Participants were instructed to treat both presentation types of the same stimulus as one single identity. All stimuli were presented in white on a black background.
In the learning session, participants had to associate, starting from guessing, each of the stimuli to one of three response buttons, while keeping their eyes on the fixation cross. All stimuli belonging to the same family were associated with one button and one outcome. Participants were told that some stimuli were coupled with monetary reward, some with losses and still others did not have monetary consequences. Pressing the correct button in response to a reward stimulus resulted in an incentive of 20 cents, whereas only 10 cents were gained with erroneous responses. Only 10 cents were lost when pressing the correct button following a loss stimulus, whereas 20 cents were lost if an incorrect choice was made. In neutral trials (i.e., with stimuli not associated with monetary consequences), feedback indicated the accuracy of the response (i.e., +0 for correct and −0 for incorrect responses). Hence, the feedback contained both the information about the trial type (i.e., reward, loss, or neutral) and the accuracy of the response. The mapping between “perceptual family” and button/outcome was fully counterbalanced. Buttons were also counterbalanced across participants in all phases of the study.
Each trial in the learning phase started with the presentation of a fixation cross for a random interval between 1500 and 2000 msec, followed by a single or pattern stimulus presented for 250 msec. Next, the fixation cross appeared again until a response was made (maximum duration 5000 msec) and continued to be displayed for an additional 500 msec. The trial ended with the presentation of the feedback, which remained on screen for 1500 msec. After feedback offset, a new fixation cross was presented to start the new trial. All 12 stimuli were presented twice in each block (once as single, once as pattern), in a random order, for a total of 24 stimuli per block. After each block, participants were informed about their current money balance. Participants needed to be correct 48 times in the last 50 trials (moving window) before they entered the consolidation phase (all trials of the block in which this criterion was reached were completed to keep the amount of stimuli from each family equal). The consolidation phase consisted of 10 additional blocks of 24 trials of the exact same task, and ERP analyses are performed on recordings of this part (because of its constant length across participants and the fact that the association between stimuli and outcomes had been fully established for all three stimulus groups).
For the old/new recognition task, we selected 80 stimuli per family that were never presented in the learning task (Figure 1B); 40 stimuli of each family were presented as single stimuli and 40 as textures (randomly selected for each participant) and composed the 240 new stimuli. The four “old” stimuli per family used in the learning task were presented 20 times each (half of the times as single stimuli and half as textures) to obtain an equal number of old and new stimuli (480 in total).
During this task, stimuli were presented one by one, and participants were required to decide for each of them whether they were old or new. No feedback was provided during this phase, nor could participants win or lose money. A trial of the old/new judgment task started with a fixation cross that was presented for a random period between 500 and 1000 msec, followed by a stimulus (250 msec) and again a fixation cross (1500 msec) during which participants could respond. This part of the task required a speeded response (maximum 1500 msec).
When the main task was completed, participants received two additional blocks containing 80 trials each to confirm that the first visual evoked potential elicited by the peripheral textures during the main task corresponded to a reliable retinotopic C1 component (see Rossi & Pourtois, 2014). In these localizer blocks, participants were presented pattern stimuli they had never seen before above or below fixation in a random order, under passive viewing conditions.
Electrophysiological Data Recording and Reduction
EEG was continuously recorded from a Biosemi Active Two System with 128 Ag/AgCl electrodes, referenced online to the CMS-DRL electrodes and sampled at 512 Hz. Eye movements were recorded through additional bipolar electrodes placed respectively above and below the left eye and at the outer canthi of both eyes. Data reduction was identical for the consolidation phase, the old/new judgment task, and the localizer blocks and was performed with Brain Vision Analyzer 2.0 (Brain Products GmbH, Munich, Germany).
EEG signals were referenced offline to the linked mastoids; band-pass filters between 0.016 and 70 Hz and a notch filter (50 Hz) were applied. Next, the EEG data were segmented relative to the onset of either single or pattern stimuli (from 200 msec prestimulus to 1000 msec poststimulus onset). Artifacts due to eye blinks were automatically corrected (Gratton, Coles, & Donchin, 1983). A spherical spline procedure was used for interpolating noisy channels (average interpolated channels = 1.5 channels; range = 0–6 channels). The epochs were baseline-corrected using the entire prestimulus interval. Epochs containing residual artifacts were semiautomatically rejected using an absolute voltage criterion of ±100 μV exceeding baseline. Using this procedure, 8.8% of the epochs were excluded from further analysis. To isolate the ERPs elicited by stimuli falling outside the foveal area, we subtracted from each single subject average for the pattern stimuli the corresponding average for the single stimuli.
The C1 was identified, based on the topographical properties of the current data set and its typical distribution, as the most negative peak between 60 and 100 msec poststimulus onset at electrodes A4, A19, A20, and A21. The component amplitude was quantified as the average of a 20-msec interval around the semiautomatically detected most negative peak in the window. For the P300, based on visual inspection of the temporal and spatial distribution of this component, we quantified it as the mean amplitude of the ERPs between 450 and 600 msec at electrodes A4, A19, A20, and A21.
Averages for the pattern stimuli were calculated separately per participant, depending on previously associated Valence (Reward, Neutral, Loss) and in the analysis on the recognition task, Stimulus Type (Old vs. New). For the localizer stimuli, individual averages were calculated per Hemifield (Upper vs. Lower).
All statistical analyses were performed using IBM SPSS statistics (version 22.0; Armonk, NY). Simple slopes of the cumulative accuracy of performance during the learning phase were calculated separately for rewarded, neutral, and punished stimuli for the blocks before criterion was reached. The linear model was significant for all estimations (all participants, all conditions). Therefore, observed slopes (B) were then compared with characterize learning speed and efficiency with a one-way ANOVA with Valence as factor.
Concerning the old/new judgment task, behavioral indices of memory performance were calculated. Hit rates (old stimuli recognized as old) and False Alarm rates (FA, new stimuli incorrectly recognized as old) were used to calculate two parameters of memory performance (Snodgrass & Corwin, 1988): recognition accuracy (Pr = p(HIT) − p(FA)) and response (performance) bias (Pb = p(FA)/p(1 − Pr)). According to Snodgrass and Corwin (1988), recognition accuracy indicates how good the discrimination between old and new items is (higher values indicate better discrimination); Pb values indicate a liberal response tendency when higher than 0.5 (bias to respond “old”); on the contrary, Pb values lower than 0.5 indicate a more conservative style (tendency to respond “new”). Both Pr and Pb were calculated and submitted to analyses of variance with Valence as factor to explore if memory performance was influenced by previous reward associations.
For the C1 and the P300 data, we tested the role of associated valence in combination with task and familiarity. First, we performed an ANOVA with Valence (Reward, Neutral, Loss), Lead (A4, A19, A20, A21), and Task (Categorization, Old/New judgment) on the stimuli that were physically paired with positive, neutral, and negative incentives during the learning phase. Our goal was to test if any effect of valence would extinguish when moving to a context in which participants did not even have to categorize the stimuli based on their response/outcome association but simply recognize them among distractors.
In a second analysis, we excluded task set as factor, comparing only stimuli that had to be recognized as old or new in the old/new judgment task. In this case, the two types of stimuli (old and new) were physically different, but both types were embedded in the same nonreinforced task. This second ANOVA comprised the factors Valence (Reward, Neutral, Loss), Lead (A4, A19, A20, A21), and Stimulus Type (Old, New) and was aimed at exploring if valence effects could generalize, within a new context, to stimuli that were never seen before but shared perceptual features with the old (previously reinforced) ones.
For both behavioral and ERP data, follow-up analyses were conducted with paired-samples t tests, integrated by bootstrapped (1000 samples) 95% confidence intervals of mean differences.
Learning and Consolidation Phase (Figure 2)
All participants learned the stimulus–outcome/button association, reaching the criterion after 4–18 blocks. Learning slopes showed that Reward stimuli were learned faster than Neutral and Loss ones (F(2, 46) = 11.53; p = .0004; ηp2 = .33). In the Reward condition, the slope was significantly steeper (B = .87, SD = .08) than in the Neutral (B = .68, SD = .19; t(23) = 4.13, p = .001; 95% CI [0.10, 0.27]) and Loss condition (B = .75, SD = .13; t(23) = 4.54, p = .001; 95% CI [0.07, 0.17]). Learning rate was comparable for Neutral and Loss stimuli (t(23) = 1.52, p = .14; 95% CI [−0.15, 0.01]). After criterion was reached, accuracy was at ceiling for all conditions (Rew = 99.0%; Neu = 97.9%; Pun = 99.2%).
Old/New Judgment Task
Participants were able to perform the old/new task, despite the fact that they were not informed about the need to memorize the items for later recognition during the learning phase. Recognition probability (Pr) ranged from .75 (SD = .15) for the Reward stimuli to .63 (SD = .23, Neu) and .66 (SD = .20) for the Loss stimuli, with a significant difference across Valence levels (F(2, 46) = 5.40; p = .008; ηp2 = .19). Stimuli previously associated with monetary gains were better recognized, as compared with neutral (t(23) = 3.02, p = .0006; 95% CI [0.04, 0.20]) and loss stimuli (t(23) = 2.79, p = .008; 95% CI [0.04, 0.17]), which did not differ from each other (t(23) = 0.55, p = .60; 95% CI [−0.10, 0.06]). Participants also overall showed a conservative response bias (Pb lower than 0.5), with an effect of Valence (F(2, 46) = 4.00; p = .03; ηp2 = .15). Follow-up two-tailed one-sample t tests against 0.5 (cutoff between conservative and liberal criterion) showed that there was actually no response bias in the Reward condition (Pb Rew = 0.43, SD = 0.22, t(23) = 1.51, p = .13; 95% CI [−1.15, 0.02]), in line with the fact that the probability of recognition was maximal, while participants tended to respond in a conservative way to previously Neutral (Pb = 0.29, t(23) = 4.36, p = .001; 95% CI [−0.29, −0.11]) and Loss stimuli (Pb = 0.24, t(23) = 4.57, p = .002; 95% CI [−0.31, −0.13]).
A clear polarity reversal was evident for stimuli presented in the upper versus lower visual field, with a distribution of this component at parieto-occipital leads (see Figure 3).
Motivational Effects in V1
The comparison of valence effects in two task contexts (one where the stimulus/outcome association was task relevant and incentives were still delivered, the other in which the same stimuli only had to be detected and not categorized) highlighted a main effect of stimulus Valence (F(2, 46) = 5.93; p = .005; ηp2 = .21) across the two contexts (cf. Table 1). Stimuli linked to a monetary loss elicited C1 responses with higher amplitude than neutral (t(23) = 1.57, p = .006; 95% CI [0.65, 2.54]) and reward-associated stimuli (t(23) = 1.61, p = .009; 95% CI [0.49, 2.69]), which did not differ from each other (t(23) = 0.04, p = .93; 95% CI [−1.10, 1.01]). The interaction between Valence and Task was nonsignificant (F(2, 46) = 0.72; p = .49; ηp2 = .03), as well as the three-way interaction among Valence, Task, and Lead (F(6, 138) = 0.74; p = .55; ηp2 = .03).
|Reward||Neutral||Loss||Family 1||Family 2||Family 3|
|Consolidation||−9.47 (3.82)||−9.78 (4.02)||−10.58 (4.21)||−9.77 (3.82)||−9. 80 (4.12)||−10.27 (4.18)|
|Recognition Old||−9.77 (4.45)||−9.53 (4.37)||−11.86 (4.44)||−9.83 (4.11)||−11.35 (4.96)||−9.98 (4.36)|
|Recognition New||−10.81 (4.97)||−9.40 (4.61)||−11.30 (4.38)||−9.88 (3.70)||−11.17 (4.55)||−10.45 (5.66)|
|Reward||Neutral||Loss||Family 1||Family 2||Family 3|
|Consolidation||−9.47 (3.82)||−9.78 (4.02)||−10.58 (4.21)||−9.77 (3.82)||−9. 80 (4.12)||−10.27 (4.18)|
|Recognition Old||−9.77 (4.45)||−9.53 (4.37)||−11.86 (4.44)||−9.83 (4.11)||−11.35 (4.96)||−9.98 (4.36)|
|Recognition New||−10.81 (4.97)||−9.40 (4.61)||−11.30 (4.38)||−9.88 (3.70)||−11.17 (4.55)||−10.45 (5.66)|
Once ascertained that, even in the new context, stimuli previously paired with monetary incentives showed valence effects comparable to stimuli that were currently reinforced, we set out to investigate whether the motivational valence effect for the old stimuli could spill over to new stimuli with shared perceptual features. Therefore, we analyzed the C1 amplitude to all the stimuli in the old/new judgment task, directly contrasting effects of valence for old and new stimuli, bearing in mind that the old stimuli had been paired with incentives, whereas the new ones were never encountered before but were comparable in shape to either reward, loss, or neutral old stimuli. The analysis revealed a main effect of Valence (see Table 1 for descriptive statistics), across old and new stimuli (F(2, 46) = 5.72; p = .006; ηp2 = .20), and no interaction between Valence and Stimulus Type (F(2, 46) = 1.05; p = .36; ηp2 = .04). The amplitude of the C1 was again higher for the perceptual family previously coupled with negative outcomes, regardless if the stimuli had actually been paired with the incentives (old stimuli) or not (new stimuli). This difference was significant with neutral (t(23) = 2.12, p = .006; 95% CI [0.91, 3.44]) as well as reward stimuli (t(23) = 1.29, p = .04; 95% CI [0.16, 2.34]), which did not differ from each other (t(23) = 0.82, p = .23; 95% CI [−2.09, 0.38]). Old and New stimuli also elicited comparable C1s (F(1, 23) = 0.08; p = .78; ηp2 = .004.
Control analysis: C1 analyzed based on perceptual family (see Table 1)
Notwithstanding the full counterbalancing of our perceptual family/incentive association, we tested if the C1s differed between stimuli when perceptual family was used as factor, instead of the associated valence (thus, Perceptual Family was averaged across participants, regardless of the previous association, which differed across participants). This was done to exclude that our C1 results of valence could be due to any residual differences in the brain responses to the physically different stimuli. To do so, we performed a repeated-measure ANOVA (4 Leads × 3 Perceptual Families × Context, including 3 levels: Consolidation, Old and New stimuli). We reasoned that including all stimuli in the analysis (old and new, paired or not with incentives) would be the most stringent approach, maximizing the chance to observe any residual effect of low-level stimulus properties on the C1 amplitude. Results of this analysis helped us exclude that our C1 findings were spurious: Although we observed a clear effect of Lead (F(3, 69) = 39.90; p < .001; ηp2 = .63), compatible with the analyses based on Valence, no effect of Perceptual Family was present, neither as a main effect (F(2, 46) = 1.68; p = .20; ηp2 = .07) nor in interaction with other factors (all Fs < 1).
Late Effects: P300 (Figure 5)
Role of task context
Concerning the role of task in the processing of stimuli with associated valence (contrasting brain responses to identical stimuli when in one task participants had to categorize them based on their response/outcome association and in the other case simply recognize them among new distractors), the analysis showed a main effect of Valence (F(2, 46) = 8.61; p = .001; ηp2 = .27), further qualified by a Valence × Task interaction (F(2, 46) = 5.94; p = .005; ηp2 = .21). This interaction was justified by the presence of a (quadratic) effect of valence during the categorization task (F(2, 46) = 12.12; p < .005; ηp2 = .35, see Figure 5A) but absence of a valence effect when the same stimuli were embedded in the old/new judgment task (F(2, 46) = 2.03; p = .14; ηp2 = .08; Figure 5B). Reward- and loss-associated stimuli both elicited stronger P300 responses as compared with neutral ones (t(23) = 5.10; p < .005; t(23) = 3.05; p = .006), when the task required an active learning of the stimulus/response/outcome association. However, the pattern was different when the task required simple recognition: The relationship was linear, with a gradual decrease of the P300 from (previously) rewarded to neutral to loss stimuli (the only contrast that approached significance was reward vs. loss: t(23) = 1.85; p = .077).
Role of stimulus familiarity
When comparing old and new stimuli, we observed a classical old/new effect on the P300 amplitude (old stimuli elicited a stronger parietal positivity as compared with new ones, F(1, 23) = 83.30; p < .001; ηp2 = .78; see Figure 5B vs. C) and confirmed the complete lack of an effect of Valence at this processing stage (F(2, 46) = 1.13; p = .33; ηp2 = .05). The interaction between stimulus novelty and valence was also not significant (F(2, 46) = 1.14; p = .33; ηp2 = .05).
P300 amplitude during learning
In addition, we explored our data analyzing the P300 amplitude in the initial learning phase (interindividually different amount of trials), thus between the beginning of the task and the moment when participants reached the criterion, following the same procedure as for the analysis of the consolidation phase. The repeated-measures ANOVA on the waveforms calculated for this interval also showed a main effect of stimulus Valence (F(2, 46) = 4.52, p = .016), with both Reward and Loss trials eliciting a stronger P300 as compared with the Neutral ones (t(23) = 2.61, p = .02 and t(23) = 2.24, p = .04 respectively), indicating that already during learning the increased behavioral significance of motivationally relevant stimuli was reflected in an increased P300 amplitude.
The rapid prioritization of motivationally relevant visual information lies at the basis of goal-adaptive behavior. In new environments, an individual must rapidly learn which stimuli should be approached or avoided and must generalize this motivational value to unknown stimuli that share low-level properties with the ones previously reinforced. The question if this process can be carried out with the coarse information available in V1 is at the core of this study. Here, we employed an associative learning paradigm to investigate changes in early sensory processing for meaningless stimuli that acquired a positive or negative motivational value in contrast to neutral stimuli. We also considered later postperceptual stages of processing (at the P300 level) that reflect top–down attentional control and are sensitive to effects of motivation (Schacht et al., 2012; Sato et al., 2005). Our main questions regarded the possible generalization of motivational value to perceptually similar stimuli and across different contexts to assess whether these motivational effects in visual cortex were stimulus-specific or instead feature-based. If they were stimulus-specific, we reasoned that they could spill over to a new context but should be restricted to the exact same stimuli. Alternatively, if they were feature-based, they should show generalization to new stimuli sharing common features with the learned ones. Therefore, participants were asked to recognize the learned stimuli among new ones with shared low-level properties after (implicitly) learning that stimuli with certain perceptual features were predictive of monetary rewards or losses.
Results showed that participants learned to recognize the stimuli associated with reward faster compared with neutral stimuli during the learning phase, and this was reflected by enhanced P300 amplitudes for rewarded stimuli, both while learning was taking place and after it had been established, but the reward schedule was still in place. The behavioral advantage for gain-associated stimuli persisted in the subsequent old/new recognition task, where they were recognized more successfully as compared with neutral ones. This result replicates earlier findings showing that reward biases sensory processing (and memory) and shifts behavior towards the optimal response criterion (Navalpakkam, Koch, & Perona, 2009; Seitz et al., 2009). Moreover, in our sample, stimuli previously associated with a monetary loss yielded stronger C1 responses compared with neutral and reward-associated stimuli. Additionally, the C1 amplitude was in general higher for the whole perceptual family previously associated with negative outcomes, compared with rewarded or neutral families, showing generalization of the effect to the yet unknown stimuli.
Given that the motivational effects on the behavioral level were mainly driven by reward, it is remarkable that only loss-prospect gained automatic priority in V1. More specifically, the earliest electrical responses recorded in visual cortex (C1) differentiated between the neutral and the negative class of stimuli, with the latter eliciting the largest C1 amplitudes. This effect, measurable within 90 msec from stimulus onset, indicates that low-level features predicting negative outcomes are extracted very early in the processing stream, so that threat processing can be maximized, even over reward seeking. Similar effects were previously reported for negative images, as compared with neutral stimuli (Krusemark & Li, 2013), albeit at a later onset, possibly due to the nature and visual characteristics of the stimuli. Our findings extend these data, demonstrating that even when the visual properties of the stimuli are fully controlled their negative predictive value can bias early visual cortex responses as early as 75 msec after stimulus onset.
By contrast, the previously rewarded stimulus categories did not elicit C1 amplitudes that differed from responses to neutral stimuli. Several reasons might account for the lack of V1 sensitivity to stimuli with a positive motivational value. First, the task parameters could account for this unexpected finding, which might be partly due to the presentation of stimuli in isolation (i.e., neutral and rewarded stimuli are not presented simultaneously), whereas most effects of reward on sensory processing have been observed in competitive contexts (e.g., Serences, 2008), where prioritization effects are obviously maximized. Furthermore, given that the recognition task heavily depended on feature-based attention, the influence of reward might have been overshadowed by the effects of voluntary feature-based attention allocation on early visual processing (Lee & Shomstein, 2013; Stanisor, van der Togt, Pennartz, & Roelfsema, 2013) that was presumably set to perform the task.
Second, it is also possible that the use of monetary reinforcers might have played a role in the divergence between learning speed and neural changes in early visual cortex. Primary reinforcements, such as food or painful stimuli, could elicit different effects compared with small monetary rewards/losses in university students. However, substantial evidence suggests that different types of positive incentives activate the same reward circuitry and rely on the same underlying networks, with the ventromedial pFC and striatum as core structures (Clithero & Rangel, 2014; Luking & Barch, 2013). On the basis of a review of numerous studies using a wide array of reward types, Smith and Delgado (2015) recently suggested a general nature of reward processing underlying the different types of reward, independent of it being monetary (Knutson, Fong, Adams, Varner, & Hommer, 2001), pleasurable tastes (Kim, Shimojo, & O'Doherty, 2011; Small, Zatorre, Dagher, Evans, & Jones-Gotman, 2001) and touch (Rolls et al., 2003), or humor (Watson, Matthews, & Allman, 2007). Hence, the use of secondary incentives should not qualitatively change the pattern of results.
Finally, this surprising difference might also be due to the way loss and reward was operationalized in the current study. The action disposition triggered in the punishment trials was more heterogeneous compared with the reward trials, as that participants could limit the losses in the punishment trials by exerting a correct response. As a consequence, we cannot exclude that the effects observed for the loss trials were driven also by an appetitive motivation, in addition to loss avoidance. However, if one could assume a linear increase of appetitive motivation across the trial types (with neutral trials eliciting the least appetitive motivation, followed by the loss trials and finally the reward trials), one would expect such linear increase reflected in the pattern of results as well. This was indeed true for behavioral and postperceptual (P300) data, but not for the C1 amplitude. The observation of a significant increase in sensory processing only for loss trials at this processing level might be suggestive of the aversive aspect of loss trials driving the effect or at least unlocking it. Nonetheless, based on our data, we cannot unequivocally establish whether the combination of loss and appetitive motivation, or rather the loss alone, was pivotal in the effect observed at the C1 level.
Interestingly, in a previous study we observed a similar effect by using a paradigm in which a cue signaled motivational value in such a way that a correct response to a subsequent stimulus yielded reward and an incorrect response a loss (Bayer et al., 2016). In that study, we aimed at investigating differences in motivation between nonreinforced (neutral) compared with incentive trials, only taking into account the motivational salience but not the actual value of the trials. Results revealed that incentive trials elicited larger C1 amplitudes compared with the neutral ones, showing thus an effect of motivation, possibly the resultant of the combination of appetitive and avoidant motivational drives. However, although the paradigm of the previous study did not allow us to disentangle effects of the motivation to avoid losses or to gain rewards, the present results suggest that the earliest stages of processing might be more sensitive to loss-related information, even when not forewarned on the value of the upcoming trial.
Nonetheless, previous studies have shown that early processing in V1 can be influenced by reward prospect, while we could not identify effects of monetary rewards in our sample. This might have to do with methodological differences across our study and the classical animal (or reward prospect) studies. In our task, participants were presented with stimuli without a warning informing on their motivational value, whereas in the animal literature and in most studies in humans a cue predicts the reward value of a certain trial. Reward prospect effects can be observed in V1 in monkeys (as pointed out in Gavornik, Shuler, Loewenstein, Bear, & Shouval, 2009) and other animals (e.g., rats; Shuler & Bear, 2006) when the reward delivery is also contingent upon performance. In our case, on the other hand, reward and punishment were partially independent on behavior (and even suspended in the second task). This might have interfered with the potency of the monetary incentive in installing changes at the neural level in visual cortex, while the negative stimuli still maintained this ability, possibly due to the priority and salience humans seem to attribute to negative stimuli (Baumeister et al., 2001). From an evolutionary point of view, it is conceivable that an early detection and generalization of stimuli associated with negative outcomes is more potent compared with potentially rewarding events (Susskind et al., 2008; Smith, Cacioppo, Larsen, & Chartrand, 2003) because missing such information could jeopardize the survival and well-being of the individual (Carretié, Albert, López-Martín, & Tapia, 2009).
Similarly, at the level of the P300, no effect of valence was found in the recognition task, whereas a substantial increase in amplitude was observed for old compared with new stimuli (“old/new effect”). Given its association with top–down, voluntary attention and goal-directed processes (Nieuwenhuis, Aston-Jones, & Cohen, 2005), it is not surprising that the P300 is not altered by the motivational value of stimuli shown in a previous, but not the current, task context. During the consolidation phase however, when participants were still confronted with monetary rewards and losses, the P300 showed the typical increased amplitude to rewarded stimuli (e.g., Schacht et al., 2012). This P300 effect was mirrored in behavior, as participants showed steeper learning curves for rewarded compared with neutral or punished stimuli. Moreover, the faster learning of rewarded stimuli might in part contribute to the effects of reward on the reported P300 effect, because they might have made the classification less challenging during the subsequent consolidation phase (Rossi & Pourtois, 2014; Polich, 2007).
Our second question concerned the possibility that these effects would generalize to previously unencountered stimuli. Indeed, our data demonstrate that punishment can induce sensitivity for low-level stimulus features in early sensory cortex, as previously shown for high rewards (Stanisor et al., 2013) in a way that new stimuli that share those features will also benefit from enhanced sensory processing in V1. However, our data do not allow to specify the exact stimulus feature(s) for which this plasticity occurred. Future studies should consider exploring if such effects are more likely driven by stimulus orientation, shape, or spatial frequency. In our data, the C1 effect efficiently generalized to new stimuli that shared perceptual features with the old ones, suggesting that activity in early visual cortex can be modulated in a top–down fashion (Gilbert & Li, 2013; Muckli & Petro, 2013). Although category-based visual search has been shown to be slower and less efficient than item-based search (Wu et al., 2013), our data show that categorization of new items based on (motivational) associative learning occurs very early in the stream of processing and automatically draws from priorities set during learning. Such efficient implementation of feature-based attention could be at the base of transferability of motivational value to new materials. More specifically, these results suggest a representation of threat-related features already in early visual cortex, possibly mediating generalization of behavioral responses in new environments (Dunsmoor, Kragel, Martin, & LaBar, 2013). This in turn could be at the foundation of processes such as the generalization of fear (Onat & Büchel, 2015; Dunsmoor, Prince, Murty, Kragel, & LaBar, 2011) and could explain why anxiety maintenance can be sustained by attentional and perceptual biases also involving coarse visual mechanisms (Rossi & Pourtois, 2013).
Our data suggest a differentiation between effects of reward and monetary loss after associative learning. Although previously rewarded stimuli receive enhanced resources at late processing stages (around 400 msec) and in behavior, likely indicating prioritization by voluntary attention (Nieuwenhuis et al., 2005), early visual cortex responses are boosted by the negative predictive value of perceptual features. Acquired motivational value can thus bias very early responses in visual cortex in favor of events predicting negative outcomes, in line with the idea that increased and rapid acquisition of sensory information (Susskind et al., 2008) might be more important in risky situations than to quickly approach reward (Carretié et al., 2009; Baumeister et al., 2001). Hence, threat-related stimuli attain advantage in early perceptual selection, even when they are not associated with losses anymore, in line with a value-driven mechanism of attentional selection (Anderson, 2013). Moreover, associative learning seems to rely on biasing sensory processing toward negative motivational value at an early stage and form durable representations of stimulus–outcome associations that generalize to different contexts to minimize losses and promote survival. This early priority of punishment-predicting information seems to be compensated by a voluntary allocation of resources toward reward at a later stage of processing. In sumnary, the system first ensures coping with threat and redirects subsequently toward increasing rewards.
The authors thank the U4 strategic partnership for the generous financial support. V. R. was supported by the Special Research Funds at Ghent University (grant BOF13/PDO/095). A. S. and M. B. were supported by the German Research Foundation (DFG; grant SCHA1848/1-1). A. S. was further supported by the Leibniz ScienceCampus Primate Cognition (Goettingen). G. P. was supported by the Belgian Science Policy, Interuniversity Attraction Poles program (P7/11), by a Concerted Research Action Grant from Ghent University, and by the FWO (Research Foundation Flanders).
Reprint requests should be sent to Valentina Rossi, Department of Experimental-Clinical and Health Psychology, Ghent University, Henri Dunantlaan 2, 9000 Ghent, Belgium, or via e-mail: email@example.com.
The stimuli consisted of linguistic symbols from the Arabic and Arabic Presentation Form A alphabets (Family 1), the Ethiopic alphabet (Family 2), and the Kannada and Bengali alphabets (Family 3) selected from www.decodeunicode.org/. In a pilot study, 20 naive observers were presented with 630 pairs of stimuli, divided in four blocks. Each pair consisted of two different stimuli, taken from a pool of 12 exemplary members of each family. Participants were asked to score their similarity on a Likert scale (1–6, where 1 = maximum similarity and 6 = maximum dissimilarity). Multidimensional scaling was then used to visualize the perceptual relationship among the different stimuli by placing them along two dimensions (in a Euclidean space), which are not predefined or labeled. The distances between the stimuli in this space represent the subjective dissimilarity as perceived by the participants in this pilot study and confirmed that the stimuli belonging to each family were perceived as more similar to each other compared with members of the other families. Of these 36 stimuli, four items per family were selected as targets for the learning task.
Affective-Motivational Salience and Attentional Sets Special Focus: This paper was presented at the Cognitive Neuroscience Society Meeting, New York City, April 2016.
These two authors contributed equally to this work.