Abstract

Everyday life, as well as psychiatric illness, is replete with examples where appetitive and aversive stimuli hijack the will, leading to maladaptive behavior. Yet the mechanisms underlying this phenomenon are not well understood. Here we investigate how motivational cues influence action tendencies in healthy individuals with a novel paradigm. Behaviorally, we observed that an appetitive cue biased go behavior (making a response), whereas an aversive cue biased no-go behavior (withholding a response). We hypothesized that the origin of this behavioral go/no-go bias occurs at the motor system level. To test this, we used single-pulse TMS as a motor system probe (rather than a disruptive tool) to index motivational biasing. We found that the appetitive cue biased the participants to go more by relatively increasing motor system excitability, and that the aversive cue biased participants to no-go more by relatively decreasing motor system excitability. These results show, first, that maladaptive behaviors arise from motivational cues quickly spilling over into the motor system and biasing behavior even before action selection and, second, that this occurs in opposing directions for appetitive and aversive cues.

INTRODUCTION

Much behavior is controlled by instrumental learning mechanisms that guide us to select action or inaction to maximize reward and minimize punishment. Behavior is also controlled by evolutionarily preprogrammed reflexes (Dayan, Niv, Seymour, & Daw, 2006), by which appetitive cues facilitate appetitive responding (i.e., behavioral activation, approach) whereas aversive cues facilitate aversive responding (i.e., behavioral inhibition, withdrawal). Notably, when motivational cues are present in situations where optimal decision-making is required, “misbehavior of the will” can occur (Geurts, Huys, Den Ouden, & Cools, 2013; Guitart-Masip et al., 2012; Harmon-Jones, Gable, & Price, 2012; Huys et al., 2011a; Dayan et al., 2006; Damasio, 1996; Breland & Breland, 1961). In a famous example from animal research, birds in a “looking glass” environment could not learn to run away from a food source to get access to it (Hershberger, 1986). In that experiment, the food source was an appetitive cue that predisposed approach behavior that counteracted the instrumental requirement to go in the opposite direction to get the food. In humans, many instances of misbehavior of the will arise from “affective/motivational cues” influencing decision-making. Examples range from how the smell of freshly baked pastries influences one to abandon a dieting goal, to the use of background music in shopping malls to encourage purchasing behavior, and to the use of pictures of diseased lungs on cigarette packages to discourage smoking.

We hypothesized that anomalous behavior reflects a motivation-to-motor “spillover” of responses evoked by affective cues even before action selection. To test this spillover hypothesis, we devised a novel instrumental learning paradigm in combination with single-pulse TMS. In this instrumental task, participants learned to go and to no-go with respect to four “stimulus combinations” based on feedback (juice reward or punishment; Figure 1). Each combination had two elements: an affective cue (a picture of sweet apple juice or a picture of bitter hops tea) and a symbol (a triangle or a square); these two elements were presented sequentially, separated by a short delay. Furthermore, each stimulus combination had a unique pattern of time-varying reinforcement contingency (e.g., whether go/no-go gave rise to reward). By separating the cue and symbol, we aimed to “force” participants to wait until the symbol was presented (when both pieces of information, cue + symbol were given) before they could decide whether to go or to no-go. And by using four unique time-varying reinforcement contingencies, we aimed to maximize “model-free” learning systems, which may be most sensitive to affective biases (Balleine & O'Doherty, 2010; Daw, O'Doherty, Dayan, Seymour, & Dolan, 2006).

Figure 1. 

The motivationally biased learning (MBL) task. (A) Each trial started with a motivational cue (either a picture of apple juice or hops tea), followed by a brief delay (Experiment 1: 200 msec, Experiment 2: 1000 msec). In Experiment 2, the TMS pulse was delivered at the midpoint of the delay (500 msec after the motivational cue). After the delay, one of two symbols (triangle or square) was presented for 1000 msec, during which participants decided to go (press a key with the right index finger) or no-go (withhold the response). Participants learned to go/no-go following each stimulus combination by trial and error. In the first trial of this example, the participant made a go response, but it was incorrect, and a drop of hops tea was delivered. In the next trial, the participant made a correct no-go response, and a drop of apple juice was delivered. (B) The four (2 pictures × 2 symbols) stimulus combinations used in the task. (C) Each stimulus combination (motivational cue + symbol) followed a unique probabilistic time-varying policy of go being correct. (D) The setup for Experiment 2. On each trial, a single TMS pulse was delivered at 500 msec after the motivational cue over the left motor cortex and the MEP was recorded from the right index finger. A go response was a lateral abduction of the right index finger. (E) The fractal image used as the “neutral” picture in Experiment 2.

Figure 1. 

The motivationally biased learning (MBL) task. (A) Each trial started with a motivational cue (either a picture of apple juice or hops tea), followed by a brief delay (Experiment 1: 200 msec, Experiment 2: 1000 msec). In Experiment 2, the TMS pulse was delivered at the midpoint of the delay (500 msec after the motivational cue). After the delay, one of two symbols (triangle or square) was presented for 1000 msec, during which participants decided to go (press a key with the right index finger) or no-go (withhold the response). Participants learned to go/no-go following each stimulus combination by trial and error. In the first trial of this example, the participant made a go response, but it was incorrect, and a drop of hops tea was delivered. In the next trial, the participant made a correct no-go response, and a drop of apple juice was delivered. (B) The four (2 pictures × 2 symbols) stimulus combinations used in the task. (C) Each stimulus combination (motivational cue + symbol) followed a unique probabilistic time-varying policy of go being correct. (D) The setup for Experiment 2. On each trial, a single TMS pulse was delivered at 500 msec after the motivational cue over the left motor cortex and the MEP was recorded from the right index finger. A go response was a lateral abduction of the right index finger. (E) The fractal image used as the “neutral” picture in Experiment 2.

In Experiment 1, we examined whether there is an affective influence on behavior. According to the spillover hypothesis, appetitive and aversive cues should exhibit opposing behavioral bias over action tendencies: Specifically, appetitive cues should promote go, and aversive cues should promote no-go (cf. Guitart-Masip et al., 2011, 2012). These behavioral predictions are consistent with the wider literature on automatic processing of affect, including demonstrations of the coupling between affective/social stimuli and behavioral dispositions of approach and avoidance, for example, how pulling a lever toward oneself is faster in the presence of a positive stimulus and how pushing a lever away from oneself is faster in the presence of a negative stimulus (e.g., Rinck & Becker, 2007; Chen & Bargh, 1999; Cacioppo, Priester, & Berntson, 1993). We contrast the “spillover” hypothesis with a different account known as “evaluative coding” (Eder & Rothermund, 2008; Rotteveel & Phaf, 2004). The evaluative coding account posits that affective cues only bias behavior when positive or negative emotions are generated via conscious processing. For example, in Rotteveel and Phaf (2004), the affective influence of behavior only occurred when participants performed emotional (or affective) evaluations of the stimuli, for example, whether a face is happy or angry, but not when they performed gender categorizations (i.e., nonaffective evaluations) on the same stimuli. Notice that, in our paradigm, although the cues are labeled as “affective,” the instrumental task does not require any affective evaluation of the cue (e.g., whether the picture is appetitive/aversive or positive/negative). Participants only need to assess, on a given trial, which of the stimulus combination is relevant for their action selection. Thus, on the assumption that participants in our study are not performing affective evaluations of the cues, the evaluative coding account does not predict any behavioral bias whereas the motivation-to-motor spillover account does.

In Experiment 2, we used single-pulse TMS to measure the corticospinal excitability of the response hand representation during the delay period after the motivational cue onset and before the action selection. The resulting measurement, that is, the motor-evoked potential (MEP), was taken as an index of motivational spillover in the motor system, independent of any action/inaction preparation. According to the spillover hypothesis, appetitive-go bias arises from an increase of motor system excitability after the appetitive cue preceding go, and by contrast, aversive-no-go bias arises from a dampening after the aversive cue preceding no-go. We could test this hypothesis because the affective cue and the symbol instructing action selection were separated in time, and TMS was delivered in the interval between them. Although the influence of affective cues on behavior has recently been studied in humans with fMRI (Geurts et al., 2013; Huys et al., 2011; Bray, Rangel, Shimojo, Balleine, & O'Doherty, 2008; Talmi, Seymour, Dayan, & Dolan, 2008), those studies did not have the temporal resolution to provide a physiological measurement that indexes the affective influence in the absence of other decision processes. Therefore, we aimed to go beyond those fMRI studies and to glean mechanistic insight into how appetitive and aversive stimuli bias behavior by using TMS in Experiment 2.

EXPERIMENT 1

Methods

Participants

Eighteen neurologically healthy volunteers (18–32 years old, mean age = 21 years; nine women; 16 right-handed) from the University of California, San Diego, participated for course credit. None reported taking medications. The experiment was conducted in accordance with a protocol approved by the institutional review board of the University of California, San Diego. All participants provided written informed consent.

Apparatus and Stimuli

We used an iMac with a 21.5-in. monitor and a standard keyboard. Stimulus presentation and behavioral data collection were controlled using custom MATLAB code (The MathWorks, Natick, MA) and the Psychophysics Toolbox 3 (Brainard, 1997). Two types of juice were used in the experiment: standard brand apple juice (appetitive juice) purchased from the store and home-brewed hops tea (aversive juice). Juice was delivered to the participant's mouth through plastic tubes from two reservoirs via a custom-made solenoid valve controlled by MATLAB.

Task and Procedure

The experimenter explained the task verbally and with trial examples. The participant was then seated in front of a computer with two tubes in his or her mouth. In this task, human participants learned, by trial-and-error, to go (press a key) or no-go (withhold pressing a key; Figure 1). Each trial started with a picture of apple juice or hops tea (motivational cue), followed by a short delay (200 msec). After the delay, a symbol (triangle or square) was presented at which point the participant decided to go or no-go by pressing (or not) a designated key within 1000 msec (Figure 1A). Note that, on each trial, it was the “motivational cue + symbol” combination that was important for the action selection. As Figure 1B shows, there were four combinations of 2 motivational cues × 2 symbols. Importantly, the stimulus–response–outcome contingencies were stochastic and varied across time, that is, 20–80% of go or no-go being correct (Figure 1C). Immediately after the symbol offset, the correct action (go/no-go) resulted in delivery of appetitive apple juice (0.15 ml); else delivery of aversive hops tea (0.15 ml) regardless of the nature of the cue (Figure 1D). A key feature of this paradigm was that, on each trial, it was the “motivational cue + symbol” combination that was important for the action selection but not the motivational cue or the symbol alone. This design allowed us to temporally isolate the influence of the motivation cue before action selection (critical for Experiment 2).

The intertrial interval was 1.8–2.5 sec during which the participants swallowed the juice. There were 420 trials. A short break occurred after every 105 trials, and overall performance (number of correct trials, go RTs) was provided as feedback. Before the task, participants were presented with pictures of (appetitive) apple juice and (aversive) hops tea while tasting 1 ml of each.

Statistical Analysis

To analyze the behavioral data, we first binned the probability of “Going being correct” into three bins with equal trial counts: 0–40%, 40–66%, and 66–100%. We then calculated the percentage of go response, the mean go RTs, and the go/no-go errors within each bin. To address the main question of interest, we extracted go errors in the lowest probability bin and no-go errors in the highest probability bin for each type of cue. We analyzed the error rates with a 2 Cue (appetitive, aversive) × 2 Patterns of Error (failed-to-no-go, failed-to-go, see below) ANOVA. Post hoc paired t tests were conducted between two conditions of interest, and we reported one-tailed Bonferroni-corrected p values when testing predicted effects.

Results

To assess whether there was an influence of motivational cue on instrumental learning, we focused on the pattern of errors (i.e., anomalous behavior). Specifically, we computed the proportion of failed-to-no-go trials, that is, Going when the best policy was to no-go (i.e., 66–100% No-going being correct) and failed-to-go trials, that is, No-going when the best policy was to go (i.e., 66–100% Going being correct). There was a significant interaction between pattern of error and cue valence, F(1, 17) = 11.91, p < .01, partial η2 = .45 (Figure 2A). Post hoc t tests showed that when the best policy was to no-go, participants failed-to-no-go more following the appetitive cue than the aversive one (55 ± 2.5% vs. 46 ± 3.4%, t(17) = 3.59, p < .01, Cohen's d = 0.77). By contrast, when the best policy was to go, participants failed-to-go more following the aversive cue than the appetitive one (40 ± 2.1% vs. 34 ± 2.2%, t(17) = 2.57, p < .05, Cohen's d = 0.52).

Figure 2. 

(A–C) Experiment 1: Behavioral study. (A) Pattern of errors: Failed-to-go when Going is more likely (i.e., 66–100% go is correct) versus failed-to-no-go when No-going is more likely (i.e., 0–40% go is correct) to be correct following appetitive and aversive cues. (B) Percentage of go response and (C) RTs plotted for each probability bin. Note that data points for an “ideal observer” in (B) represent the hypothetical performance of an observer who is able to track the probability curves perfectly. Error bars denote SEM.

Figure 2. 

(A–C) Experiment 1: Behavioral study. (A) Pattern of errors: Failed-to-go when Going is more likely (i.e., 66–100% go is correct) versus failed-to-no-go when No-going is more likely (i.e., 0–40% go is correct) to be correct following appetitive and aversive cues. (B) Percentage of go response and (C) RTs plotted for each probability bin. Note that data points for an “ideal observer” in (B) represent the hypothetical performance of an observer who is able to track the probability curves perfectly. Error bars denote SEM.

We note that, although it was the case, across the whole experiment, that the probability of Going being correct was the same for appetitive and aversive cues, participants exhibited a general go bias following appetitive cues compared with aversive cues (59 ± 2.1% vs. 53 ± 2.3%, t(17) = 4.39, p < .01, Cohen's d = 1.04; Figure 2B). Yet, importantly, this did not create a selective advantage for learning to go, as the overall learning of the underlying policy was not different between the two cues (52 ± 1.1% vs. 53 ± .86%, ns). In fact, the go bias was only beneficial in the context where Going was more likely to be correct, but it was detrimental in the opposite context.

The task was designed to induce maladaptive behavior. Therefore, participants were not expected to behave like an “ideal observer” who can perfectly track all four time-varying reinforcement contingencies (e.g., Figure 1C). Nonetheless, participants were expected to show some learning of the contingency, rather than to respond randomly. Indeed, learning is evident from the fact that the participants' go percentage increased monotonically as a function of probability bins (i.e., probability (correct|go); see Figure 2B). To further confirm that response patterns were not random, we first permuted each participant's response sequence 100 times to produce 100 random responding patterns and then calculated the average to derive the mean “chance” performance for each participant (chance: M = 50.6 ± 0.55%). The permutation procedure estimated the chance rate better because it took into account the overall go/no-go ratio within each participant, rather than assuming a fixed 0.5 ratio. We then used a t test to compare the empirical data with the mean chance rate derived from the permuted data. This analysis revealed that participants indeed performed significantly better than chance (t(17) = 2.31, p < .05, Cohen's d = 0.54).

We also found that, when participants did go, RT was faster following appetitive than aversive cues (663 ± 11.2 vs. 681 ± 10.45 msec, t(17) = 4.03, p < .05, Cohen's d = 0.95; Figure 2C). This shows that the appetitive cue induced response vigor.

Discussion

Using a novel motivationally biased learning paradigm, this experiment clearly established opposing behavioral biases on action by appetitive versus aversive cues. We show that participants were biased to go more following appetitive cues, resulting in more errors when the best policy was to no-go and, by contrast, they were biased to no-go more following aversive cues, resulting in more errors when the best policy was to go. As mentioned in the Introduction, these behavioral biasing effects are predicted by the motivation-to-motor spillover hypothesis, but not by the evaluative coding account (Eder & Rothermund, 2008; Rotteveel & Phaf, 2004).

In Experiment 2, we aimed to use single-pulse TMS to gain mechanistic insight into how appetitive and aversive stimuli bias behavior. We delivered TMS over the motor cortex representation of the right index finger between the time of the motivational cue and the symbol and measured the MEP from the right index finger. This separation of affective cue and symbol is an important feature of our experiment as it forces participants to wait until the symbol (when both pieces of information, cue + symbol are given) to either go or no-go. The fact that go percentage increased as a function of probability bins (Figure 2B) shows that participants were learning and generally selecting the right action, which they could only do if they waited for the symbol. If the spillover account is correct, then appetitive versus aversive cues should exhibit differential bias on the motor system even before an action can be selected. Furthermore, to test whether MEPs for appetitive-go and aversive-no-go bias arise from a respective increase of motor system excitability after the appetitive cue preceding go and a respective dampening after the aversive cue preceding no-go, we included a neutral picture condition (a fractal picture; Figure 1E) to serve as a baseline. No juice feedback was provided for the neutral condition, and therefore, it should not come along with any affective value.

EXPERIMENT 2

Methods

Participants

Twenty-two new volunteers (18–24 years old, mean age = 21 years; 10 women; 18 right-handed) participated for momentary compensation ($15/hr). All participants passed a TMS safety screening and reported no history of neurological impairment. None reported taking any medications. The experiment was conducted in accordance with a protocol approved by the institutional review board of the University of California, San Diego. All participants provided written informed consent. Data from two participants were excluded from analysis; one because of noncompletion of the experiment and the other because of excessive muscle tensing during the delay period.

Apparatus and Stimuli

The experiment was run on an iMac with a 17-in. monitor and a numeric keypad was used for response collection. The keypad was positioned vertically to allow for a lateral extension of the right index finger, which was used for responding in the task. This movement is optimal for attaining good EMG signals from the first dorsal interosseous (FDI) muscle. Other aspects of the setup were the same as Experiment 1.

Task and Procedure

Task and procedure were the same as Experiment 1 except for the following: (a) the delay was 1000 msec instead of 200 msec, to better separate influence of the motivational cues from action preparation; (b) one TMS pulse was delivered on every single trial 500 msec after the motivational cue offset; (c) a “neutral” picture (i.e., a fractal image; Figure 1E) condition (140 trials) was included to serve as baseline for MEP normalization; (d) neutral trials also followed a time-varying schedule, however they were followed by verbal feedback (correct/incorrect written on the screen) rather than by appetitive/aversive juice feedback. The interval between two TMS pulses was 5 sec (although this interpulse interval was relatively short, it was sufficient for EMG signals to go back to baseline). Participants in Experiment 2 completed 700 trials (560 trials with affective pictures and 140 neutral/baseline trials). A break was administered after every 100 trials, and overall performance (number of correct trials, go RTs) was provided as feedback.

TMS Procedure and Data Analysis

EMG was recorded from the FDI muscle of the right hand using pairs of 10-mm silver electrodes. A ground electrode was placed at the wrist of the right hand (see Figure 1D). The EMG signal was amplified, filtered with a 30-Hz to 1-kHz band-pass filter and a 60-Hz notch filter (Grass QP511 Quad AC Amplifier System, Glass Technologies, West Warwick, RI) and digitized at a rate of 2 kHz (CED Micro 1401 mk II acquisition system, Cambridge, UK).

Recording of the EMG sweep started simultaneously with fixation onset and continued for 2 sec. A Magstim 2002 system (Magstim Company, Whitland, Dyfed, United Kingdom) was used to deliver TMS pulses via a figure-eight coil. To find the location (“hot spot”) for eliciting the best MEPs in the right FDI muscle in left primary cortex, the coil was initially placed at a point 5 cm lateral and 2 cm anterior to the vertex approximately 45° to the midsagittal line (Figure 1D). The coil was incrementally repositioned while administering single TMS pulse to locate the position that produced reliable MEPs in the right FDI when participants were at rest. The “hot spot” was marked on a swim cap worn by the participant to ensure consistent coil placement throughout the experiment. The direction of the induced current in the coil was posterior to anterior.

After the hot spot was located, the resting motor threshold for the FDI muscle was determined to the nearest 1% of stimulator output and defined as the lowest stimulus intensity required to elicit MEPs with peak-to-peak amplitude greater than 50 μV in at least 5 of 10 consecutive trials (Rossini et al., 1994). Next, the maximum MEP amplitude was determined by increasing stimulus intensity in 5% increments until the MEP amplitude no longer increased or reached the system maximum (2 mV). After reaching the maximum, the intensity was adjusted in 2% decrements to produce an MEP that was approximately half of the maximum amplitude. This intensity was used during the whole experiment. The mean resting motor threshold was 45% (SEM = 1.25, range = 33–53%), and the mean experimental intensity was 49% (SEM = 1.46, range = 37–60%).

For each TMS trial, the peak-to-peak amplitude of the MEP was calculated and was used in the statistical analysis. To normalize data across participants, the MEP in each condition was divided by the overall mean of MEP in the neutral picture conditions. Normalized MEPs were used in the group level statistical analyses. The raw MEP data for each condition are reported in Table 1. Trials were rejected if the maximum EMG trace during the pretrial epoch (50–0 msec before TMS onset) exceeded 50 μV, because these might reflect tensing up of the muscles (mean number of trials rejected = 3.3 ± 1.1) for task-irrelevant reasons. To verify that the FDI muscle was equally at rest across conditions, the root mean square of the EMG trace from 50 to 0 msec before TMS onset was calculated and analyzed. The mean root mean square was 11.5 μV (SEM = 0.8), with no difference across conditions (ps > .05, all partial η2 < .09), suggesting that there was no differential muscle activation across experimental conditions before the TMS pulse.

Table 1. 

Raw Means and Standard Error for MEPs in Experiment 2

Participant Response
Hops Tea
Neutral
Apple Juice
No-go 0.500 (0.038) 0.513 (0.042) 0.518 (0.036) 
Go 0.528 (0.038) 0.534 (0.036) 0.541 (0.037) 
Participant Response
Hops Tea
Neutral
Apple Juice
No-go 0.500 (0.038) 0.513 (0.042) 0.518 (0.036) 
Go 0.528 (0.038) 0.534 (0.036) 0.541 (0.037) 

Statistical Analysis

Behavioral analyses were similar to those of Experiment 1. For MEP data, we performed a 2 Cue (appetitive, aversive) × 2 Action (go, no-go) ANOVA. Post hoc paired t tests were conducted between two conditions of interest, and we reported one-tailed Bonferroni-corrected p values when testing predicted effects. For the correlation analysis (see below), we calculated Cook's distance on our data to ensure that the correlation was not driven by outliers.

Results

We used TMS to index the physiological influence of motivational cues on the motor system even before action ensued. Behaviorally, we replicated the results of the first experiment by finding a significant interaction, F(1, 19) = 6.66; p < .05; partial η2 = .26. Participants again failed-to-no-go more following the appetitive cues than the aversive one (Figure 3A), although this was now only marginally significant (50 ± 2.7% vs. 47 ± 2.7%, t(19) = 1.9, p = .08; Cohen's d = 0.42) and they again failed-to-go more following the aversive cue than the appetitive one (38 ± 2.6% vs. 35 ± 2.0%, t(19) = 2.0, p < .05; Cohen's d = 0.45). Also consistent with Experiment 1, participants exhibited a general go bias following appetitive cues compared with aversive cues (57 ± 1.9% vs. 53 ± 2.0%, t(19) = 2.4, p < .05, Cohen's d = 0.55), although this strategy did not benefit overall learning following appetitive cues compared with aversive cues (53 ± 0.7% vs. 53 ± 0.6%, ns; Cohen's d = 0.04). Similar to Experiment 1, the participants were able to track the time-varying patterns. This was evident because their go percentage increased monotonically as a function of probability bins (Figure 3B), and again, they performed significantly better than chance (empirical data, M = 52.3 ± 0.4% vs. permuted data, M = 49.3 ± 0.8%, t(19) = 3.1, p < .01; Cohen's d = 0.70). However, unlike Experiment 1, we did not find increased vigor following appetitive cues compared with aversive cues (RT: 521 ± 13.3 vs. 518 ± 12.6 msec, ns).

Figure 3. 

(A–D) Experiment 2: TMS study. (A) Behavioral pattern of errors following appetitive and aversive cues. (B) Percentage of go response for different cues as a function of probability bins. (C) Normalized MEPs were plotted as a function of cue valence (appetitive, aversive) and participants' subsequent action choice (go, no-go). The significant main effect of affective cue supports the spillover hypothesis. Post hoc t tests compared appetitive-go and aversive-no-go conditions against 1.0 baseline. (D) A positive correlation was observed between the physiological index of motivational spillover (i.e., the MEP of appetitive-go minus aversive-no-go) and the behavioral bias for go/no-go (i.e., the percentage difference between failed-to-no-go for appetitive versus aversive cues, plus the percentage difference between failed-to-go for aversive cue vs. appetitive cue). Error bars denote SEM.

Figure 3. 

(A–D) Experiment 2: TMS study. (A) Behavioral pattern of errors following appetitive and aversive cues. (B) Percentage of go response for different cues as a function of probability bins. (C) Normalized MEPs were plotted as a function of cue valence (appetitive, aversive) and participants' subsequent action choice (go, no-go). The significant main effect of affective cue supports the spillover hypothesis. Post hoc t tests compared appetitive-go and aversive-no-go conditions against 1.0 baseline. (D) A positive correlation was observed between the physiological index of motivational spillover (i.e., the MEP of appetitive-go minus aversive-no-go) and the behavioral bias for go/no-go (i.e., the percentage difference between failed-to-no-go for appetitive versus aversive cues, plus the percentage difference between failed-to-go for aversive cue vs. appetitive cue). Error bars denote SEM.

On each trial, the MEP was measured as the peak-to-peak amplitude of the right index finger muscle, reflecting corticospinal excitability. For each condition of cue valence (appetitive, aversive) and participants' action choice (go, no-go), the MEP was normalized by that participant's MEP in the neutral condition. First, we performed a 2 Cues (appetitive, aversive) × 2 Action (go, no-go) ANOVA to test whether MEPs were predicted by the preceding cues, the following action choice, or an interaction of the two. There was a main effect of Cue, with MEPs being higher for appetitive versus aversive cues, F(1, 19) = 8.68, p < .01, partial η2 = .31 (Figure 3C), and marginally significant main effect of Action, with MEPs before subsequent go being higher than those before no-go trials, F(1, 19) = 3.73, p = .06, partial η2 = .17. The interaction was not significant, p > .1. These results1 are consistent with the motivation-to-motor spillover hypothesis. They suggest that appetitive and aversive cues produce differential bias on motor system excitability soon after the cues have been processed but before action selection is needed.

Next, we tested whether the behavioral appetitive-go bias relates to an increase in MEPs relative to baseline and conversely whether the behavioral aversive-no-go bias relates to a decrease in MEPs relative to baseline (here baseline is the neutral condition, i.e., no change from baseline would give a normalized MEP value of 1.0). Indeed, MEPs following the aversive cue were significantly reduced compared with the 1.0 baseline before the participant decided to withhold an action (aversive-no-go, 0.96 ± 0.02 vs. 1.0, t(19) = 2.69, p < .05, Cohen's d = 0.60), whereas MEPs following the appetitive cues showed a trend toward a significant increase above the baseline before the participant decided to make an action (appetitive-go, 1.05 ± 0.02 vs. 1.0, t(19) = 2.37, p = .057; Cohen's d = 0.45). This suggests that spillover occurs in opposing directions for appetitive versus aversive cues.

As a follow-up of the above analysis, we reasoned that there should be a relationship between the amount of behavioral bias and the degree of relative increase/decrease in MEPs generated by affective cues (i.e., physiological spillover). To test this, we correlated two summary indices across participant. For each participant, the summary index of behavioral bias was the percentage difference between failed-to-no-go for appetitive versus aversive cues, plus the percentage difference between failed-to-go for aversive cue versus appetitive cue (i.e., the interaction manifested in the behavior). For each participant, the summary index of physiological spillover was the summation of increase and decrease compared with 1.0 baseline (i.e., the MEP of appetitive-go minus aversive-no-go). There was a positive correlation between these two indices (Pearson's r = .46, p < .05; although marginal with robust regression, p = .06; Figure 3D), suggesting that those participants who exhibited a stronger disposition to inappropriately go or no-go also exhibited a greater physiological spillover generated by the two motivational cues preceding action selection. This result was not driven by outliers as the Cook's distance analysis revealed no influence of any observation on the regression result. Further analysis showed that this correlation was not driven by the appetitive or aversive effects alone because the correlation between MEPs and appetitive failed-to-no-go was only moderate and not significant (Pearson's r = .26, p > .5) and the correlation between MEPs and aversive failed-to-go was also not significant (Pearson's r = .16, p > .5).

Discussion

In two studies using a novel motivationally biased learning paradigm, we established a replicable effect of opposing behavioral biases on action by appetitive versus aversive cues. We show that participants were biased to go more following appetitive cues whereas they were biased to no-go more following aversive cues. In Experiment 2, we elucidated the mechanism underlying this effect using single-pulse TMS methodology. We show that the bias in behavioral action selection was accompanied by a differential motivation-to-motor “spillover” of the appetitive versus aversive responses into the motor system. Specifically, motor excitability was increased (relative to neutral) following the appetitive cue before taking an action and by contrast was smaller (relative to neutral) following the aversive cue before withholding an action. Across participants, those with a greater motor excitability bias also exhibited a greater behavioral bias subsequently. We thus provide a physiological explanation of how motivational cues produce anomalies in decision-making. We show that motivational cues induced opposing go versus no-go behavioral bias and crucially that these behavioral biases relate to the preceding state of the motor system induced by the motivational cues.

Unlike fMRI studies, we are able to demonstrate the reflexive nature of the affective influence by using single-pulse TMS to index motor system excitability directly. We show that the affective influence of instrumental behavior is mediated by a spillover in the motor system, consistent with the automatic processing of affect hypothesis. Our current results go much further than other recent single-pulse TMS studies of motor influences of appetitive or valuable stimuli (Klein-Flügge & Bestmann, 2012; Gupta & Aron, 2011) by showing opposing effects of appetitive versus aversive bias on go and no-go, respectively, and furthermore by showing that the degree of physiological spillover relates to the subsequent action tendencies within the same experiment. However, in common with these other studies, our results support the idea that there exist continuous interactions between the motor system and other systems (e.g., perception, motivation, emotion, cognition), rather than discrete modular processing within each system.

An alternative to the spillover hypothesis is an “evaluative coding” account (Eder & Rothermund, 2008; Rotteveel & Phaf, 2004). According to this account, affective cues do not generate action directly; instead, they generate positive or negative emotions via nonautomatic, conscious processing that subsequently interacts with the response selection process and thus biases behavior in that way. Rotteveel and Phaf (2004) demonstrated in three experiments what this account entails. First, when “affective” evaluation was applied to the affective stimuli (e.g., evaluating whether a face is happy or angry), there were action biases (i.e., arm extension vs. flexion). That is, arm flexion was facilitated by happy faces whereas arm extension was facilitated by angry faces. Second, when “nonaffective” evaluation was applied for the same set of affective stimuli (e.g., evaluating the gender of the face), there was no action bias at all. Lastly, when affective stimuli were not evaluated at all (e.g., presented as primes), there was only “priming effects” but again no action bias. In other words, although a positively valenced prime facilitated responding to a positively valenced target, it did not preferentially facilitate arm flexion or extension. Together these results of Rotteveel and Phaf (2004) suggest that, although affective cues are processed automatically to generate priming effects, they do not generate action bias without conscious affective evaluations. As our task did not require affective evaluations of the cues (i.e., we did not ask participants to discriminate whether the presented cue was appetitive or aversive), we argue that our results are more compatible with the motivation-to-motor spillover account. However, we acknowledge that we did not have a condition that requires “no evaluation” (e.g., subliminal priming with appetitive/aversive cues) to fully refute the evaluative coding account. Another aspect is that here we used primary reinforcers (sweet vs. bitter juice), which is more salient than pictures of faces with different emotions. Future research could examine how critical it is to use primary reinforcers in the task to drive the “spillover” effect (with or without conscious affective evaluation). More generally, our current approach provides a useful high temporal resolution platform for investigating how affect influences action and decision-making.

Much research suggests that appetitive and aversive biasing are implemented by different subcortical systems. For example, lesions to nucleus accumbens in rodents impairs appetitive Pavlovian-to-instrumental transfer (e.g., Corbit & Balleine, 2011) whereas lesions to the amygdala abolish aversive Pavlovian-to-instrumental transfer (Balleine & Killcross, 2006; Cardinal, Parkinson, Hall, & Everitt, 2002; Maren, 2001; Killcross, Robbins, & Everitt, 1997). Consistent with this, human fMRI studies that have examined Pavlovian control of behavior also revealed activations in accumbens/ventral striatum and in amygdala (e.g., Geurts et al., 2013; Guitart-Masip et al., 2012; Balleine & O'Doherty, 2010; Bray et al., 2008; Talmi et al., 2008; Bjork & Hommer, 2007; Pessiglione et al., 2007; Knutson, Adams, Fong, & Hommer, 2001; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000). We suppose that our appetitive-go and aversive-no-go effects could similarly relate to ventral striatum versus amydala, respectively, although other possibilities exist, for example, differential effects via the direct (go) and indirect (no-go) pathways of the BG (Maia & Frank, 2011).

An important implication of our paradigm and results is that we have operationalized “approach versus avoid” distinction in relation to motor response activation versus suppression (go vs. no-go). This is more concrete than many studies where “approach/avoid” is not so much linked to motor activation versus suppression as used metaphorically, for example, to denote the acceptance versus rejection of gambling offers (e.g., Wright et al., 2012) or to denote pulling toward versus pushing away a joystick (e.g., Wiers, Eberl, Rinck, Becker, & Lindenmeyer, 2011; Chen & Bargh, 1999). Here we show that appetitive cues ramp up the motor system and promote action whereas aversive cues dampen the motor system and promote inaction. This formulation could be useful for many area of future research, including those aiming to assay the putative opponent interactions of dopamine and serotonin (Boureau & Dayan, 2011; Cools, Nakamura, & Daw, 2011) and those aiming to examine the influence of motivational cues on behavior in purchasing, gambling, other real-world scenarios. Another implication of this paradigm is that it creates a platform for investigating how to “curb” the misbehavior induced by motivational cues (Kovach et al., 2012; Dayan et al., 2006). For example, participants could be trained to overcome motivational spillover by using cue-reappraisal strategies (Hare, Camerer, & Rangel, 2009) or perhaps via top–down response suppression in the case of appetitive cues. Such strategies could dampen the influence of motivational cues before they contaminate decision-making. Alternatively, participants could be trained to better represent goals by boosting working memory capacity. This would help to prevent capture by motivational cues that lead to maladaptive behaviors (cf., Anderson, Laurent, & Yantis, 2011). Clinically, our paradigm could be used in people with substance abuse disorders or eating disorders to investigate whether they exhibit greater susceptibility to appetitive cues than healthy individuals and, furthermore, which strategies best remediate the appetitive bias and eliminate maladaptive behaviors (e.g., drug seeking, overeating).

Our study was limited in some ways. First, we did not collect individual preference ratings to confirm a preference in each participant in these particular experiments for appetitive versus aversive cues, which is important as there are individual differences in cue preference (Ernst, Daniele, & Frantz, 2011). Yet we had established, in pilot studies, that all participants preferred apple juice to hops tea in this context. Future studies should take individual preference into account, and this could generate a larger and more easily detectable appetitive/aversive bias. Second, we did not have MEP data for a “passive viewing” condition in which no decision-making processes are required. Such a condition could provide information about whether there is an MEP amplitude change based on mere perception of appetitive versus aversive cues in our task without the decision-making context (Kapogiannis, Campion, Grafman, & Wassermann, 2008; Hajcak et al., 2007).

In summary, these results show that positive and negative affect hijacks the will by spilling into opposing action tendencies (to move or withhold) soon after affective cues are perceived and processed. This demonstrates how approach and avoidance can be operationalized in concrete terms in the motor system, and it suggests specific ways to curb the misbehavior of the will and to measure if that is successful. Finally, these results speak to an important theoretical issue in emotion research—they show that affective hijacks of the will can sometimes reflect an automatic/reflexive process rather than merely being a consequence of evaluative coding.

Acknowledgments

The authors thank Hanneke den Ouden for generating the time-varying contingencies and Melissa Aguilar for data collection in Experiment 1. Funding was gratefully received from NIDA grant DA 026452 (to A. R. A.).

Reprint requests should be sent to Yu-Chin Chiu, Duke University, Center for Cognitive Neuroscience, LSRC Box 90999, Durham, NC 27708, or via e-mail: chiu.yuchin@gmail.com.

Note

1. 

There were four left-handed participants in the original sample. However, handedness did not confound the results as main effect of Cue (appetitive vs. aversive) was still statistically significant, F(1, 15) = 5.88, p < .05, after excluding four left-handed participants.

REFERENCES

Anderson
,
B. A.
,
Laurent
,
P. A.
, &
Yantis
,
S.
(
2011
).
Value-driven attentional capture.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
10367
10371
.
Balleine
,
B. W.
, &
Killcross
,
S.
(
2006
).
Parallel incentive processing: An integrated view of amygdala function.
Trends in Neurosciences
,
29
,
272
279
.
Balleine
,
B. W.
, &
O'Doherty
,
J. P.
(
2010
).
Human and rodent homologies in action control: Corticostriatal determinants of goal-directed and habitual action.
Neuropsychopharmacology
,
35
,
48
69
.
Bjork
,
J. M.
, &
Hommer
,
D. W.
(
2007
).
Anticipating instrumentally obtained and passively-received rewards: A factorial fMRI investigation.
Behavioural Brain Research
,
177
,
165
170
.
Boureau
,
Y.-L.
, &
Dayan
,
P.
(
2011
).
Opponency revisited: Competition and cooperation between dopamine and serotonin.
Neuropsychopharmacology
,
36
,
74
97
.
Brainard
,
D. H.
(
1997
).
The Psychophysics Toolbox.
Spatial Vision
,
10
,
433
436
.
Bray
,
S.
,
Rangel
,
A.
,
Shimojo
,
S.
,
Balleine
,
B.
, &
O'Doherty
,
J. P.
(
2008
).
The neural mechanisms underlying the influence of Pavlovian cues on human decision making.
Journal of Neuroscience
,
28
,
5861
5866
.
Breland
,
K.
, &
Breland
,
M.
(
1961
).
The misbehavior of organisms.
American Psychologist
,
16
,
681
684
.
Cacioppo
,
J. T.
,
Priester
,
J. R.
, &
Berntson
,
G. G.
(
1993
).
Rudimentary determinants of attitudes. II: Arm flexion and extension have differential effects on attitudes.
Journal of Personality and Social Psychology
,
65
,
5
17
.
Cardinal
,
R. N.
,
Parkinson
,
J. A.
,
Hall
,
J.
, &
Everitt
,
B. J.
(
2002
).
Emotion and motivation: The role of the amygdala, ventral striatum, and prefrontal cortex.
Neuroscience and Biobehavioral Reviews
,
26
,
321
352
.
Chen
,
M.
, &
Bargh
,
J. A.
(
1999
).
Consequences of automatic evaluation: Immediate behavioral predispositions to approach or avoid the stimulus.
Personality and Social Psychology Bulletin
,
25
,
215
224
.
Cools
,
R.
,
Nakamura
,
K.
, &
Daw
,
N. D.
(
2011
).
Serotonin and dopamine: Unifying affective, activational, and decision functions.
Neuropsychopharmacology
,
36
,
98
113
.
Corbit
,
L. H.
, &
Balleine
,
B. W.
(
2011
).
The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell.
The Journal of Neuroscience: The Official Journal of the Society for Neuroscience
,
31
,
11786
11794
.
Damasio
,
A. R.
(
1996
).
The somatic marker hypothesis and the possible functions of the prefrontal cortex.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
351
,
1413
1420
.
Daw
,
N. D.
,
O'Doherty
,
J. P.
,
Dayan
,
P.
,
Seymour
,
B.
, &
Dolan
,
R. J.
(
2006
).
Cortical substrates for exploratory decisions in humans.
Nature
,
441
,
876
879
.
Dayan
,
P.
,
Niv
,
Y.
,
Seymour
,
B.
, &
Daw
,
N. D.
(
2006
).
The misbehavior of value and the discipline of the will.
Neural Networks
,
19
,
1153
1160
.
Delgado
,
M. R.
,
Nystrom
,
L. E.
,
Fissell
,
C.
,
Noll
,
D. C.
, &
Fiez
,
J. A.
(
2000
).
Tracking the hemodynamic responses to reward and punishment in the striatum.
Journal of Neurophysiology
,
84
,
3072
3077
.
Eder
,
A. B.
, &
Rothermund
,
K.
(
2008
).
When do motor behaviors (mis)match affective stimuli? An evaluative coding view of approach and avoidance reactions.
Journal of Experimental Psychology: General
,
137
,
262
281
.
Ernst
,
M.
,
Daniele
,
T.
, &
Frantz
,
K.
(
2011
).
New perspectives on adolescent motivated behavior: Attention and conditioning.
Developmental Cognitive Neuroscience
,
1
,
377
389
.
Geurts
,
D. E. M.
,
Huys
,
Q. J. M.
,
Den Ouden
,
H. E. M.
, &
Cools
,
R.
(
2013
).
Aversive Pavlovian control of instrumental behavior in humans.
Journal of Cognitive Neuroscience
,
25
,
1428
1441
.
Guitart-Masip
,
M.
,
Fuentemilla
,
L.
,
Bach
,
D. R.
,
Huys
,
Q. J. M.
,
Dayan
,
P.
,
Dolan
,
R. J.
,
et al
(
2011
).
Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain.
Journal of Neuroscience
,
31
,
7867
7875
.
Guitart-Masip
,
M.
,
Huys
,
Q. J. M.
,
Fuentemilla
,
L.
,
Dayan
,
P.
,
Duzel
,
E.
, &
Dolan
,
R. J.
(
2012
).
Go and no-go learning in reward and punishment: Interactions between affect and effect.
Neuroimage
,
62
,
154
166
.
Gupta
,
N.
, &
Aron
,
A. R.
(
2011
).
Urges for food and money spill over into motor system excitability before action is taken.
The European Journal of Neuroscience
,
33
,
183
188
.
Hajcak
,
G.
,
Molnar
,
C.
,
George
,
M. S.
,
Bolger
,
K.
,
Koola
,
J.
, &
Nahas
,
Z.
(
2007
).
Emotion facilitates action: A transcranial magnetic stimulation study of motor cortex excitability during picture viewing.
Psychophysiology
,
44
,
91
97
.
Hare
,
T. A.
,
Camerer
,
C. F.
, &
Rangel
,
A.
(
2009
).
Self-control in decision-making involves modulation of the vmPFC valuation system.
Science
,
324
,
646
648
.
Harmon-Jones
,
E.
,
Gable
,
P. A.
, &
Price
,
T. F.
(
2012
).
The influence of affective states varying in motivational intensity on cognitive scope.
Frontiers in Integrative Neuroscience
,
6
,
73
.
Hershberger
,
W. A.
(
1986
).
An approach through the looking-glass.
Animal Learning & Behavior
,
14
,
443
451
.
Huys
,
Q. J. M.
,
Cools
,
R.
,
Gölzer
,
M.
,
Friedel
,
E.
,
Heinz
,
A.
,
Dolan
,
R. J.
,
et al
(
2011
).
Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding.
PLoS Computational Biology
,
7
,
e1002028
.
Kapogiannis
,
D.
,
Campion
,
P.
,
Grafman
,
J.
, &
Wassermann
,
E. M.
(
2008
).
Reward-related activity in the human motor cortex.
The European Journal of Neuroscience
,
27
,
1836
1842
.
Killcross
,
S.
,
Robbins
,
T. W.
, &
Everitt
,
B. J.
(
1997
).
Different types of fear-conditioned behaviour mediated by separate nuclei within amygdala.
Nature
,
388
,
377
380
.
Klein-Flügge
,
M. C.
, &
Bestmann
,
S.
(
2012
).
Time-dependent changes in human corticospinal excitability reveal value-based competition for action during decision processing.
Journal of Neuroscience
,
32
,
8373
8382
.
Knutson
,
B.
,
Adams
,
C. M.
,
Fong
,
G. W.
, &
Hommer
,
D.
(
2001
).
Anticipation of increasing monetary reward selectively recruits nucleus accumbens.
Journal of Neuroscience
,
21
,
RC159
.
Kovach
,
C. K.
,
Daw
,
N. D.
,
Rudrauf
,
D.
,
Tranel
,
D.
,
O'Doherty
,
J. P.
, &
Adolphs
,
R.
(
2012
).
Anterior prefrontal cortex contributes to action selection through tracking of recent reward trends.
Journal of Neuroscience
,
32
,
8434
8442
.
Maia
,
T. V.
, &
Frank
,
M. J.
(
2011
).
From reinforcement learning models to psychiatric and neurological disorders.
Nature Neuroscience
,
14
,
154
162
.
Maren
,
S.
(
2001
).
Neurobiology of Pavlovian fear conditioning.
Annual Review of Neuroscience
,
24
,
897
931
.
Pessiglione
,
M.
,
Schmidt
,
L.
,
Draganski
,
B.
,
Kalisch
,
R.
,
Lau
,
H.
,
Dolan
,
R. J.
,
et al
(
2007
).
How the brain translates money into force: A neuroimaging study of subliminal motivation.
Science
,
316
,
904
906
.
Rinck
,
M.
, &
Becker
,
E. S.
(
2007
).
Approach and avoidance in fear of spiders.
Journal of Behavior Therapy and Experimental Psychiatry
,
38
,
105
120
.
Rossini
,
P. M.
,
Barker
,
A. T.
,
Berardelli
,
A.
,
Caramia
,
M. D.
,
Caruso
,
G.
, &
Cracco
,
R. Q.
(
1994
).
Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: Basic principles and procedures for routine clinical application.
Report of an IFCN committee Electroencephalography and Clinical Neurophysiology
,
91
,
79
92
.
Rotteveel
,
M.
, &
Phaf
,
R. H.
(
2004
).
Automatic affective evaluation does not automatically predispose for arm flexion and extension.
Emotion
,
4
,
156
172
.
Talmi
,
D.
,
Seymour
,
B.
,
Dayan
,
P.
, &
Dolan
,
R. J.
(
2008
).
Human Pavlovian-instrumental transfer.
Journal of Neuroscience
,
28
,
360
368
.
Wiers
,
R. W.
,
Eberl
,
C.
,
Rinck
,
M.
,
Becker
,
E. S.
, &
Lindenmeyer
,
J.
(
2011
).
Retraining automatic action tendencies changes alcoholic patients' approach bias for alcohol and improves treatment outcome.
Psychological Science
,
22
,
490
497
.
Wright
,
N. D.
,
Symmonds
,
M.
,
Hodgson
,
K.
,
Fitzgerald
,
T. H. B.
,
Crawford
,
B.
, &
Dolan
,
R. J.
(
2012
).
Approach-avoidance processes contribute to dissociable impacts of risk and loss on choice.
Journal of Neuroscience
,
32
,
7009
7020
.

Author notes

*

Current affiliation: Duke University.