## Abstract

Neural activity in the striatum has consistently been shown to scale with the value of anticipated rewards. As a result, it is common across a number of neuroscientific subdiscliplines to associate activation in the striatum with anticipation of a rewarding outcome or a positive emotional state. However, most studies have failed to dissociate expected value from the motivation associated with seeking a reward. Although motivation generally scales positively with increases in potential reward, there are circumstances in which this linkage does not apply. The current study dissociates value-related activation from that induced by motivation alone by employing a task in which motivation increased as anticipated reward decreased. This design reverses the typical relationship between motivation and reward, allowing us to differentially investigate fMRI BOLD responses that scale with each. We report that activity scaled differently with value and motivation across the striatum. Specifically, responses in the caudate and putamen increased with motivation, whereas nucleus accumbens activity increased with expected reward. Consistent with this, self-report ratings indicated a positive association between caudate and putamen activity and arousal, whereas activity in the nucleus accumbens was more associated with liking. We conclude that there exist regional limits on inferring reward expectation from striatal activation.

## INTRODUCTION

Neural activity in the striatum has consistently been shown to scale with the value of anticipated rewards in humans (Delgado, Locke, Stenger, & Fiez, 2003; Knutson, Adams, Fong, & Hommer, 2001) and other animals (Cromwell, Hassani, & Schultz, 2005; Cromwell & Schultz, 2003; Kawagoe, Takikawa, & Hikosaka, 1998). On the basis of this association, a positive emotional state or the anticipation of a rewarding outcome is often inferred based on activation in many striatal regions, including the nucleus accumbens (Cloutier, Heatherton, Whalen, & Kelley, 2008; Harbaugh, Mayr, & Burghart, 2007), putamen (Kang et al., 2009; Aron et al., 2005), and caudate (Delgado, Frank, & Phelps, 2005; King-Casas et al., 2005).

Most fMRI studies of reward have measured neural activity as people make decisions or anticipate executing actions that may lead to the acquisition of a positive outcome. However, this approach generally fails to dissociate anticipation of reward from the motivation required to obtain the reward, which often scales with value. Thus, anticipated reward and motivation are perfectly coupled, such that motivation follows from the anticipation of a potential reward or avoidance of a punishment (Niv, Daw, Joel, & Dayan, 2007). This linkage makes adaptive sense, because successful foraging calls for actions that lead to maximal rewards (Pyke, Pulliam, & Charnov, 1977).

Motivation, as we use the term herein, refers to the drive for action that leads one to work to obtain rewards (Kouneiher, Charron, & Koechlin, 2009; Pessoa, 2009; Niv et al., 2007). Under this definition, motivation precisely targets the cognitions and behaviors that occur in preparation for action. Thus, although the “energizing” nature of motivation directly follows from the availability of reward, it is itself a distinct construct that can be manipulated and studied experimentally. Moreover, motivation is preliminary to and distinct from motor preparation itself. Indeed, motivation may be expected to prioritize subsequent action planning.

There is clearly an adaptive value for a tight linkage between reward and motivation. Nonetheless, emerging evidence suggests that they can be neurally dissociated. Activation in the ventral and dorsal striatum is greatest when reward receipt requires direct action, suggesting that reward-related activity may be contingent on motivation (O'Doherty et al., 2004; Zink, Pagnoni, Martin-Skurski, Chappelow, & Berns, 2004). Furthermore, independent manipulation of outcome reward value and motor response requirements (i.e., go vs. no-go) suggests that striatal activity is more dependent on motor demands than anticipated reward (Kurniawan, Guitart-Masip, & Dayan, 2013; Guitart-Masip et al., 2011, 2012).

In this article, we argue that different parts of the striatum specialize in processing anticipated reward versus motivation. Our thesis is consistent with rodent pharmacological studies and studies of striatal anatomical connectivity, which suggest that ventral striatum function is more associated with reward anticipation, whereas dorsal striatal function is more closely associated with processes related to action preparation (Mogenson, Jones, & Yim, 1980). By distinguishing ventral from dorsal striatal components, we show that it is possible to identify distinct responses to anticipated reward value and the motivation required to obtain it.

We distinguished anticipated reward from motivation by using a task that is unique to our knowledge in that participants exerted more effort to obtain rewards with lower probability of occurrence. This reversed the correlation between expected value and motivation evident in most studies for which motivation directly follows anticipated reward. Thus, in our task, if activity in a given striatal region were driven by expected value, its activation should have increased with reward likelihood. On the other hand, if activity in that striatal region were more closely aligned with motivation, its activation should have decreased with reward probability, because increased reward likelihood also involved decreased motivation.

## METHODS

A total of 16 participants completed the study (seven men, nine women; aged 18–34 years, mean = 21.6 years, SD = 4.2 years). One participant was excluded for excessive motion (>2 mm between acquisitions), leaving 15 participants for all analyses. All participants gave informed consent before participating in the study using procedures approved the Stanford Institutional Review Board.

Before scanning, participants completed a 12-trial motor task to assess their baseline RTs. Participants responded as quickly as possible by pressing a button in response to the appearance of a target that was shown 2–4 sec after the start of each trial (uniformly distributed). The median RT across the 12 trials was taken as the participants' baseline RT for the modified monetary incentive delay task (MID; Knutson, Taylor, Kaufman, Peterson, & Glover, 2005).

During the scan, participants engaged in a modified version of the MID task, which was divided into four 40-trial blocks lasting roughly 10 min each. In this task, participants were instructed to respond as quickly as possible following the onset of an unpredictable cued target. If the response occurred before the offset of the target, money was added to the participant's earnings. Participants were informed that they would receive half of this total sum in cash at the end of the session.

Each trial was divided into four parts: (1) cue, (2) delay, (3) response, and (4) outcome (Figure 1A). During the cue phase (2 sec), participants were shown a bar (cue) whose fill and color denoted that trial's probability of reward (40%, 60%, 80%, or 99%) and magnitude of payout ($0.25 or$1.00), respectively. The bar had four levels of fill, with less fill indicating lower trial difficulty, hence greater probability of obtaining reward. The bar was either white or green in color; white signified a low magnitude trial, whereas green indicated high magnitude. Probability and magnitude were manipulated orthogonally, with each probability–magnitude pair occurring five times within each 40-trial block.

Figure 1.

(A) Participants were scanned while performing a modified version of a MID task. At the start of each trial, participants were cued for 2 sec with an image indicating the difficulty level and the magnitude of reward. After a variable 2- to 4.5-sec delay, a probe appeared on the screen. To obtain reward, participants had to respond to the probe within a threshold RT determined by the difficulty level of the trial. Following this response period, feedback was presented for 2 sec, indicating the reward received on the current trial and cumulatively throughout the experiment. (B) RTs in each condition of the task. RTs were faster for trials of higher difficulty (lower probability of success), F(3, 42) = 25.51, p < .001, and higher reward magnitude, F(1, 41) = 18.33, p < .001. Error bars represent ±1 within-subject standard error.

Figure 1.

(A) Participants were scanned while performing a modified version of a MID task. At the start of each trial, participants were cued for 2 sec with an image indicating the difficulty level and the magnitude of reward. After a variable 2- to 4.5-sec delay, a probe appeared on the screen. To obtain reward, participants had to respond to the probe within a threshold RT determined by the difficulty level of the trial. Following this response period, feedback was presented for 2 sec, indicating the reward received on the current trial and cumulatively throughout the experiment. (B) RTs in each condition of the task. RTs were faster for trials of higher difficulty (lower probability of success), F(3, 42) = 25.51, p < .001, and higher reward magnitude, F(1, 41) = 18.33, p < .001. Error bars represent ±1 within-subject standard error.

At the end of the cue phase, a blank screen with a fixation cross was shown for a random 2–4.5 sec (uniform) duration. Following this delay, the target probe (a white star) was presented briefly on the screen, and participants were instructed to press a button to respond before it disappeared. The duration of the response probe presentation was determined based on a staircase procedure (discussed in the next section) that dynamically adjusted the probe duration to achieve the target probability of reward for each trial type. Following the response period, a variable duration ISI was used to maintain a constant 6 sec between cue offset and presentation of the trial outcome.

## DISCUSSION

The major aim of this experiment was to differentiate neural responses to anticipated reward and motivation in the NAcc and CPu. Thus, we framed reward probability as difficulty when describing the task procedures to participants. Perhaps for this reason, trials involving lower probabilities of reward elicited greater motivation in this study. This interpretation is supported by the behavioral results, because RTs were faster for the lower probability and higher magnitude trials. Thus, although the lower probability trials were inferior in terms of expected value, they nonetheless elicited greater motivation.

We found that activation in the CPu decreased with probability and increased with the magnitude of anticipated reward. Together, these findings indicate that activation in the CPu scales with motivation rather than expected reward. The results from the ROI analysis in the CPu support this interpretation. A reward-centric account of striatal activity would predict that activation in the CPu should be greatest for the trial types that received the highest liking ratings. Instead, for trials with equivalent magnitudes of prospective reward, we found the opposite relationship—activation throughout the CPu decreased with liking but increased with ratings of arousal, a construct closely related to motivation. This occurred mainly because, although participants liked the lower probability trials less, they were simultaneously motivated toward more effortful responding because of the greater difficulty of those trials.

These results are difficult to reconcile with theories of dorsal striatal function, which assume that activity in this region scales positively with measures of liking or subjective preference (Balleine et al., 2007; Hikosaka, Takikawa, & Kawagoe, 2000). They instead suggest that this common finding may occur in some regions of the striatum because anticipated reward is often conflated with motivation. Our findings therefore have important implications regarding what can be inferred based on activation in the caudate and putamen. In particular, many investigators have inferred a positive emotional state or the anticipation of a positive outcome from activation in this region (e.g., King-Casas et al., 2005; De Quervain et al., 2004). The current results suggest that such interpretations need qualification, because activation in this region can have a positive or negative relationship with expected value, depending on the extent to which value differences affect motivation.

We relied on RT as a behavioral measure of motivation. This is reasonable because faster RTs can only result from greater exertion of effort and greater effort is expected to follow from increased motivation. Our measure of motivation is therefore certainly related to other experiments that manipulate expected effort (Kurniawan et al., 2010; Croxson, Walton, O'Reilly, Behrens, & Rushworth, 2009). However, the aspect of motivation that we investigated here is qualitatively distinct in an important way. In particular, previous studies investigated activation associated with making choices regarding cognitively or physically taxing actions or receiving information that such effort would be required in the future. In these latter cases, the effortful action neither occurred nor was prepared for proximate to the time of choice; rather, effort was only considered hypothetically. Our experiment therefore studied the “energizing” effects of motivation, whereas other studies have focused on discounting value based on future anticipated work. On the basis of the current findings, these two constructs appear to be distinct.

Our results may also appear to differ from those of previous work, which found that activation in the NAcc increased in response to a cue that signaled high effort requirements for the subsequent block (Botvinick, Huffstetler, & McGuire, 2009). This stands in apparent contrast to the current results, which found that activation in the NAcc responded to differences in subjective value, rather than motivation. This discrepancy is likely attributable to a major difference regarding the design of the two studies. In the current task, effort was manipulated via reward probability, which directly influences the likelihood of obtaining reward. In contrast, Botvinick et al. (2009) manipulated effort orthogonally to reward expectancy and therefore targeted a substantially different construct.

Motivation, as assayed in our experiment, is closely related to arousal and motor preparation. However, these labels fail to fully characterize the behavioral and cognitive changes induced by our manipulation. Arousal is defined as an emotional state that can occur even in the absence of active reward seeking (e.g., Watson & Tellegen, 1985). For example, many experiments have studied arousal by having participants passively view emotionally charged stimuli and report activation patterns that differ substantially from those in the present experiment (e.g., Anders, Lotze, Erb, Grodd, & Birbaumer, 2004). Similarly, although optimal performance on our task requires motor preparation, we intentionally investigated activity elicited by a cue that preceded motor activity by several seconds. Moreover, we factored out neural responses related to motor preparation as well as possible in our analyses. Although the motivation we investigate is certainly closely associated with energizing motor preparation, we believe it is distinct from motor acts themselves. Thus, although our task bears important similarities to arousal and motor preparation, it encompasses aspects of motivation that are insufficiently characterized by these alternative labels.

Our results also suggest a reinterpretation of the recent claims that the requirement for action strongly impacts activation in the CPu whereas the valence of the potential outcome (i.e., gaining vs. losing money) has a weaker relationship (Kurniawan et al., 2013; Guitart-Masip et al., 2011, 2012). In these studies, the difference in value between a successful versus unsuccessful response was the same regardless of whether participants attempted to obtain a reward or avoid a punishment. It is therefore likely that, although the potential outcomes differed between the two conditions, the motivation to succeed was similar. Additionally, action in these studies was extrinsically motivated, because trials requiring motor responses only occurred when dictated by cues indicating trial type. In contrast, the novel design employed in the current study enabled the investigation of motivated states that were intrinsically generated.

Our primary finding is that distinct striatal regions subserve different functions and specifically that the NAcc serves a distinct function from the rest of the striatum. We found that activation in the NAcc scaled positively with measures of liking and the subjective probability of obtaining reward. This is consistent with the existing literature and reinforces the role that this region plays in valuation, independent of the actual effort required, especially when the rewards are highly salient (Litt, Plassmann, Shiv, & Rangel, 2010; Knutson et al., 2005).

The ventral striatum, including the NAcc, is particularly sensitive to susceptibility artifacts and signal dropout, which can influence the variability and consistency of analyses that target this subregion (Sacchet & Knutson, 2013). It is therefore possible our analyses may have underestimated the response of the NAcc relative to the CPu. However, this would not impact the interpretation of the findings reported presently, as they are based on activation patterns that were detectable in spite of potential signal loss.

Although the current task investigated motivation as indexed by RTs, these findings may extend to more general differences in motivation. Motivation manifests in many different ways, only one of which is effort (and reduced RTs). We focused on RT in this study because it provides a relatively unambiguous measure of motivation. That said, motivation is also related to measures such as willingness to pay and a desire to seek out information, both of which have previously been associated with increased activation in the CPu (De Martino, Kumaran, Holt, & Dolan, 2009; Kang et al., 2009; Plassmann, O'Doherty, & Rangel, 2007; Weber et al., 2007).

In summary, these findings bring us closer to understanding the function of the CPu and NAcc in human decision-making and motivated behavior. Whereas some accounts assume that activation in the CPu scales with expected reward, the current results indicate that striatal responses outside the NAcc are more associated with motivation, above and beyond value or liking.

Reprint requests should be sent to Eric M. Miller, Department of Psychology, Stanford University, 450 Serra Mall, Stanford, CA 94305, or via e-mail: ermiller@stanford.edu.

## REFERENCES

Anders
,
S.
,
Lotze
,
M.
,
Erb
,
M.
,
Grodd
,
W.
, &
Birbaumer
,
N.
(
2004
).
Brain activity underlying emotional valence and arousal: A response-related fMRI study.
Human Brain Mapping
,
23
,
200
209
.
Aron
,
A.
,
Fisher
,
H.
,
Mashek
,
D. J.
,
Strong
,
G.
,
Li
,
H.
, &
Brown
,
L. L.
(
2005
).
Reward, motivation, and emotion systems associated with early-stage intense romantic love.
Journal of Neurophysiology
,
94
,
327
337
.
Balleine
,
B. W.
,
,
M. R.
, &
Hikosaka
,
O.
(
2007
).
The role of the dorsal striatum in reward and decision-making.
Journal of Neuroscience
,
27
,
8161
8165
.
Botvinick
,
M. M.
,
Huffstetler
,
S.
, &
McGuire
,
J. T.
(
2009
).
Effort discounting in human nucleus accumbens.
Cognitive, Affective, & Behavioral Neuroscience
,
9
,
16
27
.
Cloutier
,
J.
,
Heatherton
,
T. F.
,
Whalen
,
P. J.
, &
Kelley
,
W. M.
(
2008
).
Are attractive people rewarding? Sex differences in the neural substrates of facial attractiveness.
Journal of Cognitive Neuroscience
,
20
,
941
951
.
Cox
,
R. W.
(
1996
).
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages.
Computers and Biomedical Research
,
29
,
162
173
.
Cromwell
,
H. C.
,
Hassani
,
O. K.
, &
Schultz
,
W.
(
2005
).
Relative reward processing in primate striatum.
Experimental Brain Research
,
162
,
520
525
.
Cromwell
,
H. C.
, &
Schultz
,
W.
(
2003
).
Effects of expectations for different reward magnitudes on neuronal activity in primate striatum.
Journal of Neurophysiology
,
89
,
2823
2838
.
Croxson
,
P. L.
,
Walton
,
M. E.
,
O'Reilly
,
J. X.
,
Behrens
,
T. E. J.
, &
Rushworth
,
M. F. S.
(
2009
).
Effort-based cost-benefit valuation and the human brain.
Journal of Neuroscience
,
29
,
4531
4541
.
De Martino
,
B.
,
Kumaran
,
D.
,
Holt
,
B.
, &
Dolan
,
R. J.
(
2009
).
The neurobiology of reference-dependent value computation.
Journal of Neuroscience
,
29
,
3833
3842
.
De Quervain
,
D. J. F.
,
Fischbacher
,
U.
,
Treyer
,
V.
,
Schellhammer
,
M.
,
Schnyder
,
U.
,
Buck
,
A.
,
et al
(
2004
).
The neural basis of altruistic punishment.
Science
,
305
,
1254
1258
.
,
M. R.
,
Frank
,
R. H.
, &
Phelps
,
E. A.
(
2005
).
Perceptions of moral character modulate the neural systems of reward during the trust game.
Nature Neuroscience
,
8
,
1611
1618
.
,
M. R.
,
Locke
,
H. M.
,
Stenger
,
V. A.
, &
Fiez
,
J. A.
(
2003
).
Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations.
Cognitive, Affective, & Behavioral Neuroscience
,
3
,
27
38
.
Fischl
,
B.
,
Salat
,
D. H.
,
Busa
,
E.
,
Albert
,
M.
,
Dieterich
,
M.
,
Haselgrove
,
C.
,
et al
(
2002
).
Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain.
Neuron
,
33
,
341
355
.
Guitart-Masip
,
M.
,
Chowdhury
,
R.
,
Sharot
,
T.
,
Dayan
,
P.
,
Duzel
,
E.
, &
Dolan
,
R. J.
(
2012
).
Action controls dopaminergic enhancement of reward representations.
Proceedings of the National Academy of Sciences, U.S.A.
,
109
,
7511
7516
.
Guitart-Masip
,
M.
,
Fuentemilla
,
L.
,
Bach
,
D. R.
,
Huys
,
Q. J. M.
,
Dayan
,
P.
,
Dolan
,
R. J.
,
et al
(
2011
).
Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain.
Journal of Neuroscience
,
31
,
7867
7875
.
Haber
,
S. N.
, &
Knutson
,
B.
(
2009
).
The reward circuit: Linking primate anatomy and human imaging.
Neuropsychopharmacology
,
35
,
4
26
.
Harbaugh
,
W. T.
,
Mayr
,
U.
, &
Burghart
,
D. R.
(
2007
).
Neural responses to taxation and voluntary giving reveal motives for charitable donations.
Science
,
316
,
1622
1625
.
Hikosaka
,
O.
,
Takikawa
,
Y.
, &
Kawagoe
,
R.
(
2000
).
Role of the basal ganglia in the control of purposive saccadic eye movements.
Physiological Reviews
,
80
,
953
978
.
Kang
,
M. J.
,
Hsu
,
M.
,
Krajbich
,
I. M.
,
Loewenstein
,
G.
,
McClure
,
S. M.
,
Wang
,
J. T.-Y.
,
et al
(
2009
).
The wick in the candle of learning: Epistemic curiosity activates reward circuitry and enhances memory.
Psychological Science
,
20
,
963
973
.
Kawagoe
,
R.
,
Takikawa
,
Y.
, &
Hikosaka
,
O.
(
1998
).
Expectation of reward modulates cognitive signals in the basal ganglia.
Nature Neuroscience
,
1
,
411
416
.
King-Casas
,
B.
,
Tomlin
,
D.
,
Anen
,
C.
,
Camerer
,
C. F.
,
Quartz
,
S. R.
, &
Montague
,
P. R.
(
2005
).
Getting to know you: Reputation and trust in a two-person economic exchange.
Science
,
308
,
78
83
.
Knutson
,
B.
,
,
C. M.
,
Fong
,
G. W.
, &
Hommer
,
D.
(
2001
).
Anticipation of increasing monetary reward selectively recruits nucleus accumbens.
Journal of Neuroscience
,
21
,
1
5
.
Knutson
,
B.
,
Taylor
,
J.
,
Kaufman
,
M.
,
Peterson
,
R.
, &
Glover
,
G.
(
2005
).
Distributed neural representation of expected value.
Journal of Neuroscience
,
25
,
4806
4812
.
Kouneiher
,
F.
,
Charron
,
S.
, &
Koechlin
,
E.
(
2009
).
Motivation and cognitive control in the human prefrontal cortex.
Nature Publishing Group
,
12
,
939
945
.
Kurniawan
,
I. T.
,
Guitart-Masip
,
M.
, &
Dayan
,
P.
(
2013
).
Effort and valuation in the brain: The effects of anticipation and execution.
Journal of Neuroscience
,
33
,
6160
6169
.
Kurniawan
,
I. T.
,
Seymour
,
B.
,
Talmi
,
D.
,
Yoshida
,
W.
,
Chater
,
N.
, &
Dolan
,
R. J.
(
2010
).
Choosing to make an effort: The role of striatum in signaling physical effort of a chosen action.
Journal of Neurophysiology
,
104
,
313
321
.
Litt
,
A.
,
Plassmann
,
H.
,
Shiv
,
B.
, &
Rangel
,
A.
(
2010
).
Dissociating valuation and saliency signals during decision-making.
Cerebral Cortex
,
21
,
95
102
.
Mogenson
,
G. J.
,
Jones
,
D. L.
, &
Yim
,
C. Y.
(
1980
).
From motivation to action: Functional interface between the limbic system and the motor system.
Progress in Neurobiology
,
14
,
69
97
.
Niv
,
Y.
,
Daw
,
N. D.
,
Joel
,
D.
, &
Dayan
,
P.
(
2007
).
Tonic dopamine: Opportunity costs and the control of response vigor.
Psychopharmacology
,
191
,
507
520
.
O'Doherty
,
J.
,
Dayan
,
P.
,
Schultz
,
J.
,
Deichmann
,
R.
,
Friston
,
K.
, &
Dolan
,
R. J.
(
2004
).
Dissociable roles of ventral and dorsal striatum in instrumental conditioning.
Science
,
304
,
452
454
.
Pessoa
,
L.
(
2009
).
How do emotion and motivation direct executive control?
Trends in Cognitive Sciences
,
13
,
160
166
.
Plassmann
,
H.
,
O'Doherty
,
J.
, &
Rangel
,
A.
(
2007
).
Orbitofrontal cortex encodes willingness to pay in everyday economic transactions.
Journal of Neuroscience
,
27
,
9984
9988
.
Pyke
,
G. H.
,
Pulliam
,
H. R.
, &
Charnov
,
E. L.
(
1977
).
Optimal foraging: A selective review of theory and tests.
Quarterly Review of Biology
,
52
,
137
154
.
Sacchet
,
M. D.
, &
Knutson
,
B.
(
2013
).
Spatial smoothing systematically biases the localization of reward-related brain activity.
Neuroimage
,
66
,
270
277
.
Tobler
,
P. N.
,
O'Doherty
,
J. P.
,
Dolan
,
R. J.
, &
Schultz
,
W.
(
2007
).
Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems.
Journal of Neurophysiology
,
97
,
1621
1632
.
Watson
,
D.
, &
Tellegen
,
A.
(
1985
).
Toward a consensual structure of mood.
Psychological Bulletin
,
98
,
219
235
.
Weber
,
B.
,
Aholt
,
A.
,
Neuhaus
,
C.
,
Trautner
,
P.
,
Elger
,
C. E.
, &
Teichert
,
T.
(
2007
).
Neural evidence for reference-dependence in real-market-transactions.
Neuroimage
,
35
,
441
447
.
Zink
,
C. F.
,
Pagnoni
,
G.
,
Martin-Skurski
,
M. E.
,
Chappelow
,
J. C.
, &
Berns
,
G. S.
(
2004
).
Human striatal responses to monetary reward depend on saliency.
Neuron
,
42
,
509
517
.