Choosing between smaller prompt rewards and larger later rewards is a common choice problem, and studies widely agree that frontostriatal circuits heavily innervated by dopamine are centrally involved. Understanding how dopamine modulates intertemporal choice has important implications for neurobiological models and for understanding the mechanisms underlying maladaptive decision-making. However, the specific role of dopamine in intertemporal decisions is not well understood. Dopamine may play a role in multiple aspects of intertemporal choices—the valuation of choice outcomes and sensitivity to reward delays. To assess the role of dopamine in intertemporal decisions, we tested Parkinson disease patients who suffer from dopamine depletion in the striatum, in either high (on medication, PDON) or low (off medication, PDOFF) dopaminergic states. Compared with both PDOFF and healthy controls, PDON made more farsighted choices and reduced their valuations less as a function of increasing time to reward. Furthermore, reduced discounting in the high dopaminergic state was robust across multiple measures, providing new evidence for dopamine's role in making decisions about the future.
Intertemporal decisions—choices between sooner smaller and larger later rewards—are ubiquitous. Studies widely agree that the striatum as well as the pFC (particularly the ventromedial and the lateral pFC) are centrally involved in intertemporal choice (Figner et al., 2010; Kable & Glimcher, 2007, 2010; Pine, Shiner, Seymour, & Dolan, 2010; McClure, Laibson, Loewenstein, & Cohen, 2004). Notably, these frontostriatal circuits are heavily innervated by dopamine (Dagher & Robbins, 2009; Dalley, Mar, Economidou, & Robbins, 2008). Understanding how dopamine modulates intertemporal choice has important implications for neurobiological models of this process, as well as for understanding the mechanisms underlying maladaptive decision-making (such as occurs in addiction). However, the specific role of dopamine in intertemporal decisions is not well understood.
Pharmacological manipulations allow insight into the causal role of dopamine in intertemporal decisions. One approach to studying the role of dopamine in decision-making is to use Parkinson disease as a model system. In its early stages, Parkinson disease involves degeneration of dopaminergic inputs mainly to the striatum, providing a test of the importance of striatal dopamine in decision-making. Importantly, because Parkinson disease is treated with dopamine replacement therapy, which can be withheld for a short period of time, patients can be tested in either high or low dopaminergic states, allowing assessment of the effects of dopamine while keeping more general disease-related factors constant.
Existing pharmacological studies have yielded complex and inconsistent results. In healthy, young adults, dopamine precursor administration (L-dopa) led to more shortsighted choices in one study (Pine et al., 2010), whereas selectively increasing pFC dopamine led to more farsighted choices in another study (Kayser, Allen, Navarro-Cebrian, Mitchell, & Fields, 2012), possibly because of different roles of striatal versus pFC dopamine. In contrast, studies of Parkinson disease patients (without impulse control problems) found no effects of dopaminergic state (Simioni, Dagher, & Fellows, 2012; Milenkova et al., 2011).
One source of such inconsistencies could be that previous studies used task paradigms that differed in their sensitivity. Investigations of intertemporal decision-making have typically used a single metric of discounting derived from pairwise choices between sooner and later rewards. Here we used two tasks that are commonly used to assess pairwise intertemporal choices, a binary choice task and a choice titration task, and a task requiring valuations of single choice options. Additionally, we used an analysis approach that allowed us to separately assess the influence of reward magnitude and reward delay. We compared early-stage Parkinson disease patients—either on or off dopaminergic medication—to a healthy control group. This allowed us to investigate how intertemporal decision-making is affected by (i) striatal dysfunction in general, comparing Parkinson disease patients to controls, or (ii) changes in dopamine levels, comparing Parkinson disease patients on versus off dopaminergic medication.
Thirty-three participants with a diagnosis of idiopathic Parkinson disease were recruited from the Center for Parkinson Disease and Other Movement Disorders at Columbia University Medical Center. Only patients in early stages of disease (Hoehn-Yahr Stages 1–3), when the striatum is the primary site of dopaminergic dysfunction (Cools, 2006; Hornykiewicz, 1989; Kish, Shannak, & Hornykiewicz, 1988; Bernheimer, Birkmayer, Hornykiewicz, Jellinger, & Seitelberger, 1973), were recruited. Patients were randomly assigned to be tested either on medication (PDON; n = 16) or off medication (PDOFF; n = 17). Twenty healthy controls, matched to Parkinson disease patients on age and education (see Table 1), were recruited from the community surrounding Columbia University. All participants provided informed consent in accordance with the guidelines of the institutional review board of Columbia University and were paid $12/hr for their participation, in addition to a variable payment based on their choices (see below).
|.||Controls 18 (5/13)a .||PDOFF17 (7/10)a .||PDON14 (6/8)a .||p .|
|M .||SD .||M .||SD .||M .||SD .|
|.||Controls 18 (5/13)a .||PDOFF17 (7/10)a .||PDON14 (6/8)a .||p .|
|M .||SD .||M .||SD .||M .||SD .|
Demographic characteristics were compared between groups using independent sample t tests.
NAART = North American Adult Reading Test; COWAT = Controlled Oral word Association Test; BDI = Beck Depression Inventory; UPDRS = Unified Parkinson disease Rating Scale; H-Y scale = Hoehn and Yahr scale.
Significant difference between PDOFF and PDON.
Significant difference between Controls and PDON.
Significant difference between Parkinson disease patients and Controls.
Note that not all Parkinson disease patients are on L-dopa.
Note that not all Parkinson disease patients are on agonists.
Participants were excluded if they had suffered brain injury or were diagnosed with neurological or psychiatric disorders other than Parkinson disease. Participants completed a neuropsychological battery to assess general cognitive functioning and were excluded if they exhibited impairment, evidenced by scoring 26 (2.5 SDs below the mean) or below on the Mini Mental State Examination (MMSE). Data from two Parkinson disease participants were excluded (both tested on medication)—one because of initially unreported brain surgery and one because of low MMSE score, leaving 17 PDOFF and 14 PDON participants. Two age-matched controls were excluded because of low MMSE scores, leaving 18 control participants in the analyses.
PDON were tested within 3 hr of their last medication dose; PDOFF were withdrawn overnight from their medications and tested at least 14 hr after their last medication dose, following prior studies (Foerde, Braun, & Shohamy, 2013; Shohamy, Myers, Geghman, Sage, & Gluck, 2006; Frank, Seeberger, & O'Reilly, 2004; Cools, Barker, Sahakian, & Robbins, 2001). As is common clinical practice, most patients were being treated with a combination of L-dopa and dopamine agonists. PDON/PDOFF: 10/16 treated with L-dopa alone or in combination with an agonist, 2/0 on an agonist alone, and 2/1 on an MAOI (e.g., rasagiline) alone; of those treated with agonists, 4 PDON and 6 PDOFF were treated with D2/D3 agonists (pramipexole, ropinirole). In addition 3 PDON and 5 PDOFF were on antidepressant or antianxiety medication. The total daily L-dopa equivalent dosage (LED) was calculated for each participant according to Tomlinson et al. (2010; Table 1). Precise medication dosage information could not be obtained from 1 PDON and 1 PDOFF, and LED was not calculated for these participants.
Comparison of PDON and PDOFF groups revealed that the groups did not differ in disease duration or severity (see Table 1). There were, however, differences in age, education, and total digit span (a measure of working memory), with PDON being older, more educated, and having lower digit spans than PDOFF. To investigate whether any of the results reported below might be due to group differences on these variables, we report not only the results of our main analyses (without covariates) but also the results of additional analyses that included age, education, and digit span as covariates of no interest (because of 2 PDON missing digit span scores, the respective analyses use a somewhat reduced data set).
Participants completed three tasks: a binary intertemporal choice task, a valuation rating task, and a choice titration task, all of which required choices between or assessments of smaller-sooner (SS) and larger-later (LL) rewards, following Figner et al. (2010).
Participants completed all tasks twice. The two test sessions were separated by about an hour, during which participants completed a neuropsychological test battery and an unrelated task. Tasks were identical across the two administrations, except that the reward amounts were slightly jittered by up to ±$3 to reduce memory effects. Additionally, SS rewards in the choice tasks were always presented on the left side of the screen in the first task and on always on the right side of the screen in the second task. It has been proposed that the ordering of SS and LL choice options could influence the degree of discounting (Weber et al., 2007). However, choice option ordering did not affect choice patterns and was not considered further in analyses (main effect of SS/LL ordering: p = .35; all interactions with SS/LL ordering: ps > .28). Four PDON, 1 PDOFF, and 1 Control completed only the first task administration; our analysis approach (see below) allowed us to keep these participants' data in the analyses.
Binary Choice Task
The first task in each of the two test sessions was the binary choice task. Binary choice tasks are commonly used in intertemporal choice research. However, the task used in this study assessed not only overt choices but also sensitivity to changes in relative amount differences, time differences, and immediacy, potentially shedding additional light on the mechanisms underlying differences in overt choices. In each of 36 choice trials per task administration (i.e., 72 total trials), participants chose between an SS and an LL option (Figure 1A). Across trials, the following factors were varied factorially: (i) the Relative Difference between rewards (the LL was either 0.5%, 1%, 10%, 15%, 20%, 25%, 30%, 50%, or 75% larger than the SS); (ii) the Time Difference between SS and LL (time of delivery of the LL was either 2 or 4 weeks later than time of delivery of the SS); (iii) the SS was either an immediate reward (available today: “now trial”) or a future reward (available in 2 weeks: “not-now” trial) (Now/Not-now). Additionally, but not part of the factorial variation and not of theoretical interest here, (iv) the SS amounts differed across trials (Outcome magnitude), as SS amounts were (pseudo)randomly drawn when designing the task (see Figner et al., 2010; McClure et al., 2004, from which the binary choice task was originally developed). In total, the presented amounts varied between $15 and $84.50 (across both SS and LL outcomes). Participants used a mouse to select their preferred option and clicked a “Next” button to proceed to the following trial (Figure 1A). One PDON and 2 PDOFF required some assistance making responses; they pointed to their answer on the computer screen, and the experimenter handled the mouse.
Valuation Rating Task
After the choice task, participants completed a valuation task that assessed the subjective value of outcomes as a function of reward magnitude (Outcome magnitude) and time of delivery (Time) through explicit rating judgments in a nonchoice valuation context (Figure 1B). In each valuation task, participants rated 12 single options (i.e., 24 total ratings), one at a time, for their attractiveness. The options were from the Choice Task and reflected the four time points used in the Choice Task (“Today,” “2 weeks,” “4 weeks,” and “6 weeks”) crossed with low (approx. $30), medium (approx. $45), and high (approx. $60) reward magnitudes (the amounts were jittered somewhat around these approximate numbers). Participants rated each option on a continuous visual analog scale with the endpoints “Very unattractive” (specified as “$20 in 6 weeks”) and “Very attractive” (specified as “$80 today”).
Choice Titration Task
Next, participants completed two “titrator” trials per task (i.e., 4 “titrators” total) in which they made a series of binary choices between a constant SS reward and a stepwise changing LL reward, presented in a table-like format (Figure 1C). On the first titrator trial, participants chose between an SS reward of $50 today and an LL reward ranging from $55 to $105 in $5 increments delivered in 3 months (“Now” titrator). On the second titrator trial, participants chose between an SS reward of $30 delivered in 2 months and an LL reward ranging from $35 to $85 in $5 increments delivered after 5 months (“Not now” titrator). The choice titration task allowed participants to indicate in a very simple format their indifference point between the SS and LL outcome.
All trials across all phases of the experiment were self-paced to eliminate time pressure related to motor difficulty in the Parkinson disease patients.
Participants were instructed that one trial would be randomly selected at the end of the experimental session and the choice on that trial paid out: If a trial with an immediate reward choice was selected, the amount was paid in cash at the end of the testing session; if a delayed reward trial was selected, the amount was sent to participants by mail 3 days before the specified time of delivery.
Binary choice data were analyzed with generalized mixed-effects models using the glmer function of the lme4 package (version 1.1-7) in R (Bates, Maechler, Bolker, & Walker, 2014). The repeated-measures nature of the data was taken into account by including a per-participant random intercept and modeling all within-subject predictors both as fixed effects and random slopes varying over participant, plus all possible random correlation terms, incorporating a “maximal” random effects structure, as recommended by Barr, Levy, Scheepers, and Tily (2013), to avoid inflated Type 1 errors. p values were determined using likelihood ratio tests using the function mixed of the package afex (Singmann, 2014). As a general strategy, we first used an omnibus model investigating Group differences between Controls, PDON, and PDOFF. Significant Group effects (see below) and interactions of Group with task factors were further investigated using pairwise follow-up models (Control vs. PDON, Control vs. PDOFF, PDON vs. PDOFF).
Valuation rating data were analyzed with an analogous approach, except that we used Gaussian (instead of binomial) mixed-effects models appropriate for the continuous response measure, using the lmer function of the lme4 package; p values were again determined using likelihood ratio tests.
Choice titration data were analyzed similarly to Weber et al. (2007) but used a hyperbolic discounting model. The discount rate k was calculated for each of the four titrator trials based on the point at which participants switched from the SS to the LL: For example, if a participant preferred $50 today over $60 in 3 months but preferred $65 in 3 months over $50 today, we assumed that their point of indifference was between $30 today and $62.50 in 3 months, determining their discount rate k for this titrator. Because the k values were highly internally consistent across the four titrators with a Cronbach's alpha of .9, we computed the average k per participant and used this value to investigate group differences using linear regression together with post hoc pairwise group comparisons using the glht function from the library multcomp (Hothorn, Bretz, & Westfall, 2008) with Tukey contrasts and adjusted p values.
Additional analyses then investigated whether any of the observed Group results might be due to group differences in covariates of no interest (Age, Education, Digit Span). Digit span scores were not obtained for two PDON participants, resulting in reduced sample sizes for these analyses.
To facilitate comparison of temporal discounting across tasks and with previous studies, we estimated the discount rate (k) for each task. For the binary choice task, we estimated for each participant separately their best-fitting k using R's glm.fit function, predicting their binary choices as a function of the implied discount rate for each choice trial (i.e., if the participants had been indifferent between the two options offered on that trial). For the valuation rating task—as for the binary choice and the titrator tasks—the best-fitting k was estimated using Mazur's (1987) standard one-parameter model of hyperbolic discounting (see Figner et al., 2010). Numerous previous studies, including those in patients with Parkinson disease (Simioni et al., 2012; Milenkova et al., 2011; Housden, O'Sullivan, Joyce, Lees, & Roiser, 2010), have assumed hyperbolic discounting. For the valuation ratings, separate linear regressions were run step-wise for a broad range of ks (ranging from −50 to 200 in steps of 0.05) for each participant, and the k that best explained their pattern of subjective values represented in the valuation ratings was selected as their valuation-implied discount rate k (based on the linear regressions' R2 value). We also tested whether an exponential model would be more appropriate for the valuation rating data and found that R2 values were significantly larger for the hyperbolic than the exponential model (Wilcoxon signed rank test, p < .001; Mhyp = 0.64, SD = 0.21; Mexp = 0.63, SD = 0.21), whereas there was no significant difference between the two models in how well they predicted participants' choices in the binary choice task (Wilcoxon signed rank test, p = .105). This slight superiority of the hyperbolic compared with the exponential model is consistent with a previous study using these tasks (Figner et al., 2010). For the choice titration task, k was calculated as outlined above. Finally, pairwise correlations between ks from each task were computed across participants.
Binary Choice Task
All task factors except Now/Not-now showed clear and significant main effects in the expected directions. LL choice increased as the Relative Difference between options increased (χ2(1) = 36.20, p < .001) and as Outcome magnitude increased (χ2(1) = 55.89, p < .001, a pattern known as the “magnitude effect” in intertemporal choice (Thaler, 1981). LL choices decreased as Time Difference increased, and participants had to wait longer for the LL (χ2(1) = 29.28, p < .001). Whether the SS was an immediate or future reward (Now/Not-now trial) did not influence choice significantly (χ2(1) = 0.04, p = .83). This pattern of significant main effects was observed in all pairwise follow-up models and will not be further reported on in detail; thus, for the remainder of the choice analyses, we focus on Group differences.
Overall, there was a significant effect of Group (χ2(2) = 7.66, p = .02), indicating that disease and/or medication status affected choices between sooner versus later rewards. Pairwise follow-up models revealed that this effect was driven by greater preference for LL rewards in the PDON group relative to both Controls and PDOFF (Figure 2; PDON vs. Control: χ2(1) = 5.65, p = .02; PDON vs. PDOFF: χ2(1) = 6.25, p = .01). PDOFF and Controls did not differ significantly from each other (PDOFF vs. Control: χ2(1) = 0.1, p = .75).
Group showed significant interactions with Time Difference (χ2(2) = 7.74, p = .02) and Outcome magnitude (χ2(2) = 8.95, p = .01), whereas interactions with Relative Difference and Now/Not-now were not significant (χ2(2) = 1.29, p = .53; χ2(2) = 2.48, p = .29, respectively). Pairwise follow-up models revealed that the significant Time Difference interaction was due to the Controls significantly differing from both PDON (χ2(1) = 4.39, p = .04) and from PDOFF (χ2(1) = 6.53, p = .01): In Controls, increasing the Time Difference from 2 to 4 weeks led to a somewhat steeper drop in LL choices, compared with the PDON and the PDOFF groups. PDON and PDOFF did not differ significantly from each other in the effect of Time Difference on choice (χ2(1) = 0.01, p = .97 (Proportion LL choices: Control M(2weeks) = 0.40, M(4weeks) = 0.30, PDOFFM(2weeks) = 0.37, M(4weeks) = 0.31, PDONM(2weeks) = 0.69, M(4weeks) = 0.61). Thus, this result suggests that Controls may be more sensitive to time information than Parkinson disease patients both on and off medication. The follow-up models showed that the significant Group difference in the magnitude effect was due to the PDON group showing a weaker magnitude effect than the other two groups (PDON vs. Control: χ2(1) = 5.63, p = .02; PDON vs. PDOFF: χ2(1) = 4.65, p = .03); Controls and PDOFF did not differ significantly on this dimension (χ2(1) = 0.16, p = .69).
When we reran the above models including the covariates Age, Education, and Digit Span, the Group effect was virtually the same as in the analysis without covariates (χ2(2) = 9.03, p = .01), and none of the three covariates showed a significant effect on choice (Age: χ2(1) = 0.43, p = .51; Education: χ2(1) = 2.68, p = .10; Digit Span: χ2(1) = 0.02, p = .89).
The results of the pairwise follow-up analyses with covariates also did not differ substantially from the models without covariates. As two pairwise follow-up models with covariates had to be simplified to reach convergence due to the increased model complexity and/or reduced sample size, we followed suggestions by Barr et al. (2013) for simplification. As in the main analysis, the PDON group chose the LL reward significantly more often relative to both Controls and PDOFF (PDON vs. Control: χ2(1) = 6.39, p = .01; PDON vs. PDOFF: χ2(1) = 7.74, p = .005). PDOFF and Controls did not again differ significantly from each other (PDOFF vs. Control: χ2(1) = 0.08, p = .78). All other results remained qualitatively the same as well, with the exception that the interactions between PDON versus Controls and (i) Time Difference and (ii) Outcome magnitude only approached significance in the models with covariates (Time Difference: χ2(1) = 2.65, p = .10; Outcome magnitude: χ2(1) = 2.87, p = .09). Thus, we cannot rule out that these latter two Group differences were partly due to group differences in the covariates (alternatively the reduced sample size and/or increased model complexity might explain the differences in p values).
The valuation task assessed the subjective value of reward magnitude (Outcome magnitude) and time of delivery (Time) through explicit rating judgments of the attractiveness of single options presented in a nonchoice context (Figure 1B).
As for the binary choice data, there were significant main effects of Outcome magnitude and Time in the expected directions, with higher Outcome magnitude and earlier time of delivery leading to higher attractiveness ratings (Outcome magnitude: χ2(1) = 85.41, p < .001; Time: χ2(1) = 63.63, p < .001; Figure 3). Although there was no significant main effect of Group in the overall option valuation ratings (χ2(2) = 3.90, p = .14) suggesting no overall Group differences in perceived attractiveness of the rewards, there was a significant interaction between Group and Time (χ2(2) = 13.24, p = .001): Pairwise follow-up models revealed that the interaction was due to the PDON group being significantly less sensitive to increasing time (see Figure 3A) compared with both PDOFF (χ2(1) = 5.64, p = .02) and Controls (χ2(1) = 15.39, p < .001); that is, attractiveness declined less steeply with increasing time of delivery in PDON compared with the other two groups. PDOFF and Controls did not differ significantly in their Time effect (χ2(1) = 1.58, p = .21). Additionally, pairwise follow-up models revealed that, although the main effect of Group was not significant, the PDOFF Group differed marginally significantly from the other groups in that they exhibited overall lower valuations (PDON vs. PDOFF: χ2(1) = 3.26, p = .07; PDOFF vs. Controls: χ2(1) = 3.26, p = .07). PDON and Controls did not differ from each other (χ2(1) = 0.23, p = .63).
To investigate whether the results might be due to group differences in Age, Education, or Digit Span, we ran the main model with these covariates added. Consistent with the choice analysis, none of the covariates showed a significant main effect (Age: χ2(1) = 1.54, p = .21; Education: χ2(1) = 0.003, p = .96; Digit Span: χ2(1) = 2.01, p = .16). The Group effect remained nonsignificant (χ2(2) = 3.81, p = .15), and the effects of Time and Outcome magnitude as well the interaction between Group and Time were still significant (Amount: χ2(1) = 83.90, p < .001; Time: χ2(1) = 59.89, p < .001; Group × Time: χ2(2) = 11.42, p = .003). Given these virtually identical results, we did not compute the pairwise follow-ups with covariates.
Taken together, these valuation results suggest that dopaminergic status affected time sensitivity in valuations. In particular, the PDON group reduced their valuations less than did the Control and PDOFF groups as time to reward increased (Figure 3A suggests that this was the case particularly beyond 2 weeks). The results observed for valuation ratings are consistent with those observed for PDON in the binary choices and could suggest that a decreased sensitivity to delays may contribute to more farsighted behavior in PDON.
PDON differed significantly from controls (t = 2.47, p = .045), whereas no other groups differed significantly (PDON vs. PDOFF: t = 1.68, p = .22; PDOFF vs. Control, t = 0.81, p = .70). Thus, the choice titration results were consistent with the main results observed across multiple binary choices described above, that PDON showed a greater preference for LL rewards.
Discount Rate (k)
Previous studies of intertemporal decision-making have primarily focused on a single outcome measure—the discount rate k. To allow comparison with previous studies, we report the estimated mean discount rates for each group on each task in Table 2. Importantly, the same pattern of results seen in the previous analyses is observed when considering k as the outcome measure (Table 2). However, caution is warranted when comparing absolute values of k across studies. The particular value of k obtained is not likely to be meaningful, as various task factors may influence the discount rate. For example, whether one considers reward receipt in an acceleration or delay framework significantly alters discounting (Weber et al., 2007). Nonetheless, when we assessed the consistency of discount rates across tasks, by computing pairwise correlations between the discount rates (k) for each task, we found that k was significantly correlated across all tasks, regardless of variation in mean k (see Figure 4). In the valuation rating task, two participants had extreme estimated k values (>2.5 SD above mean). However, we obtained the same results when we excluded these outliers from the correlation analyses (Choice titration k × Valuation rating k: r = .32, p = .028; Binary choice k × Valuation rating k: r = .51, p = .0003), or when we used Spearman's rank-order correlation (Choice titration k × Valuation rating k: rho = .42, p = .003; Binary choice k × Valuation rating k: rho = .52, p = .0001; Binary choice k × Choice titration k: rho = .81, p < .0001). Furthermore, k values were also correlated with the proportion later choices in the binary choice task (Choice titration k × Proportion LL choices: r = −.76, p < .0001; Valuation rating k × Proportion LL choices: r = −.41, p = .0042; negative correlations are expected as higher proportions of later choices should be associated with lower discount rates k). As seen in Figure 4, the correlations were highest between the binary choice and titration choice tasks and lowest with the valuation ratings task. This is not surprising given the similarity between the binary choice and choice titration tasks—simultaneous presentation of two choice outcomes with discrete values, whereas the valuation rating task presents a single outcome and responses are made on a continuous rating scale.
|Task .||Controls .||PDOFF .||PDON .|
|M .||SD .||M .||SD .||M .||SD .|
|Task .||Controls .||PDOFF .||PDON .|
|M .||SD .||M .||SD .||M .||SD .|
We found that modulation of dopamine levels in Parkinson disease patients affected intertemporal decision-making. Compared with both patients off medication and healthy controls, Parkinson disease patients on dopaminergic medication exhibited greater farsightedness in their choices. This pattern of behavior was apparent both in greater likelihood of choosing LL rewards in binary choices between SS and LL rewards and in reduced discounting of LL rewards, as assessed via the valuation ratings of single outcomes. In particular, Parkinson disease patients on dopaminergic medication showed decreased sensitivity to increasing delays to rewards. Patients withdrawn from dopaminergic medication showed a trend toward reduced valuation of single outcomes but did not differ significantly in their binary choices from healthy controls. Additionally, individual discount rates were correlated across all three measures of intertemporal decisions. In summary, these results implicate dopamine across multiple methods for querying intertemporal choice preferences.
Locus of Dopamine Effects
Our results clearly suggest that loss of striatal dopamine per se—the central feature of Parkinson disease in its early to moderate stages—does not affect intertemporal decision-making uniformly. Rather, differences in dopamine levels as a function of being on versus off dopaminergic medication impacted choice behavior. Dopamine is necessary for functioning of both the striatum and pFC, regions known to be involved in intertemporal decisions (for a review, see Carter, Meyer, & Huettel, 2010). Open questions remain regarding the specific role of each region: By one account, the striatum is particularly engaged when rewards are available immediately, whereas pFC is engaged regardless of delay and particularly when decisions are more difficult (McClure et al., 2004). Another account has the striatum and medial pFC representing the subjective value of rewards regardless of delay and their immediacy (Kable & Glimcher, 2007, 2010). Although the striatum may respond preferentially to immediate rewards, it may additionally be more responsive to delayed rewards with increasing age (Samanez-Larkin et al., 2011). Our results are compatible with both points of view and, together with prior work showing increased impulsivity in young adults administered L-dopa (Pine et al., 2010), suggest that increasing dopamine levels in the striatum may affect young and old adults differentially, with greater benefits for farsighted choice in older adults (as found in the current study).
Previous work has causally implicated the lateral pFC in farsighted decisions (Figner et al., 2010), and pFC dopamine levels may influence baseline activity levels underlying individual differences in intertemporal choice (Gianotti, Figner, Ebstein, & Knoch, 2012). It is tempting to speculate that dopaminergic modulation of the lateral pFC could have played a role in biasing the choices of Parkinson disease patients on medication toward farsightedness, consistent with results reporting increased patience because of selective increases in pFC dopamine (Kayser et al., 2012). Because pFC receives dopaminergic projections from the ventral tegmental area, which is not severely affected in the early stages of Parkinson disease (relative to the substantia nigra), it is possible that dopaminergic medication enhances dopamine levels in pFC in Parkinson disease patients to an extent that results in farsightedness even beyond that of healthy controls.
Finally, connectivity between striatum and pFC may be relevant. Stronger structural and functional connectivity between striatum and lateral pFC has been linked to greater patience across individuals (Van den Bos, Rodriguez, Schweitzer, & McClure, 2014), and functional connectivity is influenced by dopamine: Pharmacologically increasing dopamine in pFC affects connectivity with the striatum in intertemporal choice (Kayser et al., 2012), and functional connectivity between striatum and lateral pFC during rest is enhanced by L-dopa relative to placebo (Kelly et al., 2009). Thus, enhanced connectivity through dopaminergic medication could also contribute to the results we report here.
Dopamine and Consideration of Future Events
Studies investigating the link between dopamine and consideration of future events in decision-making have yielded mixed results. Our results are consistent with findings in many pharmacological studies that increasing dopamine levels leads to more patient choices, whereas decreasing dopamine levels leads to decreases in choice of delayed rewards or willingness to exert effort for larger rewards (Kayser et al., 2012; Salamone & Correa, 2012; Floresco, Tse, & Ghods-Sharifi, 2008; Bizot et al., 2007; Salamone, Correa, Farrar, & Mingote, 2007; van Gaalen, van Koten, Schoffelmeer, & Vanderschuren, 2006; Pietras, Cherek, Lane, Tcheremissine, & Steinberg, 2003; de Wit, Enggasser, & Richards, 2002; Cardinal, Robbins, & Everitt, 2000; Wade, de Wit, & Richards, 2000; Richards, Sabol, & de Wit, 1999). In particular, a recent study showed that administration of methylphenidate, which increases extracellular levels of dopamine in the BG and pFC, decreased temporal discounting in nonhuman primates (Rajala, Jenison, & Populin, 2015). However, other studies have found that increases in dopamine lead to decreases in patient choices (Pine et al., 2010; Evenden & Ryan, 1996; Logue et al., 1992) or to no differences (Acheson & de Wit, 2008; Hamidovic, Kang, & de Wit, 2008). These findings may be reconciled by the idea that the relationship between dopamine levels and performance on a given task follows an inverted U-shape function (Joutsa et al., 2015; Cools & D'Esposito, 2011) such that individual or population level differences in baseline dopamine influence whether manipulating dopamine levels will be beneficial or detrimental. Consistent with this idea, some studies found dose- or population-dependent effects (Floresco et al., 2008; Bizot et al., 2007; Cardinal et al., 2000). Such baseline differences (see also Gianotti et al., 2012; Eisenegger et al., 2010) may explain why Pine et al. (2010) found that L-dopa increased temporal impulsivity. In healthy young participants, administering L-dopa may result in overdosing of frontostriatal circuits depending on individual differences in baseline dopamine levels (Kayser et al., 2012; Clatworthy et al., 2009; Cools, Sheridan, Jacobs, & D'Esposito, 2007). In this study, medication dosage is titrated to each individual's need, perhaps allowing the enhanced patience often seen with moderate increases in dopamine levels.
The current results are also broadly consistent with a recent study investigating the role of dopamine in goal-directed behavior. Using a two-step reward-learning task that allowed computational dissection of habitual versus goal-directed control, Parkinson disease patients on and off L-dopa were compared in a within-subject design. Goal-directed choices were impaired in Parkinson disease patients off medication, but performance was fully restored to the level of healthy controls when Parkinson patients were on medication (Sharp, Foerde, Daw, & Shohamy, 2016). Interestingly, a separate study assessed the relationship between goal-directed learning on the same two-step reward-learning task used by Sharp et al. (2016) while also assessing intertemporal choice in the same participants (Hunter, Shi, Soufian, Decker, & Hartley, 2015). They found that goal-directed learning was predictive of greater patience in intertemporal decisions. Thus, dopamine may promote behaviors that involve prospection in some form, as is the case for making farsighted choices and predictions about future rewards based on complete models of the environment, potentially by simulating paths to future outcomes (Doll, Duncan, Simon, Shohamy, & Daw, 2015).
Impulsivity in Parkinson Disease
A considerable number of studies have found that Parkinson disease patients diagnosed with impulse control disorders show signs of temporal impulsivity and increased reward sensitivity (Voon & Dalley, 2011; Housden et al., 2010; Voon & Fox, 2007), in particular when treated with dopamine agonists (Voon, Mehta, & Hallett, 2011). Only a few studies have looked at temporal discounting in Parkinson disease patients without impulse control disorders: Two studies found no difference between Parkinson disease and controls (Simioni et al., 2012; Housden et al., 2010), and one study found differences in discounting between Parkinson disease and controls (Milenkova et al., 2011), that is, increased impulsivity in Parkinson disease patients both on and off medication. Two of the aforementioned studies made within-subject comparisons of Parkinson disease patients on and off medication, but found no effects of dopaminergic medication (Simioni et al., 2012; Milenkova et al., 2011). There are several differences between studies that could have contributed to the discrepant results. Milenkova et al. specifically assessed patients taking D2/D3 agonists, which are associated with the development of impulsivity in Parkinson disease (Dagher & Robbins, 2009), whereas only a small subset of participants in our study were on D2/D3 agonists (see Methods section). Another potential difference between our study and previous work is that we investigated multiple factors contributing to choices (i.e., sensitivity to delay and magnitude of outcomes) and trial-by-trial analyses of behavior, which may be more sensitive to differences in discounting. Future studies should measure multiple components of intertemporal choice while accounting for individual differences in baseline dopamine to fully understand the complex effects of dopamine on choice. Additionally, the current study employed relatively small group sizes that were not matched on age, education, and digit span. However, the inclusion of age, education, and digit span as covariates in our analyses did not change the results, and subsample analysis on groups matched for age and education (not reported) also yielded results consistent with those reported here. Thus, despite the inclusion of a relatively small sample that is often characteristic of patient studies, these additional analyses suggest that our results are reliable.
Our results show that dopamine, administered in doses that are titrated to an individual to alleviate motor symptoms associated with Parkinson disease, can enhance farsighted choice in patients who do not suffer from impulse control disorders. In particular, our results suggest that dopamine influences sensitivity to delays when evaluating larger later reward outcomes. These results suggest that dopamine plays a crucial role in processes supporting intertemporal decision-making.
We are grateful to Drs. Lucien Cote, Paul Greene, and Roy Alcalay at the Center for Parkinson Disease and Other Movement Disorders at Columbia University Medical for recruitment of participants with Parkinson disease and to Nina Rouhani for assistance with data collection. This work was supported by NIH-NINDS NRSA 5F32NS063632 to K. F., NINDS R01 GG006201 to D. S., and an NSF Career Development Award GG002716 to D. S.
Reprint requests should be sent to Karin Foerde, Department of Psychology, New York University, 6 Washington Place, Rm 277, New York, NY 10003, or via e-mail: firstname.lastname@example.org or Bernd Figner, Behavioural Science Institute/Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, PO Box 9104, 6500 HE Nijmegen, The Netherlands, or via e-mail: email@example.com.
These authors contributed equally to this work.