Abstract

Humans carry out many daily tasks in a seemingly automatic fashion. However, when unexpected changes in the environment occur, we have the capacity to inhibit prepotent behavior and replace it with an alternative one. Such behavioral flexibility is a hallmark of executive functions. The neurotransmitter dopamine is known to be crucial for fast, efficient, and accurate cognitive flexibility. Despite the perceived similarities between cognitive and motor flexibility, less is known regarding the role of dopamine within the motor domain. Therefore, the aim of this study was to determine the role of dopamine in motor flexibility. In a double-blind, five-session, within-subject pharmacological experiment, human participants performed an RT task within a probabilistic context that was either predictable or unpredictable. The probabilistic nature of the predictable context resulted in prediction errors. This required participants to replace the prepotent or prepared action with an unprepared action (motor flexibility). The task was overlearned, and changes in context were explicitly instructed, thus controlling for contributions from other dopamine-related processes such as probabilistic or reversal learning and interactions with other types of uncertainty. We found that dopamine receptor blockade by high-dose haloperidol (D1/D2 dopamine receptors) impaired participants' ability to react to unexpected events occurring in a predictable context, which elicit large prediction errors and necessitate motor flexibility. This effect was not observed with selective D2 receptor blockade (sulpiride), with a general increase in tonic dopamine levels (levodopa), or during an unpredictable context, which evoked minimal prediction error. We propose that dopamine is vital in responding to low-level prediction errors about stimulus outcome that requires motor flexibility.

INTRODUCTION

Much human behavior is executed in a seemingly automatic fashion. These behaviors can be viewed as prepotent in that they take precedence over any other potential alternatives (Isoda & Hikosaka, 2011; Hikosaka & Isoda, 2010). When this behavior becomes inappropriate through an unexpected change in the environment, humans are capable of engaging resources that inhibit these prepotent responses and replace them with alternative ones. Such behavioral flexibility in a short period of time is a hallmark of executive functions (Isoda & Hikosaka, 2011; Hikosaka & Isoda, 2010).

There is considerable evidence that the neuromodulator dopamine plays a crucial role in behavioral flexibility (van der Schaaf et al., 2014; Stelzel, Fiebach, Cools, Tafazoli, & D'Esposito, 2013; Stelzel, Basten, Montag, Reuter, & Fiebach, 2010). Previous human work has highlighted the importance of dopamine in cognition-based switching tasks (van Holstein et al., 2011; Cools et al., 2009; Cools, Barker, Sahakian, & Robbins, 2001a), with continuing interest in the specific function of dopamine D1 and D2 receptors. Animal (Haluk & Floresco, 2009; Floresco, Magyar, Ghods-Sharifi, Vexelman, & Tse, 2006), theoretical (Durstewitz & Seamans, 2008), and human (Stelzel et al., 2013; van Holstein et al., 2011) work has suggested that D2 receptor signaling is an essential component for efficient cognitive flexibility. However, there is also considerable evidence that D1 receptor signaling plays a significant role (Ragozzino, 2002), with recent proposals that cognitive flexibility relies on a cooperative interaction of both D1 and D2 receptors (Floresco, 2013).

A distinction can be made between cognitive and motor flexibility (Stelzel et al., 2013). Whereas the former will generally involve a complex rule change (set-shift: naming digits following naming letters; Cools et al., 2001a), the latter requires the replacement of a prepared action with an unprepared one (Galea, Bestmann, Beigi, Jahanshahi, & Rothwell, 2012; Neubert, Mars, Buch, Olivier, & Rushworth, 2010). Motor flexibility has also been termed behavioral adaptation (Stelzel et al., 2013), motor-based behavioral switching (Hikosaka & Isoda, 2010) or action reprogramming (Galea et al., 2012; Neubert et al., 2010; Mars et al., 2009). We regard this terminology to reflect a similar process and therefore will use motor flexibility as a term that encompasses them all. Parkinson disease patients who suffer from a dopamine deficit show specific impairments in motor (Galea et al., 2012; Cools, van den Bercken, Horstink, van Spaendonck, & Berger, 1984) and cognitive flexibility tasks (Cools et al., 2001a; Beatty & Monson, 1990), with performance restored by dopaminergic medication (Galea et al., 2012; Cools, Barker, Sahakian, & Robbins, 2001b). Despite this work, the dissociable influence of D1 and D2 receptor signaling in motor flexibility is relatively unknown (Stelzel et al., 2013).

We addressed this issue by testing for the effects of a range of dopaminergic drugs on motor flexibility. Individuals use past experience to prepare movements in advance of upcoming events, and the degree of preparation is closely related to the predictability of a future event (Bestmann et al., 2008). Such preparation is advantageous when an event is predicted correctly but occurs at the expense of a prolonged RT when an unexpected event results in a prediction error. In this case, a prepared action has to be replaced by an unprepared alternative movement. Importantly, the greater the prediction error is, the greater the cost in terms of a prolonged RT (Galea et al., 2012; Bestmann et al., 2008).

The brain's sensitivity to prediction error has been explained by models that implement optimal Bayesian inference (Iglesias et al., 2013; Friston et al., 2012; Kording & Wolpert, 2006). Interestingly, indirect evidence from human neuroimaging studies suggests that brain areas associated with dopamine release are sensitive to low-level prediction errors about stimulus outcome (Iglesias et al., 2013). In contrast, prediction errors at high or abstract levels of uncertainty are thought to be encoded by other neuromodulatory systems (Yu & Dayan, 2002, 2005). Such hierarchical models therefore predict that dopamine may have an important role for responding to prediction errors, even in the absence of other learning-related types of uncertainty (volatility; Behrens, Woolrich, Walton, & Rushworth, 2007).

This study sought to isolate the contribution of dopamine for enabling motor flexibility in response to low-level prediction errors. To this end, we controlled for any role dopamine might have in the learning of probabilistic task contexts or reversal learning (Seo, Beigi, Jahanshahi, & Averbeck, 2010; Cools et al., 2009; Frank, Moustafa, Haughey, Curran, & Hutchison, 2007), by explicitly highlighting the current context to participants and providing them with extended practice on each context before the main experiment. This also ensured that participants did not experience volatility/unexpected uncertainty, that is, changes in the probability of the upcoming stimuli (Behrens et al., 2007; Yu & Dayan, 2002, 2005). Instead, participants experienced different degrees of expected uncertainty wherein they had advanced and stable knowledge about the overall predictability of the context (overall predictable context, PC, and overall unpredictable context, UC), but because of the probabilistic nature of each context a degree of irreducible uncertainty persisted. Therefore, participants knew about the context they were currently experiencing and consequently the (expected) uncertainty associated with each context but had to quickly respond to violations of these expectations in the form of rare and surprising events. Consequently, the prediction errors experienced in the PC would specifically examine the role of dopamine in responding to low-level stimulus prediction errors whose behavioral outcome is termed motor flexibility.

In a double-blind, five-session, within-subject design, participants performed a simple cued RT task (Galea et al., 2012). In the absence of learning, we found that combined D1/D2 receptor blockade (haloperidol) impaired participants' ability to react to unexpected events that elicit large prediction errors and thus require motor flexibility. This effect is neither observed with specific D2 receptor blockade (sulpiride) nor with a general increase in tonic dopamine levels (levodopa) or during an unpredictable context, which evoked minimal prediction error.

METHODS

Participants

Fifteen self-assessed, right-handed individuals with no current health problems or history of neurological/psychiatric illness (nine men, mean age = 27 ± 6 years) participated in the study, with written informed consent. The study was approved by the research ethics committee of the Institute of Neurology, University College London.

General Procedure

In a double-blind, placebo-controlled design, each participant took part in four experimental drug sessions, each separated by at least 1 week, in which they performed a probabilistic sequence RT task. Eight participants were involved in an additional fifth session. Session order was pseudorandomized so that placebo was evenly distributed across the five sessions. Participants sat in front of a computer screen positioned 30 cm away and placed each of the fingers of their right hand on one of four response buttons. At the start of each block, a screen was displayed to inform the participant whether the upcoming block consisted of either a UC or PC. Participants were therefore explicitly instructed about changes in the context. To focus participants' attention, an uninformative warning cue (white box) was then presented. This was followed by the presentation of one of four imperative stimuli (IS). Participants responded to the IS as quickly as possible, but not at the expense of accuracy. A fixation-cross followed (Figure 1A). Each imperative stimulus was associated with pressing a specific button (Figure 1A). These stimulus–response associations were learnt by the participants during a familiarization period (100 trials) at the beginning of the training session (see below) in which feedback was provided to signal whether their response was correct. Learnt stimulus–response associations were characterized by an error rate of less than 7%.

Figure 1. 

(A) Task. (B) Event probability matrices: Numbers represent transition probabilities. (C) Training: RT: The predictable (red) and unpredictable (blue) contexts were overlearned during training, with no significant differences between the end of training and placebo. (D) Training: surprise and MI: RT was positively correlated with surprise during the predictable context, whereas RT was negatively correlated with MI. In contrast, surprise and MI had no effect on RT during the unpredictable context. Error bars indicate standard error.

Figure 1. 

(A) Task. (B) Event probability matrices: Numbers represent transition probabilities. (C) Training: RT: The predictable (red) and unpredictable (blue) contexts were overlearned during training, with no significant differences between the end of training and placebo. (D) Training: surprise and MI: RT was positively correlated with surprise during the predictable context, whereas RT was negatively correlated with MI. In contrast, surprise and MI had no effect on RT during the unpredictable context. Error bars indicate standard error.

During the main experimental sessions, participants were exposed to 72 blocks of 12 trials (864 trials), which was equivalent to 40 min of testing. Blocks alternated between UC and PC. During UC, there was a .25 probability of each imperative stimulus being presented on trial t. In contrast, during the PC the current stimulus on trial t was conditionally dependent on the stimulus of the previous trial, t 1. This generated sequences in which the imperative stimulus order 1-2-3-4 occurred with high probability (Figure 1A, B). Because of the probabilistic nature of the sequence, however, occasional violations of this predominant sequence order occurred.

Dopamine is known to play a pivotal role in multiple learning processes such as reinforcement (Niv, Daw, Joel, & Dayan, 2007; Schultz, Dayan, & Montague, 1997) and probabilistic learning (den Ouden et al., 2013; Cools et al., 2009; Wilkinson, Khan, & Jahanshahi, 2009). Here, we wanted to test for the specific role of dopamine in behavioral flexibility and therefore sought to eliminate learning-related contributions of dopamine. Thus, before the main experimental sessions, all participants underwent a training session in which the task was overlearned. During training, participants were exposed to 72 blocks of 12 trials (864 trials), which was equivalent to 40 min of testing. To reiterate, at the start of each block, a screen was displayed to inform the participant whether the upcoming block consisted of either a UC or PC. At the end of training, the experimenters plotted the RT for the UC and PC. RT was averaged over every 12 trials (1 block) across training. Plateau performance was defined as participants showing no improvements in RT, which were greater than 10 msec from the previous UC or PC block. A plateau in RT performance was assumed to reflect overlearning (Figure 1C). For both the UC and PC, all participants exhibited plateau performance during the last 10 blocks. Training ensured participants had strong priors about the nature of each context and were able to switch between them on the basis of the explicit feedback. This was crucial in ensuring the results were specific to the role of dopamine in behavioral flexibility and not probabilistic or reversal learning (den Ouden et al., 2013; Cools et al., 2009).

Pharmacology

For each experimental session, participants arrived 2 hr before completing the task, and received either 100 mg of the dopamine precursor levodopa (levo), 400 mg of the D2 antagonist sulpiride (sulp), 2.5 mg of the D1/D2-antagonist haloperidol (halo2.5), or placebo (plac). Participants who underwent an additional fifth session were administered 1 mg of haloperidol (halo1). Haloperidol was administered 2 hr before the onset of the experiment, whereas levodopa and sulpiride were administered 1 hr prior. Placebo was administered randomly at either 1 or 2 hr before. The doses and administration times are similar to previous studies that have shown clear behavioral and neurophysiological effects for levodopa (Adam et al., 2012), sulpiride (Nitsche et al., 2006), and haloperidol (Frank & O'Reilly, 2006).

This combination of drugs was used in an attempt to partially dissociate the roles of dopamine D1 and D2 receptors during motor flexibility. Because selective D1-anatagonists available for human use have severe reported side effects (Hou & Schumacher, 2001), we exploited the hypothesized relative D1 and D2 receptor affinity differences between halo2.5, halo1, and sulp. Our approach was based on these drugs-predicted effects in the BG; however, it is important to emphasize that the role of the BG during this task must be taken with caution, simply because of the widespread changes the drugs have both in terms of receptor affinity and on different brain regions. First, sulpiride specifically blocks D2 receptors (O'Connor & Brown, 1982). Second, although haloperidol blocks D1 and D2 receptors, its affinity for D1 receptors is 25 times lower than for D2 (Bymaster et al., 1999). A dose of 1 mg should be sufficient to detect a behavioral impairment on tasks that are D2 receptor dependent (Fitzgerald, Kapur, Remington, Roy, & Zipursky, 2000). By contrast, a dose of 2.5 mg haloperidol ought to partially block both D1 and D2 receptors, while minimizing the extrapyramidal and sedative side effects that occur at higher doses. However, we are acutely aware that these differences between high- and low-dose haloperidol are not categorical. For instance, Fitzgerald et al. (2000) showed approximately 50% D2 occupancy at 1 mg and 75% at 2.5 mg. In addition, only a 14% increase in D1 occupancy has been found at 8.5 mg of haloperidol (Reimold et al., 2007), but in support of our hypothesis in vivo binding of D1 receptors was found to be 100% times greater than D2 receptors with haloperidol (Zhang & Bymaster, 1999). Finally, as haloperidol blocks the phasic uptake of dopamine, it can lead to increased levels of tonic dopamine in the BG (Kuroki, Meltzer, & Ichikawa, 1999). To dissociate these effects, we compared halo2.5 with levo (which increases tonic dopamine). Although we assume this task is mediated by the BG, one cannot exclude that any observed effects are mediated via the frontal cortex. As such, we note that although sulpiride led to increased tonic dopamine in the pFC, haloperidol did not (Li et al., 2005; Kuroki et al., 1999). Although it is not possible to directly test the isolated role of D1 receptors, this combination of drugs provided an insight into the possible independent roles of D1 and D2 receptors during motor flexibility.

At the end of each session, participants reported their attention and fatigue using a self-scored visual analog scale (1 = poorest attention/maximal fatigue; 7 = maximal attention/least fatigue) and reported whether they thought they had received an active or placebo drug.

Behavioral Analysis

For all correct responses, RT was calculated as the time between imperative stimulus onset and the subsequent button press. A Kolmogorov–Smirnov test was performed on the standardized data (subtracted by mean and divided by SD) across all trial types and participants. This test rejected the null hypothesis that the data were normally distributed at p < .05. Therefore, the data were log-transformed (Galea, Miall, & Woolley, 2007). A Kolmogorov–Smirnov test revealed the data now no longer significantly deviated from a normal distribution (p > .05). Using this log-transformed data, average RTs were calculated for each trial type (unpredictable, UC); predictable-expected, PC; predictable-unexpected, PC). The average RT from the UC was subtracted from both expected (predictable-expected) and unexpected (predictable-unexpected) trial types during the PC (ΔRT). This analysis mirrors the analyses in our previous work in Parkinson disease patients (Galea et al., 2012), allowed a simple comparison with minimal assumptions between sessions for the expected (>0.75) and unexpected (<0.12) trial types during the PC, and enabled us to quantify the cost of violation of expectations within the PC (Galea et al., 2012).

To assess whether the task was overlearned during training, the last 10 blocks of the training session were compared with the plac session. A within-subject repeated-measures ANOVA (rmANOVA) compared session (training, plac) with context (PC, UC).

For the main experimental sessions, we compared the percentage of incorrect button responses (error), average RT across trial types, and average ΔRT for expected and unexpected trial types during the PC. Within-subject rmANOVA compared these parameters across the four main Drug sessions (plac, levo, sulp, halo2.5). Because of multiple comparisons, a Bonferroni adjustment was made in which we accepted statistical significance at p < .025. Post hoc paired t tests explored significant effects (two-tailed). Because of only a subpopulation of participants being tested with 1 mg of haloperidol, halo1 was compared with halo2.5 using a paired t test. Estimates of effect size are given as partial eta squared (ηp2). All data presented represent mean ± standard error.

Event-related Surprise

In the current task, participants were required to respond as fast as possible but would always have some degree of uncertainty about the upcoming IS. Because of the probabilistic nature of the overlearned sequences, occasional violations would occur in the form of unexpected IS. For these surprising IS, participants had to respond against their prior expectation. Increases in RT to such surprising IS should then relate to the magnitude of the prediction error during the PC (Galea et al., 2012). In contrast, during the UC, when prior expectations would be overall small, occasionally more surprising IS occur against an overall UC, with little prediction error. We quantified the surprise enacted by a particular IS on a trial-by-trial basis. We estimated the conditional probability of each IS using a Bayesian update scheme (Harrison, Duggins, & Friston, 2006; Strange et al., 2005) in which we assumed that, at the beginning of each session, participants started with the prior expectation of all IS being equally likely. Note that, in each context, the overall probability of occurrence of all four stimuli was equal, and because these contexts had been overlearned and were explicitly signaled at every change, a uniform prior reflects the expectations participants should have.

For each trial (t) there were four possible IS. Therefore, the conditional probability of IS E at trial t, p(Et), was estimated from the number of occurrences of IS i up to trial t (written as nit, where i indexes the current IS type and t the trial number relating to the start of each session). Thus, the estimate at trial t is given by
formula
As a result of the first-order Markov sequence, the IS occurring on the previous trial, E(t 1), could be used to form predictions for the IS on trial t. An approximation of the joint probability distribution can be estimated from a count of IS pairs up to trial t (written as nijt, where i and j index the current and previous IS type) and is given by
formula
The degree of surprise conveyed by a particular IS pair is then quantified as
formula
The surprise (S) of observing IS type i on trial t after experiencing IS type j on trial t 1 is given by the negative log of its predicted joint probability. Therefore, surprise is a stimulus-specific measure that reflects the unexpectedness of the current IS, given the previous IS, that is, prediction error. The amount of surprise conveyed by the occurrence of an IS is high when an IS pair is infrequent. Accordingly during the PC surprise will be overall low, but occasional violations occur in the form of highly surprising infrequent IS pairs. Surprising trials also occur in the UC, but in this case, such IS appear in the context of an overall absence of predictability.

For training, we estimated surprise and mutual information (MI) independently for the PC and UC by separately combining these block types across a session. MI is a measure of learning that quantifies the predictability of the current trial, t, based on the IS presented on the previous trial, t 1 (Harrison et al., 2006). During the PC, MI steadily increases as the uncertainty of an IS type on trial t, which is afforded by an IS type on trial t 1, decreases. In contrast, MI remains low during the UC (Galea et al., 2012). To assess how RT was associated with MI and surprise on a trial-by-trial basis, a robust multiple regression (iteratively reweighted least squares) was performed with RT as the dependent variable and MI and surprise as independent variables. This produced β-values for surprise and MI in the PC and UC. A MANOVA compared β-values for surprise and MI between the PC and UC. Because of multiple comparisons, a Bonferroni adjustment was made in which we accepted statistical significance at p < .025.

As participants overlearned each context during training, only surprise was estimated for the main experimental data for the PC and UC by separately combining these block types across a session. Although participants did not receive any direct feedback regarding errors, we presumed they were still likely to be noticed. Despite trials in which an error occurred being removed, we hypothesized that post-error trials would still be influenced and exhibit the phenomenon of post-error slowing (Dutilh et al., 2012). To assess how RTs were associated with surprise and post-error trials on a trial-by-trial basis, independent robust multiple regressions (iteratively reweighted least squares) were performed for the PC and UC, with RT as the dependent variable and surprise and post-error trials as independent variables. A repeated-measures MANOVA compared the β-values for surprise and post-error trials across the four main Drug sessions (plac, levo, sulp, halo2.5) and Context (predictable, unpredictable). Because of multiple comparisons (surprise and post-error trials), a Bonferroni adjustment was made in which we accepted statistical significance at p < .025. Post hoc paired t tests explored significant effects with two-tailed scores being presented. Because of only a subpopulation of participants being tested with 1 mg of haloperidol, halo1 was compared with halo2.5 using a paired t test. Estimates of effect size are given as partial eta squared (ηp2).

We assumed that surprise should be estimated for the PC and UC separately, given the contexts were fully known and prior expectations about the IS sequence differed between the two contexts. However, this assumption was tested by estimating surprise across all blocks irrespective of its explicit context. For each main Drug session (plac, levo, sulp, halo2.5), we performed a formal model comparison between these scenarios using Bayesian model selection to detect which model explained a greater amount of the RT variance (Stephan, Penny, Daunizeau, Moran, & Friston, 2009). Initially, for each participant and model, a parametric empirical Bayes estimate of the log evidence was calculated (Dempster, Rubin, & Tsutakawa, 1981). Given these log evidences from all participants, we treated each model as a random variable and computed the exceedance probability of one model being more likely than any other model (Stephan et al., 2009). All analysis and statistics were performed using Matlab (Mathworks, Natick, MA).

RESULTS

Training

There was no significant difference between the training and plac sessions for RT, suggesting the task was overlearned by the end of training. RT for the PC was significantly faster than for the UC,F(1, 14) = 17, p = .001, ηp2 = 0.54. However there was no main effect of Session (training vs. plac; F(1, 14) = 1, p = .3, ηp2 = 0.07) or interaction, F(1, 14) = 0.9, p = .35, ηp2 = 0.06 (Figure 1C). We obtained β-values by regressing surprise and MI against RT throughout training. During the PC, increasing surprise was related to slower RTs, whereas increasing MI was associated with faster RTs. In contrast, surprise and MI had no influence on RT during the UC (Figure 1D). There was a statistically significant difference in β-values between the UC and PC, MANOVA: F(2, 29) = 22, p = .0005, Wilk's λ = 0.39, ηp2 = 0.61. Context had a significant effect on the β-values obtained for both surprise, F(1, 30) = 45, p = .0005, ηp2 = 0.60, and MI, F(1, 30) = 9.8, p = .004, ηp2 = 0.25. This signifies that the task was overlearned and so enables the dissociation between the role of dopamine in probabilistic learning, evident during training, and motor flexibility.

Psychological Parameters

Participants' reporting of attention and fatigue was similar across sessions. This is represented by the nonsignificant difference between ratings of attention (plac = 4.6 ± 0.4, levo = 4.4 ± 0.2, sulp = 4.3 ± 0.4, halo2.5 = 4.8 ± 0.3; F(3, 42) = 0.2, p = .7, ηp2 = 0.04) and fatigue (plac = 3.7 ± 0.5, levo = 3.3 ± 0.3, sulp = 2.9 ± 0.4, halo2.5 = 3.5 ± 0.5; F(3, 42) = 0.6, p = .6, ηp2 = 0.04) across sessions. However, 33% of the participants believed they had taken an active drug during the placebo session, which was substantially less than during the three main Drug sessions (levo = 80%, sulp = 73%, halo2.5 = 80%).

Error Rates

All participants were able to perform the task without difficulty, with low error rates observed for the entire experiment. Across all trial types, the percentage of errors did not significantly differ between sessions (plac = 6.3 ± 1.5%, levo = 5.1 ± 0.7%, sulp = 6.7 ± 1.3%, halo2.5 = 6.1 ± 1%; F(3, 42) = 0.06, p = .8, ηp2 = 0.03). In addition, post-error RTs were generally slower for all trial types, irrespective of session. However, predictable-expected trials were still faster than predictable-unexpected or unpredictable trials. This suggests that a similar pattern (see below) is observed, albeit the PC is less influential. An rmANOVA compared RTs for trials after an error across Trial type (predictable-expected, predictable-unexpected, unpredictable) and Session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, F(2, 28) = 12.1, p = .0005, ηp2 = 0.46; however, the main effect of Session, F(3, 42) = .1, p = .35, ηp2 = 0.07, and interaction were not significant, F(6, 84) = 0.06, p = .99, ηp2 = 0.004. Paired t tests revealed that predictable-expected trials (6.47 ± 0.03 LOG msec) were associated with faster RTs relative to either predictable-unexpected (6.53 ± 0.03) or unpredictable trials (6.52 ± 0.03), irrespective of session, t(14) > 3.8, p < .002.

RT

Participants' RTs were compared across Trial type (predictable-expected, predictable-unexpected, unpredictable) and Session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, rmANOVA: F(2, 28) = 92, p = .0005, ηp2 = 0.87, Session, F(3, 42) = 4.6, p = .007, ηp2 = 0.25, and interaction between Trial type and Session, F(6, 84) = 3.9, p = .002, ηp2 = 0.22 (Figure 2A). Paired t tests revealed significant differences between each trial type, t(14) > 6.3, p = .0005; predictable-expected trials were associated with the fastest RTs, whereas predictable-unexpected trials led to the slowest RTs (Figure 2A). In addition, there were global differences in RT between sessions. Paired t tests revealed that plac and levo were significantly slower than sulp and halo2.5, t(14) > 2.1, p < .04, irrespective of trial type. However, there were no significant differences between plac and levo, t(14) = 1.1, p < .30, or sulp and halo2.5, t(14) = 0.94, p = .36.

Figure 2. 

(A) RT across trial types: RT (LOG msec) across the predictable-expected, predictable-unexpected and unpredictable trial types for the placebo (black), levodopa (blue), sulpiride (green) and haloperidol2.5 (red) sessions. (B) Delta RT: ΔRT (subtracted by RT of unpredictable context) for unexpected events during the predictable context across the four main experimental sessions. halo2.5 showed a significant prolongation of RT relative to the three other sessions. Black lines indicate significant difference: all ps < .003. (C) Surprise: halo2.5 showed a significantly greater sensitivity to surprising events during the predictable context. Surprise did not influence RT during the unpredictable context. Black lines indicate significant difference: all ps < .008. Error bars indicate standard error.

Figure 2. 

(A) RT across trial types: RT (LOG msec) across the predictable-expected, predictable-unexpected and unpredictable trial types for the placebo (black), levodopa (blue), sulpiride (green) and haloperidol2.5 (red) sessions. (B) Delta RT: ΔRT (subtracted by RT of unpredictable context) for unexpected events during the predictable context across the four main experimental sessions. halo2.5 showed a significant prolongation of RT relative to the three other sessions. Black lines indicate significant difference: all ps < .003. (C) Surprise: halo2.5 showed a significantly greater sensitivity to surprising events during the predictable context. Surprise did not influence RT during the unpredictable context. Black lines indicate significant difference: all ps < .008. Error bars indicate standard error.

To account for these global differences and investigate the dynamic changes in RT during the PC, we subtracted the average RT for the UC from all RTs during the PC (ΔRT; Galea et al., 2012). Participants displayed a selective slowing in RT to unexpected trials during halo2.5 (Figure 2B). Participants' ΔRTs were compared across trial type (predictable-expected, predictable-unexpected) and session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, rmANOVA: F(1, 14) = 99, p = .0005, ηp2 = 0.87, Session, F(3, 42) = 3.8, p = .016, ηp2 = 0.22, and interaction between Trial type and Session, F(3, 42) = 3.9, p = .015, ηp2 = 0.22 (Figure 2B). There was no significant difference between sessions for expected trials (plac = 0.11 ± 0.01 LOG ΔRT), levo = −0.11 ± 0.02, sulp = −0.12 ± 0.01, halo2.5 = 0.10 ± 0.02), F(3, 42) = 0.05, p = .98, ηp2 = 0.003. In contrast, there was a significant difference for unexpected trials, F(3, 42) = 20.4, p = .0005, ηp2 = 0.59. ΔRT was significantly slower during halo2.5 relative to the three other sessions, t(14) > 5.8, p < .0005.

Event-related Surprise

To investigate the halo2.5 effect on unexpected trials in greater detail, trial-by-trial surprise was used as a proxy for the level of prediction error. We obtained β-values by regressing surprise and post-error trials against RT separately for the PC and UC within each session. Across all sessions, surprise was associated with slower RTs during the PC, as reflected by a larger β-value. Crucially, during the PC, this RT deficit to surprise was exacerbated during the halo2.5 session (Figure 2C). In contrast, RT was unaffected by post-error trials for the PC (plac = 0.02 ± 0.02 β-value, levo = 0.02 ± 0.01, sulp = −0.02 ± 0.02, halo2.5 = 0.02 ± 0.01) or UC (plac = −0.02 ± 0.01, levo = −0.001 ± 0.03, sulp = −0.01 ± 0.01, halo2.5 = −0.0042 ± 0.02). We tested how RTs were influenced by surprise and post-error by obtaining β-values from trial-by-trial regression analysis. There was a statistically significant difference in β-values between Context (PC,UC), MANOVA: F(2, 13) = 87.7, p = .0005, Wilk's λ = 0.07, ηp2 = 0.1, but not Session, F(6, 84) = 2.0, p = .07, Wilk's λ = 2.0, ηp2 = 0.70. However, the interaction between context and session was significant, F(6, 84) = 2.3, p = .02, Wilk's λ = 2.3, ηp2 = 0.78. There were no significant differences for the β-values pertaining to post-error trials for session, F(3, 42) = 0.15, p = .93, ηp2 = 0.075, context, F(1, 14) = 3.9, p = .066, ηp2 = 0.46, or interaction between session and context, F(3, 42) = 0.45, p = .98, ηp2 = 0.06. In contrast, there was a significant difference for the β-values relating to surprise between session, F(3, 42) = 4.2, p = .01, ηp2 = 0.83, context, F(1, 14) = 182, p = .0005, ηp2 = 1, and interaction between session and context, F(3, 42) = 4.8, p = .006, ηp2 = 0.88. This was driven by surprise having a significantly larger influence on RT in halo2.5 relative to the three other sessions specifically during the PC, t(14) > 3.34, p < .004 (Figure 2C).

Model Comparison

We assumed that surprise should be estimated for the PC and UC by separately combining these block types across a session (model1). However, this assumption was tested by estimating surprise across all blocks irrespective of its explicit context (model2). For each main Drug session (plac, levo, sulp, halo2.5), we performed a formal model comparison between these scenarios using Bayesian model selection to detect which model explained a greater amount of the RT variance (Stephan et al., 2009). For all sessions, the exceedance probabilities show that model1 provides the most parsimonious explanation of the data (model1: plac = 0.71 ± 0.2, levo = 0.73 ± 0.007, sulp = .73 ± 0.008, halo2.5 = 0.72 ± 0.02). Importantly, this analysis provides credence to our model assumption that participants were able to effectively switch between contexts using the explicit cues provided.

2.5 mg versus 1 mg Haloperidol

Within a subset of participants, we compared halo2.5 and halo1 and found that only the higher dose of haloperidol caused an increased RT impairment to unexpected events during the PC. A similar proportion of participants believed that they had taken an active drug during halo2.5 (80%) and halo1 (75%), whereas participants' ratings of attention and fatigue were not significantly different, t(7) < 1.5, p > .17. Participants' RT deficit to unexpected events and surprise during the PC was significantly greater in halo2.5 relative to halo1, t(7) > 3.4, p < .01 (Figure 3).

Figure 3. 

Predictable context: unexpected event and surprise: RT deficit to unexpected events during the predictable context was significantly greater during halo2.5 (red) relative to halo1 (gray). Similarly, halo2.5 showed a significantly greater sensitivity to surprising events during the predictable context. Black lines indicate significant difference: all ps < .01. Error bars indicate standard error.

Figure 3. 

Predictable context: unexpected event and surprise: RT deficit to unexpected events during the predictable context was significantly greater during halo2.5 (red) relative to halo1 (gray). Similarly, halo2.5 showed a significantly greater sensitivity to surprising events during the predictable context. Black lines indicate significant difference: all ps < .01. Error bars indicate standard error.

DISCUSSION

The present results provide novel evidence of the importance of dopamine for motor flexibility. We show that during a task that used overlearned probabilistic contexts and signaled changes in these explicitly to remove any potential role of dopamine in probabilistic learning, the dopamine antagonist haloperidol reduced motor flexibility. Specifically, participants were selectively impaired in reacting to unexpected events that elicited large prediction errors and thus required a prepared action to be replaced with an unprepared one. This effect was not observed with specific D2 receptor blockade, a general increase in tonic dopamine levels, or during an overall unpredictable context.

Several lines of evidence have pointed to the importance of dopamine in both motor and cognitive flexibility. For example, dopamine-depleted Parkinson disease patients display deficits in tasks which examine motor (Galea et al., 2012; Cools et al., 1984) and cognitive flexibility (Cools et al., 2001a), with performance being restored by dopaminergic medication (Cools et al., 2001b; Galea et al., 2012). Previous animal and human work identified a specific role for dopamine D2 receptor signaling in cognitive flexibility (Haluk & Floresco, 2009; Cools, Sheridan, Jacobs, & D'Esposito, 2007). For instance, the D2 agonist bromocriptine reduces the error cost associated with set-shifting (van Holstein et al., 2011). These results have been explained using the dual-state theory (Durstewitz & Seamans, 2008) in which D2 receptor stimulation favors fast flexible behavioral switching. However, animal work also shows the importance of D1 receptor signaling (Floresco et al., 2006; Ragozzino, 2002), with suggestions that cognitive flexibility relies on a cooperative interaction of both D1 and D2 receptors (Floresco, 2013). Yet in humans there is a paucity of attempts to isolate the relative importance of D1 and D2 receptors for behavioral switching. We propose that the current results could be explained by a specific D1 or a D1/D2 receptor combination account. Neither sulpiride nor low-dose haloperidol had an effect on motor flexibility, which suggests that D2 receptor inhibition alone is not sufficient to impair motor flexibility. By contrast, high-dose haloperidol, which blocks both D1 and D2 receptors, did impair this ability, which suggests that either D1 receptor function or a combination of both D1/D2 receptor signaling is required for efficient motor flexibility. It is important to emphasize that the forthcoming discussion of the dissociable roles of D1 and D2 receptors during motor flexibility must be taken with caution, simply because of the widespread effects these drugs can have in terms of both receptor affinity and their complex effects within different brain regions.

Although it is the case that systematic pharmacological manipulations do not allow for direct testing of the involvement of a specific brain area, we believe that a discussion of possible circuit-level mechanisms is warranted. Our previous work (Galea et al., 2012) showed Parkinson disease patients off dopaminergic medication exhibited a similar impairment in motor flexibility (patients were selectively impaired in reacting to unexpected events that elicited large prediction errors and thus required a prepared action to be replaced with an unprepared one) as observed in this study with haloperidol. Both results could be explained by the role dopamine plays in facilitating a “focusing function” of the BG (Frank, 2005; Cools et al., 2001a; Redgrave, Prescott, & Gurney, 1999b; Mink, 1996). Specifically, during behavioral switching, dopamine acting on D1 receptors is thought to exert a “premotor bias” that promotes the selection of the unprepared action, via the BG's direct pathway (Hikosaka & Isoda, 2010; Gerfen, 1992). As the BG appear to be more concerned with switching from an automatic to a more difficult task than vice versa (Cameron, Coe, Watanabe, Stroman, & Munoz, 2009), D1 receptor activation could be particularly important in boosting weaker response signals to overcome dominant response signals. This suggests that the relative weighting of these signals would be important in terms of the switching cost associated with D1 receptor blockade. In fact, our results support this view because the deficit observed with haloperidol was highly specific to the unexpectedness (surprise) of the upcoming action.

On a more theoretical level, our results point at a role for dopamine to convey contextual confidence in situations when unexpected sensory information requires fast corrections of the prepotent but incorrect action. Put differently, when participants have strong prior expectations about a given context and the sequence of events therein, dopamine depletion leads to overreliance on top–down predictions and an inability to react to bottom–up sensory information. First, when a violation occurs in such a situation of strong prior expectation, a low-level sensory prediction error occurs. To respond to this prediction error, one has to reprogram the selected action based on the correct sensory information. It appears as if D1 depletion diminishes the value of this information, which in turn delays motor flexibility because there is a need for a greater accumulation of sensory information before bottom–up information overcomes the overreliance on top–down predictions (Friston et al., 2012; Galea et al., 2012). As the sensory prediction error increases with the surprise of the upcoming action, so would the motor flexibility deficit associated with it. Physiologically, this scheme is compatible with the current notion that dopamine bursts in the BG are not related solely to unexpected rewards but also occur at short latency after any salient event, whether rewarding or not (Redgrave & Gurney, 2006; Redgrave, Prescott, & Gurney, 1999a). For the present results, we propose that such bursts would occur in response to unexpected cues and that the size of the burst would be proportional to the prediction error. Animal experiments show that high levels of dopamine shift striatal neurons into an “up state” in which they respond more readily to corticostriatal inputs (Plotkin, Day, & Surmeier, 2011). We suggest that this is a way in which the system can highlight the relevance of the cortical inputs that occur during surprising events (Galea et al., 2012).

The brain's sensitivity to prediction errors has been conceptualized through models that implement optimal Bayesian inference (Iglesias et al., 2013; Friston et al., 2012; Kording & Wolpert, 2006). These models assume that the brain continuously updates a hierarchical generative model of its sensory inputs to predict future events and infer on the causal structure of the world. This updating process involves multiple, hierarchically related prediction errors that are weighted by their precision (perceived reliability). The prediction errors concern low-level sensory events as well as their probabilistic associations and how these change in time (Behrens et al., 2007). Interestingly, it has been suggested that brain areas associated with dopamine release are sensitive to low-level prediction errors about stimulus outcome, albeit on indirect grounds (Iglesias et al., 2013). In contrast, brain regions tied to acetylcholine (Ach) release are thought to be sensitive to prediction errors at a high or abstract level regarding stimulus probabilities (Iglesias et al., 2013). More generally, these neuromodulators are proposed to play a role in encoding the precision of these hierarchical prediction errors (Friston et al., 2012). This study provided participants with two levels of expected uncertainty, in the form of an overall predictable and unpredictable context (Galea et al., 2012). Through the overlearning of stimulus probabilities and providing explicit context feedback, we were able to control for prediction errors associated with estimation and unexpected uncertainty, which are linked to high-level volatility and the neuromodulator Ach (Iglesias et al., 2013; Behrens et al., 2007; Yu & Dayan, 2002, 2005). Therefore, participants knew the uncertainty associated with each context (controlling for estimation uncertainty) and which context they were currently experiencing (controlling for unexpected uncertainty). Even under these conditions, there is irreducible low-level stimulus uncertainty because of the probabilistic nature of each context. Consequently, the prediction errors experienced in the predictable context specifically examined the role of dopamine in responding to low-level stimulus prediction errors whose behavioral outcome is termed motor flexibility.

The current results indicate that the blockade of D2 receptor activation is not sufficient in altering motor flexibility. However, why has the apparent importance of D1 receptor activation during behavioral switching not been suggested by previous human work? One explanation is that preceding human work has focused on D2 receptors (Stelzel et al., 2013; van Holstein et al., 2011). Although it has been shown that bromocriptine alters the neural activity associated with motor flexibility (Stelzel et al., 2013), it had a nonsignificant effect on behavior. Therefore, this previous work is in support of our conclusion that inhibition of D2 receptor activity alone does not impair motor flexibility.

A surprising result was the global reduction in RTs observed in both the haloperidol and sulpiride sessions. Generally, tonic dopamine is thought to enhance behavioral vigor (Niv et al., 2007), so one would expect a relative decrease in RT under levodopa, yet an increase in RT for dopamine antagonists. We have no definitive theoretical explanation to these findings. It is important to highlight RTs during the haloperidol, and sulpiride sessions were only identical for trials without a prediction error, that is, all trials except the unexpected trials in a predictable context. In addition, because this effect was observed with sulpiride, it can be assumed that it was dependent on D2 receptor signaling. As D2 receptor function is crucial for behavioral exploration (Galea, Ruge, Buijink, Bestmann, & Rothwell, 2013), it is probable that D2 receptor activation would lead to greater competition between alternative actions. For trials without a prediction error, this would provide greater competition from alternative actions during the predictable context and a general increase in competition during the unpredictable context. If the BGs were required to bias selection toward the correct action and away from other response alternatives, then such increased competition could slow RTs. D2 blockade would hypothetically decrease competition and allow the prepotent and correct response to be expressed at a faster rate.

We observed no behavioral effect with levodopa. This inability to show opponent behavioral results with levodopa and haloperidol are a common experimental finding (Pine, Shiner, Seymour, & Dolan, 2010; Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006). This may appear at odds with our previous results where levodopa restored motor flexibility in Parkinson disease patients (Galea et al., 2012). Although levodopa causes an increase in tonic dopamine, we previously suggested that it could restore phasic dopamine function in mild Parkinson disease patients (Galea et al., 2012; Grace, 1991). Yet, this effect may only occur in clinical populations who have initial low levels of phasic dopamine function (Grace, 1991). Therefore, in healthy individuals, levodopa might not lead to increased phasic dopamine activity, the mechanism proposed to be crucial for behavioral flexibility (Redgrave et al., 2010). Alternatively the task employed was very simple. Thus, following training, it is likely that participants were at a performance ceiling in terms of their ability to switch behaviors. If this was true then it would not have been possible for levodopa to improve motor flexibility.

Haloperidol not only blocks D1/D2 receptors but also acts as a serotonin and adrenergic receptor antagonist (Kroeze et al., 2003). Our previous results showed that Parkinson disease patients display an identical motor flexibility deficit as haloperidol (Galea et al., 2012). This result lends additional support for a critical role of dopamine during motor flexibility. Nonetheless, it is possible that the behavioral consequences of haloperidol are a result of its inhibitory effect on any of these neuromodulators. For instance, serotonin plays an important role in cognitive flexibility (Clarke, Walker, Dalley, Robbins, & Roberts, 2007; Clarke, Dalley, Crofts, Robbins, & Roberts, 2004), and levels of serotonin are affected in Parkinson disease patients (Politis et al., 2012). Therefore, it is feasible that these effects on serotonin contributed to both the haloperidol results reported here and previous patient results (Galea et al., 2012). However, to counter this, the Parkinson patient's action reprogramming ability improved with levodopa. While levodopa is known to cause dopamine to be released from serotonin terminals, it does not produce serotonin (Carta, Carlsson, Munoz, Kirik, & Bjorklund, 2010), thus suggesting that dopamine provided the dominant contribution to our results. Future research could examine the influence of other serotonin and adrenergic antagonists. However, a general caveat of human pharmacology studies is the unspecific nature of many available neuroactive drugs. Thus, we can only speculate that motor flexibility in humans is particularly sensitive to D1 receptor activity.

In conclusion, high-dose haloperidol impairs the ability to react to unexpected events that elicit large prediction errors and thus requires motor flexibility. This effect is not observed with either specific D2 receptor blockade or a general increase in tonic dopamine levels. To reiterate, as this task was overlearned, such effects were independent of the general role of dopamine in learning. We propose that dopamine D1 receptor function or a combination of D1/D2 receptor function could play an important role in responding to low-level prediction errors about stimulus outcome.

Acknowledgments

This work was supported by the European Union framework 7 initiative REPLACES (J. M. G., J. C. R.), Birmingham Fellows scheme (J. M. G.), Tourette Syndrome Association (D. R., J. C. R.), The Dorothy Feiss Scientific Research Grant (D. R.), Biotechnology and Biological Sciences Research Council (S. B.), and European Research Council (ActSelectContext, 260424, S. B.). We thank Louise Marshall for helpful comments.

Reprint requests should be sent to Joseph M. Galea, School of Psychology, University of Birmingham, UK, or via e-mail: j.galea@bham.ac.uk.

REFERENCES

REFERENCES
Adam
,
R.
,
Leff
,
A.
,
Sinha
,
N.
,
Turner
,
C.
,
Bays
,
P.
,
Draganski
,
B.
,
et al
(
2012
).
Dopamine reverses reward insensitivity in apathy following globus pallidus lesions.
Cortex
,
49
,
1292
1303
.
Beatty
,
W. W.
, &
Monson
,
N.
(
1990
).
Problem solving in Parkinson's disease: Comparison of performance on the Wisconsin and California Card Sorting Tests.
Journal of Geriatric Psychiatry and Neurology
,
3
,
163
171
.
Behrens
,
T. E.
,
Woolrich
,
M. W.
,
Walton
,
M. E.
, &
Rushworth
,
M. F.
(
2007
).
Learning the value of information in an uncertain world.
Nature Neuroscience
,
10
,
1214
1221
.
Bestmann
,
S.
,
Harrison
,
L. M.
,
Blankenburg
,
F.
,
Mars
,
R. B.
,
Haggard
,
P.
,
Friston
,
K. J.
,
et al
(
2008
).
Influence of uncertainty and surprise on human corticospinal excitability during preparation for action.
Current Biology
,
18
,
775
780
.
Bymaster
,
F.
,
Perry
,
K. W.
,
Nelson
,
D. L.
,
Wong
,
D. T.
,
Rasmussen
,
K.
,
Moore
,
N. A.
,
et al
(
1999
).
Olanzapine: A basic science update.
The British Journal of Psychiatry
,
37 Supplement
,
36
40
.
Cameron
,
I. G.
,
Coe
,
B. C.
,
Watanabe
,
M.
,
Stroman
,
P. W.
, &
Munoz
,
D. P.
(
2009
).
Role of the basal ganglia in switching a planned response.
European Journal of Neuroscience
,
29
,
2413
2425
.
Carta
,
M.
,
Carlsson
,
T.
,
Munoz
,
A.
,
Kirik
,
D.
, &
Bjorklund
,
A.
(
2010
).
Role of serotonin neurons in the induction of levodopa- and graft-induced dyskinesias in Parkinson's disease.
Movement Disorders
,
25(Suppl. 1)
,
S174
S179
.
Clarke
,
H. F.
,
Dalley
,
J. W.
,
Crofts
,
H. S.
,
Robbins
,
T. W.
, &
Roberts
,
A. C.
(
2004
).
Cognitive inflexibility after prefrontal serotonin depletion.
Science
,
304
,
878
880
.
Clarke
,
H. F.
,
Walker
,
S. C.
,
Dalley
,
J. W.
,
Robbins
,
T. W.
, &
Roberts
,
A. C.
(
2007
).
Cognitive inflexibility after prefrontal serotonin depletion is behaviorally and neurochemically specific.
Cerebral Cortex
,
17
,
18
27
.
Cools
,
A. R.
,
van den Bercken
,
J. H.
,
Horstink
,
M. W.
,
van Spaendonck
,
K. P.
, &
Berger
,
H. J.
(
1984
).
Cognitive and motor shifting aptitude disorder in Parkinson's disease.
Journal of Neurology, Neurosurgery and Psychiatry
,
47
,
443
453
.
Cools
,
R.
,
Barker
,
R. A.
,
Sahakian
,
B. J.
, &
Robbins
,
T. W.
(
2001a
).
Mechanisms of cognitive set flexibility in Parkinson's disease.
Brain
,
124
,
2503
2512
.
Cools
,
R.
,
Barker
,
R. A.
,
Sahakian
,
B. J.
, &
Robbins
,
T. W.
(
2001b
).
Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands.
Cerebral Cortex
,
11
,
1136
1143
.
Cools
,
R.
,
Frank
,
M. J.
,
Gibbs
,
S. E.
,
Miyakawa
,
A.
,
Jagust
,
W.
, &
D'Esposito
,
M.
(
2009
).
Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration.
Journal of Neuroscience
,
29
,
1538
1543
.
Cools
,
R.
,
Sheridan
,
M.
,
Jacobs
,
E.
, &
D'Esposito
,
M.
(
2007
).
Impulsive personality predicts dopamine-dependent changes in frontostriatal activity during component processes of working memory.
Journal of Neuroscience
,
27
,
5506
5514
.
Dempster
,
A. P.
,
Rubin
,
D. B.
, &
Tsutakawa
,
R. K.
(
1981
).
Estimation in covariance component models.
Journal of the American Statistical Association
,
76
,
341
353
.
den Ouden
,
H. E.
,
Daw
,
N. D.
,
Fernandez
,
G.
,
Elshout
,
J. A.
,
Rijpkema
,
M.
,
Hoogman
,
M.
,
et al
(
2013
).
Dissociable effects of dopamine and serotonin on reversal learning.
Neuron
,
80
,
1090
1100
.
Durstewitz
,
D.
, &
Seamans
,
J. K.
(
2008
).
The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-o-methyltransferase genotypes and schizophrenia.
Biological Psychiatry
,
64
,
739
749
.
Dutilh
,
G.
,
Vandekerckhove
,
J.
,
Forstmann
,
B. U.
,
Keuleers
,
E.
,
Brysbaert
,
M.
, &
Wagenmakers
,
E. J.
(
2012
).
Testing theories of post-error slowing.
Attention, Perception & Psychophysics
,
74
,
454
465
.
Fitzgerald
,
P. B.
,
Kapur
,
S.
,
Remington
,
G.
,
Roy
,
P.
, &
Zipursky
,
R. B.
(
2000
).
Predicting haloperidol occupancy of central dopamine D2 receptors from plasma levels.
Psychopharmacology (Berlin)
,
149
,
1
5
.
Floresco
,
S. B.
(
2013
).
Prefrontal dopamine and behavioral flexibility: Shifting from an “inverted-U” toward a family of functions.
Frontiers in Neuroscience
,
7
,
62
.
Floresco
,
S. B.
,
Magyar
,
O.
,
Ghods-Sharifi
,
S.
,
Vexelman
,
C.
, &
Tse
,
M. T.
(
2006
).
Multiple dopamine receptor subtypes in the medial prefrontal cortex of the rat regulate set-shifting.
Neuropsychopharmacology
,
31
,
297
309
.
Frank
,
M. J.
(
2005
).
Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism.
Journal of Cognitive Neuroscience
,
17
,
51
72
.
Frank
,
M. J.
,
Moustafa
,
A. A.
,
Haughey
,
H. M.
,
Curran
,
T.
, &
Hutchison
,
K. E.
(
2007
).
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning.
Proceedings of the National Academy of Sciences, U.S.A.
,
104
,
16311
16316
.
Frank
,
M. J.
, &
O'Reilly
,
R. C.
(
2006
).
A mechanistic account of striatal dopamine function in human cognition: Psychopharmacological studies with cabergoline and haloperidol.
Behavioral Neuroscience
,
120
,
497
517
.
Friston
,
K. J.
,
Shiner
,
T.
,
FitzGerald
,
T.
,
Galea
,
J. M.
,
Adams
,
R.
,
Brown
,
H.
,
et al
(
2012
).
Dopamine, affordance and active inference.
PLOS Computational Biology
,
8
,
e1002327
.
Galea
,
J. M.
,
Bestmann
,
S.
,
Beigi
,
M.
,
Jahanshahi
,
M.
, &
Rothwell
,
J. C.
(
2012
).
Action reprogramming in Parkinson's disease: Response to prediction error is modulated by levels of dopamine.
Journal of Neuroscience
,
32
,
542
550
.
Galea
,
J. M.
,
Miall
,
R. C.
, &
Woolley
,
D. G.
(
2007
).
Asymmetric interlimb transfer of concurrent adaptation to opposing dynamic forces.
Experimental Brain Research
,
182
,
267
273
.
Galea
,
J. M.
,
Ruge
,
D.
,
Buijink
,
A.
,
Bestmann
,
S.
, &
Rothwell
,
J. C.
(
2013
).
Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters.
Journal of Neuroscience
,
33
,
3981
3988
.
Gerfen
,
C. R.
(
1992
).
The neostriatal mosaic: Multiple levels of compartmental organization in the basal ganglia.
Annual Review of Neuroscience
,
15
,
285
320
.
Grace
,
A. A.
(
1991
).
Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia.
Neuroscience
,
41
,
1
24
.
Haluk
,
D. M.
, &
Floresco
,
S. B.
(
2009
).
Ventral striatal dopamine modulation of different forms of behavioral flexibility.
Neuropsychopharmacology
,
34
,
2041
2052
.
Harrison
,
L. M.
,
Duggins
,
A.
, &
Friston
,
K. J.
(
2006
).
Encoding uncertainty in the hippocampus.
Neural Networks
,
19
,
535
546
.
Hikosaka
,
O.
, &
Isoda
,
M.
(
2010
).
Switching from automatic to controlled behavior: Cortico-basal ganglia mechanisms.
Trends in Cognitive Sciences
,
14
,
154
161
.
Hou
,
D.
, &
Schumacher
,
D.
(
2001
).
The selection of a commercial route for the D1 antagonist Sch-39166.
Current Opinion in Drug Discovery & Development
,
4
,
792
799
.
Iglesias
,
S.
,
Mathys
,
C.
,
Brodersen
,
K. H.
,
Kasper
,
L.
,
Piccirelli
,
M.
,
den Ouden
,
H. E.
,
et al
(
2013
).
Hierarchical prediction errors in midbrain and basal forebrain during sensory learning.
Neuron
,
80
,
519
530
.
Isoda
,
M.
, &
Hikosaka
,
O.
(
2011
).
Cortico-basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors.
European Journal of Neuroscience
,
33
,
2058
2069
.
Kording
,
K. P.
, &
Wolpert
,
D. M.
(
2006
).
Bayesian decision theory in sensorimotor control.
Trends in Cognitive Sciences
,
10
,
319
326
.
Kroeze
,
W. K.
,
Hufeisen
,
S. J.
,
Popadak
,
B. A.
,
Renock
,
S. M.
,
Steinberg
,
S.
,
Ernsberger
,
P.
,
et al
(
2003
).
H1-histamine receptor affinity predicts short-term weight gain for typical and atypical antipsychotic drugs.
Neuropsychopharmacology
,
28
,
519
526
.
Kuroki
,
T.
,
Meltzer
,
H. Y.
, &
Ichikawa
,
J.
(
1999
).
Effects of antipsychotic drugs on extracellular dopamine levels in rat medial prefrontal cortex and nucleus accumbens.
The Journal of Pharmacology and Experimental Therapeutics
,
288
,
774
781
.
Li
,
Z.
,
Ichikawa
,
J.
,
Huang
,
M.
,
Prus
,
A. J.
,
Dai
,
J.
, &
Meltzer
,
H. Y.
(
2005
).
ACP-103, a 5-HT2A/2C inverse agonist, potentiates haloperidol-induced dopamine release in rat medial prefrontal cortex and nucleus accumbens.
Psychopharmacology
,
183
,
144
153
.
Mars
,
R. B.
,
Klein
,
M. C.
,
Neubert
,
F. X.
,
Olivier
,
E.
,
Buch
,
E. R.
,
Boorman
,
E. D.
,
et al
(
2009
).
Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict.
Journal of Neuroscience
,
29
,
6926
6931
.
Mink
,
J. W.
(
1996
).
The basal ganglia: Focused selection and inhibition of competing motor programs.
Progress in Neurobiology
,
50
,
381
425
.
Neubert
,
F. X.
,
Mars
,
R. B.
,
Buch
,
E. R.
,
Olivier
,
E.
, &
Rushworth
,
M. F.
(
2010
).
Cortical and subcortical interactions during action reprogramming and their related white matter pathways.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
13240
13245
.
Nitsche
,
M. A.
,
Lampe
,
C.
,
Antal
,
A.
,
Liebetanz
,
D.
,
Lang
,
N.
,
Tergau
,
F.
,
et al
(
2006
).
Dopaminergic modulation of long-lasting direct current-induced cortical excitability changes in the human motor cortex.
European Journal of Neuroscience
,
23
,
1651
1657
.
Niv
,
Y.
,
Daw
,
N. D.
,
Joel
,
D.
, &
Dayan
,
P.
(
2007
).
Tonic dopamine: Opportunity costs and the control of response vigor.
Psychopharmacology
,
191
,
507
520
.
O'Connor
,
S. E.
, &
Brown
,
R. A.
(
1982
).
The pharmacology of sulpiride-A dopamine receptor antagonist.
General Pharmacology
,
13
,
185
193
.
Pessiglione
,
M.
,
Seymour
,
B.
,
Flandin
,
G.
,
Dolan
,
R. J.
, &
Frith
,
C. D.
(
2006
).
Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.
Nature
,
442
,
1042
1045
.
Pine
,
A.
,
Shiner
,
T.
,
Seymour
,
B.
, &
Dolan
,
R. J.
(
2010
).
Dopamine, time, and impulsivity in humans.
Journal of Neuroscience
,
30
,
8888
8896
.
Plotkin
,
J. L.
,
Day
,
M.
, &
Surmeier
,
D. J.
(
2011
).
Synaptically driven state transitions in distal dendrites of striatal spiny neurons.
Nature Neuroscience
,
14
,
881
888
.
Politis
,
M.
,
Wu
,
K.
,
Loane
,
C.
,
Quinn
,
N. P.
,
Brooks
,
D. J.
,
Oertel
,
W. H.
,
et al
(
2012
).
Serotonin neuron loss and nonmotor symptoms continue in Parkinson's patients treated with dopamine grafts.
Science Translational Medicine
,
4
,
128ra41
.
Ragozzino
,
M. E.
(
2002
).
The effects of dopamine D(1) receptor blockade in the prelimbic-infralimbic areas on behavioral flexibility.
Learning & Memory
,
9
,
18
28
.
Redgrave
,
P.
, &
Gurney
,
K.
(
2006
).
The short-latency dopamine signal: A role in discovering novel actions?
Nature Reviews Neuroscience
,
7
,
967
975
.
Redgrave
,
P.
,
Prescott
,
T. J.
, &
Gurney
,
K.
(
1999a
).
Is the short-latency dopamine response too short to signal reward error?
Trends in Neurosciences
,
22
,
146
151
.
Redgrave
,
P.
,
Prescott
,
T. J.
, &
Gurney
,
K.
(
1999b
).
The basal ganglia: A vertebrate solution to the selection problem?
Neuroscience
,
89
,
1009
1023
.
Redgrave
,
P.
,
Rodriguez
,
M.
,
Smith
,
Y.
,
Rodriguez-Oroz
,
M. C.
,
Lehericy
,
S.
,
Bergman
,
H.
,
et al
(
2010
).
Goal-directed and habitual control in the basal ganglia: Implications for Parkinson's disease.
Nature Reviews Neuroscience
,
11
,
760
772
.
Reimold
,
M.
,
Solbach
,
C.
,
Noda
,
S.
,
Schaefer
,
J. E.
,
Bartels
,
M.
,
Beneke
,
M.
,
et al
(
2007
).
Occupancy of dopamine D(1), D (2) and serotonin (2A) receptors in schizophrenic patients treated with flupentixol in comparison with risperidone and haloperidol.
Psychopharmacology
,
190
,
241
249
.
Schultz
,
W.
,
Dayan
,
P.
, &
Montague
,
P. R.
(
1997
).
A neural substrate of prediction and reward.
Science
,
275
,
1593
1599
.
Seo
,
M.
,
Beigi
,
M.
,
Jahanshahi
,
M.
, &
Averbeck
,
B. B.
(
2010
).
Effects of dopamine medication on sequence learning with stochastic feedback in Parkinson's disease.
Frontiers in Systems Neuroscience
,
4
,
1
9
.
Stelzel
,
C.
,
Basten
,
U.
,
Montag
,
C.
,
Reuter
,
M.
, &
Fiebach
,
C. J.
(
2010
).
Frontostriatal involvement in task switching depends on genetic differences in d2 receptor density.
Journal of Neuroscience
,
30
,
14205
14212
.
Stelzel
,
C.
,
Fiebach
,
C. J.
,
Cools
,
R.
,
Tafazoli
,
S.
, &
D'Esposito
,
M.
(
2013
).
Dissociable fronto-striatal effects of dopamine D2 receptor stimulation on cognitive versus motor flexibility.
Cortex
,
49
,
2799
2811
.
Stephan
,
K. E.
,
Penny
,
W. D.
,
Daunizeau
,
J.
,
Moran
,
R. J.
, &
Friston
,
K. J.
(
2009
).
Bayesian model selection for group studies.
Neuroimage
,
46
,
1004
1017
.
Strange
,
B. A.
,
Duggins
,
A.
,
Penny
,
W.
,
Dolan
,
R. J.
, &
Friston
,
K. J.
(
2005
).
Information theory, novelty and hippocampal responses: Unpredicted or unpredictable?
Neural Networks
,
18
,
225
230
.
van der Schaaf
,
M. E.
,
van Schouwenburg
,
M. R.
,
Geurts
,
D. E.
,
Schellekens
,
A. F.
,
Buitelaar
,
J. K.
,
Verkes
,
R. J.
,
et al
(
2014
).
Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning.
Cerebral Cortex
,
24
,
633
642
.
van Holstein
,
M.
,
Aarts
,
E.
,
van der Schaaf
,
M. E.
,
Geurts
,
D. E.
,
Verkes
,
R. J.
,
Franke
,
B.
,
et al
(
2011
).
Human cognitive flexibility depends on dopamine D2 receptor signaling.
Psychopharmacology
,
218
,
567
578
.
Wilkinson
,
L.
,
Khan
,
Z.
, &
Jahanshahi
,
M.
(
2009
).
The role of the basal ganglia and its cortical connections in sequence learning: Evidence from implicit and explicit sequence learning in Parkinson's disease.
Neuropsychologia
,
47
,
2564
2573
.
Yu
,
A. J.
, &
Dayan
,
P.
(
2002
).
Acetylcholine in cortical inference.
Neural Networks
,
15
,
719
730
.
Yu
,
A. J.
, &
Dayan
,
P.
(
2005
).
Uncertainty, neuromodulation, and attention.
Neuron
,
46
,
681
692
.
Zhang
,
W.
, &
Bymaster
,
F. P.
(
1999
).
The in vivo effects of olanzapine and other antipsychotic agents on receptor occupancy and antagonism of dopamine D1, D2, D3, 5HT2A and muscarinic receptors.
Psychopharmacology
,
141
,
267
278
.