Humans carry out many daily tasks in a seemingly automatic fashion. However, when unexpected changes in the environment occur, we have the capacity to inhibit prepotent behavior and replace it with an alternative one. Such behavioral flexibility is a hallmark of executive functions. The neurotransmitter dopamine is known to be crucial for fast, efficient, and accurate cognitive flexibility. Despite the perceived similarities between cognitive and motor flexibility, less is known regarding the role of dopamine within the motor domain. Therefore, the aim of this study was to determine the role of dopamine in motor flexibility. In a double-blind, five-session, within-subject pharmacological experiment, human participants performed an RT task within a probabilistic context that was either predictable or unpredictable. The probabilistic nature of the predictable context resulted in prediction errors. This required participants to replace the prepotent or prepared action with an unprepared action (motor flexibility). The task was overlearned, and changes in context were explicitly instructed, thus controlling for contributions from other dopamine-related processes such as probabilistic or reversal learning and interactions with other types of uncertainty. We found that dopamine receptor blockade by high-dose haloperidol (D1/D2 dopamine receptors) impaired participants' ability to react to unexpected events occurring in a predictable context, which elicit large prediction errors and necessitate motor flexibility. This effect was not observed with selective D2 receptor blockade (sulpiride), with a general increase in tonic dopamine levels (levodopa), or during an unpredictable context, which evoked minimal prediction error. We propose that dopamine is vital in responding to low-level prediction errors about stimulus outcome that requires motor flexibility.
Much human behavior is executed in a seemingly automatic fashion. These behaviors can be viewed as prepotent in that they take precedence over any other potential alternatives (Isoda & Hikosaka, 2011; Hikosaka & Isoda, 2010). When this behavior becomes inappropriate through an unexpected change in the environment, humans are capable of engaging resources that inhibit these prepotent responses and replace them with alternative ones. Such behavioral flexibility in a short period of time is a hallmark of executive functions (Isoda & Hikosaka, 2011; Hikosaka & Isoda, 2010).
There is considerable evidence that the neuromodulator dopamine plays a crucial role in behavioral flexibility (van der Schaaf et al., 2014; Stelzel, Fiebach, Cools, Tafazoli, & D'Esposito, 2013; Stelzel, Basten, Montag, Reuter, & Fiebach, 2010). Previous human work has highlighted the importance of dopamine in cognition-based switching tasks (van Holstein et al., 2011; Cools et al., 2009; Cools, Barker, Sahakian, & Robbins, 2001a), with continuing interest in the specific function of dopamine D1 and D2 receptors. Animal (Haluk & Floresco, 2009; Floresco, Magyar, Ghods-Sharifi, Vexelman, & Tse, 2006), theoretical (Durstewitz & Seamans, 2008), and human (Stelzel et al., 2013; van Holstein et al., 2011) work has suggested that D2 receptor signaling is an essential component for efficient cognitive flexibility. However, there is also considerable evidence that D1 receptor signaling plays a significant role (Ragozzino, 2002), with recent proposals that cognitive flexibility relies on a cooperative interaction of both D1 and D2 receptors (Floresco, 2013).
A distinction can be made between cognitive and motor flexibility (Stelzel et al., 2013). Whereas the former will generally involve a complex rule change (set-shift: naming digits following naming letters; Cools et al., 2001a), the latter requires the replacement of a prepared action with an unprepared one (Galea, Bestmann, Beigi, Jahanshahi, & Rothwell, 2012; Neubert, Mars, Buch, Olivier, & Rushworth, 2010). Motor flexibility has also been termed behavioral adaptation (Stelzel et al., 2013), motor-based behavioral switching (Hikosaka & Isoda, 2010) or action reprogramming (Galea et al., 2012; Neubert et al., 2010; Mars et al., 2009). We regard this terminology to reflect a similar process and therefore will use motor flexibility as a term that encompasses them all. Parkinson disease patients who suffer from a dopamine deficit show specific impairments in motor (Galea et al., 2012; Cools, van den Bercken, Horstink, van Spaendonck, & Berger, 1984) and cognitive flexibility tasks (Cools et al., 2001a; Beatty & Monson, 1990), with performance restored by dopaminergic medication (Galea et al., 2012; Cools, Barker, Sahakian, & Robbins, 2001b). Despite this work, the dissociable influence of D1 and D2 receptor signaling in motor flexibility is relatively unknown (Stelzel et al., 2013).
We addressed this issue by testing for the effects of a range of dopaminergic drugs on motor flexibility. Individuals use past experience to prepare movements in advance of upcoming events, and the degree of preparation is closely related to the predictability of a future event (Bestmann et al., 2008). Such preparation is advantageous when an event is predicted correctly but occurs at the expense of a prolonged RT when an unexpected event results in a prediction error. In this case, a prepared action has to be replaced by an unprepared alternative movement. Importantly, the greater the prediction error is, the greater the cost in terms of a prolonged RT (Galea et al., 2012; Bestmann et al., 2008).
The brain's sensitivity to prediction error has been explained by models that implement optimal Bayesian inference (Iglesias et al., 2013; Friston et al., 2012; Kording & Wolpert, 2006). Interestingly, indirect evidence from human neuroimaging studies suggests that brain areas associated with dopamine release are sensitive to low-level prediction errors about stimulus outcome (Iglesias et al., 2013). In contrast, prediction errors at high or abstract levels of uncertainty are thought to be encoded by other neuromodulatory systems (Yu & Dayan, 2002, 2005). Such hierarchical models therefore predict that dopamine may have an important role for responding to prediction errors, even in the absence of other learning-related types of uncertainty (volatility; Behrens, Woolrich, Walton, & Rushworth, 2007).
This study sought to isolate the contribution of dopamine for enabling motor flexibility in response to low-level prediction errors. To this end, we controlled for any role dopamine might have in the learning of probabilistic task contexts or reversal learning (Seo, Beigi, Jahanshahi, & Averbeck, 2010; Cools et al., 2009; Frank, Moustafa, Haughey, Curran, & Hutchison, 2007), by explicitly highlighting the current context to participants and providing them with extended practice on each context before the main experiment. This also ensured that participants did not experience volatility/unexpected uncertainty, that is, changes in the probability of the upcoming stimuli (Behrens et al., 2007; Yu & Dayan, 2002, 2005). Instead, participants experienced different degrees of expected uncertainty wherein they had advanced and stable knowledge about the overall predictability of the context (overall predictable context, PC, and overall unpredictable context, UC), but because of the probabilistic nature of each context a degree of irreducible uncertainty persisted. Therefore, participants knew about the context they were currently experiencing and consequently the (expected) uncertainty associated with each context but had to quickly respond to violations of these expectations in the form of rare and surprising events. Consequently, the prediction errors experienced in the PC would specifically examine the role of dopamine in responding to low-level stimulus prediction errors whose behavioral outcome is termed motor flexibility.
In a double-blind, five-session, within-subject design, participants performed a simple cued RT task (Galea et al., 2012). In the absence of learning, we found that combined D1/D2 receptor blockade (haloperidol) impaired participants' ability to react to unexpected events that elicit large prediction errors and thus require motor flexibility. This effect is neither observed with specific D2 receptor blockade (sulpiride) nor with a general increase in tonic dopamine levels (levodopa) or during an unpredictable context, which evoked minimal prediction error.
Fifteen self-assessed, right-handed individuals with no current health problems or history of neurological/psychiatric illness (nine men, mean age = 27 ± 6 years) participated in the study, with written informed consent. The study was approved by the research ethics committee of the Institute of Neurology, University College London.
In a double-blind, placebo-controlled design, each participant took part in four experimental drug sessions, each separated by at least 1 week, in which they performed a probabilistic sequence RT task. Eight participants were involved in an additional fifth session. Session order was pseudorandomized so that placebo was evenly distributed across the five sessions. Participants sat in front of a computer screen positioned 30 cm away and placed each of the fingers of their right hand on one of four response buttons. At the start of each block, a screen was displayed to inform the participant whether the upcoming block consisted of either a UC or PC. Participants were therefore explicitly instructed about changes in the context. To focus participants' attention, an uninformative warning cue (white box) was then presented. This was followed by the presentation of one of four imperative stimuli (IS). Participants responded to the IS as quickly as possible, but not at the expense of accuracy. A fixation-cross followed (Figure 1A). Each imperative stimulus was associated with pressing a specific button (Figure 1A). These stimulus–response associations were learnt by the participants during a familiarization period (100 trials) at the beginning of the training session (see below) in which feedback was provided to signal whether their response was correct. Learnt stimulus–response associations were characterized by an error rate of less than 7%.
During the main experimental sessions, participants were exposed to 72 blocks of 12 trials (864 trials), which was equivalent to 40 min of testing. Blocks alternated between UC and PC. During UC, there was a .25 probability of each imperative stimulus being presented on trial t. In contrast, during the PC the current stimulus on trial t was conditionally dependent on the stimulus of the previous trial, t− 1. This generated sequences in which the imperative stimulus order 1-2-3-4 occurred with high probability (Figure 1A, B). Because of the probabilistic nature of the sequence, however, occasional violations of this predominant sequence order occurred.
Dopamine is known to play a pivotal role in multiple learning processes such as reinforcement (Niv, Daw, Joel, & Dayan, 2007; Schultz, Dayan, & Montague, 1997) and probabilistic learning (den Ouden et al., 2013; Cools et al., 2009; Wilkinson, Khan, & Jahanshahi, 2009). Here, we wanted to test for the specific role of dopamine in behavioral flexibility and therefore sought to eliminate learning-related contributions of dopamine. Thus, before the main experimental sessions, all participants underwent a training session in which the task was overlearned. During training, participants were exposed to 72 blocks of 12 trials (864 trials), which was equivalent to 40 min of testing. To reiterate, at the start of each block, a screen was displayed to inform the participant whether the upcoming block consisted of either a UC or PC. At the end of training, the experimenters plotted the RT for the UC and PC. RT was averaged over every 12 trials (1 block) across training. Plateau performance was defined as participants showing no improvements in RT, which were greater than 10 msec from the previous UC or PC block. A plateau in RT performance was assumed to reflect overlearning (Figure 1C). For both the UC and PC, all participants exhibited plateau performance during the last 10 blocks. Training ensured participants had strong priors about the nature of each context and were able to switch between them on the basis of the explicit feedback. This was crucial in ensuring the results were specific to the role of dopamine in behavioral flexibility and not probabilistic or reversal learning (den Ouden et al., 2013; Cools et al., 2009).
For each experimental session, participants arrived 2 hr before completing the task, and received either 100 mg of the dopamine precursor levodopa (levo), 400 mg of the D2 antagonist sulpiride (sulp), 2.5 mg of the D1/D2-antagonist haloperidol (halo2.5), or placebo (plac). Participants who underwent an additional fifth session were administered 1 mg of haloperidol (halo1). Haloperidol was administered 2 hr before the onset of the experiment, whereas levodopa and sulpiride were administered 1 hr prior. Placebo was administered randomly at either 1 or 2 hr before. The doses and administration times are similar to previous studies that have shown clear behavioral and neurophysiological effects for levodopa (Adam et al., 2012), sulpiride (Nitsche et al., 2006), and haloperidol (Frank & O'Reilly, 2006).
This combination of drugs was used in an attempt to partially dissociate the roles of dopamine D1 and D2 receptors during motor flexibility. Because selective D1-anatagonists available for human use have severe reported side effects (Hou & Schumacher, 2001), we exploited the hypothesized relative D1 and D2 receptor affinity differences between halo2.5, halo1, and sulp. Our approach was based on these drugs-predicted effects in the BG; however, it is important to emphasize that the role of the BG during this task must be taken with caution, simply because of the widespread changes the drugs have both in terms of receptor affinity and on different brain regions. First, sulpiride specifically blocks D2 receptors (O'Connor & Brown, 1982). Second, although haloperidol blocks D1 and D2 receptors, its affinity for D1 receptors is 25 times lower than for D2 (Bymaster et al., 1999). A dose of 1 mg should be sufficient to detect a behavioral impairment on tasks that are D2 receptor dependent (Fitzgerald, Kapur, Remington, Roy, & Zipursky, 2000). By contrast, a dose of 2.5 mg haloperidol ought to partially block both D1 and D2 receptors, while minimizing the extrapyramidal and sedative side effects that occur at higher doses. However, we are acutely aware that these differences between high- and low-dose haloperidol are not categorical. For instance, Fitzgerald et al. (2000) showed approximately 50% D2 occupancy at 1 mg and 75% at 2.5 mg. In addition, only a 14% increase in D1 occupancy has been found at 8.5 mg of haloperidol (Reimold et al., 2007), but in support of our hypothesis in vivo binding of D1 receptors was found to be 100% times greater than D2 receptors with haloperidol (Zhang & Bymaster, 1999). Finally, as haloperidol blocks the phasic uptake of dopamine, it can lead to increased levels of tonic dopamine in the BG (Kuroki, Meltzer, & Ichikawa, 1999). To dissociate these effects, we compared halo2.5 with levo (which increases tonic dopamine). Although we assume this task is mediated by the BG, one cannot exclude that any observed effects are mediated via the frontal cortex. As such, we note that although sulpiride led to increased tonic dopamine in the pFC, haloperidol did not (Li et al., 2005; Kuroki et al., 1999). Although it is not possible to directly test the isolated role of D1 receptors, this combination of drugs provided an insight into the possible independent roles of D1 and D2 receptors during motor flexibility.
At the end of each session, participants reported their attention and fatigue using a self-scored visual analog scale (1 = poorest attention/maximal fatigue; 7 = maximal attention/least fatigue) and reported whether they thought they had received an active or placebo drug.
For all correct responses, RT was calculated as the time between imperative stimulus onset and the subsequent button press. A Kolmogorov–Smirnov test was performed on the standardized data (subtracted by mean and divided by SD) across all trial types and participants. This test rejected the null hypothesis that the data were normally distributed at p < .05. Therefore, the data were log-transformed (Galea, Miall, & Woolley, 2007). A Kolmogorov–Smirnov test revealed the data now no longer significantly deviated from a normal distribution (p > .05). Using this log-transformed data, average RTs were calculated for each trial type (unpredictable, UC); predictable-expected, PC; predictable-unexpected, PC). The average RT from the UC was subtracted from both expected (predictable-expected) and unexpected (predictable-unexpected) trial types during the PC (ΔRT). This analysis mirrors the analyses in our previous work in Parkinson disease patients (Galea et al., 2012), allowed a simple comparison with minimal assumptions between sessions for the expected (>0.75) and unexpected (<0.12) trial types during the PC, and enabled us to quantify the cost of violation of expectations within the PC (Galea et al., 2012).
To assess whether the task was overlearned during training, the last 10 blocks of the training session were compared with the plac session. A within-subject repeated-measures ANOVA (rmANOVA) compared session (training, plac) with context (PC, UC).
For the main experimental sessions, we compared the percentage of incorrect button responses (error), average RT across trial types, and average ΔRT for expected and unexpected trial types during the PC. Within-subject rmANOVA compared these parameters across the four main Drug sessions (plac, levo, sulp, halo2.5). Because of multiple comparisons, a Bonferroni adjustment was made in which we accepted statistical significance at p < .025. Post hoc paired t tests explored significant effects (two-tailed). Because of only a subpopulation of participants being tested with 1 mg of haloperidol, halo1 was compared with halo2.5 using a paired t test. Estimates of effect size are given as partial eta squared (ηp2). All data presented represent mean ± standard error.
In the current task, participants were required to respond as fast as possible but would always have some degree of uncertainty about the upcoming IS. Because of the probabilistic nature of the overlearned sequences, occasional violations would occur in the form of unexpected IS. For these surprising IS, participants had to respond against their prior expectation. Increases in RT to such surprising IS should then relate to the magnitude of the prediction error during the PC (Galea et al., 2012). In contrast, during the UC, when prior expectations would be overall small, occasionally more surprising IS occur against an overall UC, with little prediction error. We quantified the surprise enacted by a particular IS on a trial-by-trial basis. We estimated the conditional probability of each IS using a Bayesian update scheme (Harrison, Duggins, & Friston, 2006; Strange et al., 2005) in which we assumed that, at the beginning of each session, participants started with the prior expectation of all IS being equally likely. Note that, in each context, the overall probability of occurrence of all four stimuli was equal, and because these contexts had been overlearned and were explicitly signaled at every change, a uniform prior reflects the expectations participants should have.
For training, we estimated surprise and mutual information (MI) independently for the PC and UC by separately combining these block types across a session. MI is a measure of learning that quantifies the predictability of the current trial, t, based on the IS presented on the previous trial, t− 1 (Harrison et al., 2006). During the PC, MI steadily increases as the uncertainty of an IS type on trial t, which is afforded by an IS type on trial t− 1, decreases. In contrast, MI remains low during the UC (Galea et al., 2012). To assess how RT was associated with MI and surprise on a trial-by-trial basis, a robust multiple regression (iteratively reweighted least squares) was performed with RT as the dependent variable and MI and surprise as independent variables. This produced β-values for surprise and MI in the PC and UC. A MANOVA compared β-values for surprise and MI between the PC and UC. Because of multiple comparisons, a Bonferroni adjustment was made in which we accepted statistical significance at p < .025.
As participants overlearned each context during training, only surprise was estimated for the main experimental data for the PC and UC by separately combining these block types across a session. Although participants did not receive any direct feedback regarding errors, we presumed they were still likely to be noticed. Despite trials in which an error occurred being removed, we hypothesized that post-error trials would still be influenced and exhibit the phenomenon of post-error slowing (Dutilh et al., 2012). To assess how RTs were associated with surprise and post-error trials on a trial-by-trial basis, independent robust multiple regressions (iteratively reweighted least squares) were performed for the PC and UC, with RT as the dependent variable and surprise and post-error trials as independent variables. A repeated-measures MANOVA compared the β-values for surprise and post-error trials across the four main Drug sessions (plac, levo, sulp, halo2.5) and Context (predictable, unpredictable). Because of multiple comparisons (surprise and post-error trials), a Bonferroni adjustment was made in which we accepted statistical significance at p < .025. Post hoc paired t tests explored significant effects with two-tailed scores being presented. Because of only a subpopulation of participants being tested with 1 mg of haloperidol, halo1 was compared with halo2.5 using a paired t test. Estimates of effect size are given as partial eta squared (ηp2).
We assumed that surprise should be estimated for the PC and UC separately, given the contexts were fully known and prior expectations about the IS sequence differed between the two contexts. However, this assumption was tested by estimating surprise across all blocks irrespective of its explicit context. For each main Drug session (plac, levo, sulp, halo2.5), we performed a formal model comparison between these scenarios using Bayesian model selection to detect which model explained a greater amount of the RT variance (Stephan, Penny, Daunizeau, Moran, & Friston, 2009). Initially, for each participant and model, a parametric empirical Bayes estimate of the log evidence was calculated (Dempster, Rubin, & Tsutakawa, 1981). Given these log evidences from all participants, we treated each model as a random variable and computed the exceedance probability of one model being more likely than any other model (Stephan et al., 2009). All analysis and statistics were performed using Matlab (Mathworks, Natick, MA).
There was no significant difference between the training and plac sessions for RT, suggesting the task was overlearned by the end of training. RT for the PC was significantly faster than for the UC,F(1, 14) = 17, p = .001, ηp2 = 0.54. However there was no main effect of Session (training vs. plac; F(1, 14) = 1, p = .3, ηp2 = 0.07) or interaction, F(1, 14) = 0.9, p = .35, ηp2 = 0.06 (Figure 1C). We obtained β-values by regressing surprise and MI against RT throughout training. During the PC, increasing surprise was related to slower RTs, whereas increasing MI was associated with faster RTs. In contrast, surprise and MI had no influence on RT during the UC (Figure 1D). There was a statistically significant difference in β-values between the UC and PC, MANOVA: F(2, 29) = 22, p = .0005, Wilk's λ = 0.39, ηp2 = 0.61. Context had a significant effect on the β-values obtained for both surprise, F(1, 30) = 45, p = .0005, ηp2 = 0.60, and MI, F(1, 30) = 9.8, p = .004, ηp2 = 0.25. This signifies that the task was overlearned and so enables the dissociation between the role of dopamine in probabilistic learning, evident during training, and motor flexibility.
Participants' reporting of attention and fatigue was similar across sessions. This is represented by the nonsignificant difference between ratings of attention (plac = 4.6 ± 0.4, levo = 4.4 ± 0.2, sulp = 4.3 ± 0.4, halo2.5 = 4.8 ± 0.3; F(3, 42) = 0.2, p = .7, ηp2 = 0.04) and fatigue (plac = 3.7 ± 0.5, levo = 3.3 ± 0.3, sulp = 2.9 ± 0.4, halo2.5 = 3.5 ± 0.5; F(3, 42) = 0.6, p = .6, ηp2 = 0.04) across sessions. However, 33% of the participants believed they had taken an active drug during the placebo session, which was substantially less than during the three main Drug sessions (levo = 80%, sulp = 73%, halo2.5 = 80%).
All participants were able to perform the task without difficulty, with low error rates observed for the entire experiment. Across all trial types, the percentage of errors did not significantly differ between sessions (plac = 6.3 ± 1.5%, levo = 5.1 ± 0.7%, sulp = 6.7 ± 1.3%, halo2.5 = 6.1 ± 1%; F(3, 42) = 0.06, p = .8, ηp2 = 0.03). In addition, post-error RTs were generally slower for all trial types, irrespective of session. However, predictable-expected trials were still faster than predictable-unexpected or unpredictable trials. This suggests that a similar pattern (see below) is observed, albeit the PC is less influential. An rmANOVA compared RTs for trials after an error across Trial type (predictable-expected, predictable-unexpected, unpredictable) and Session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, F(2, 28) = 12.1, p = .0005, ηp2 = 0.46; however, the main effect of Session, F(3, 42) = .1, p = .35, ηp2 = 0.07, and interaction were not significant, F(6, 84) = 0.06, p = .99, ηp2 = 0.004. Paired t tests revealed that predictable-expected trials (6.47 ± 0.03 LOG msec) were associated with faster RTs relative to either predictable-unexpected (6.53 ± 0.03) or unpredictable trials (6.52 ± 0.03), irrespective of session, t(14) > 3.8, p < .002.
Participants' RTs were compared across Trial type (predictable-expected, predictable-unexpected, unpredictable) and Session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, rmANOVA: F(2, 28) = 92, p = .0005, ηp2 = 0.87, Session, F(3, 42) = 4.6, p = .007, ηp2 = 0.25, and interaction between Trial type and Session, F(6, 84) = 3.9, p = .002, ηp2 = 0.22 (Figure 2A). Paired t tests revealed significant differences between each trial type, t(14) > 6.3, p = .0005; predictable-expected trials were associated with the fastest RTs, whereas predictable-unexpected trials led to the slowest RTs (Figure 2A). In addition, there were global differences in RT between sessions. Paired t tests revealed that plac and levo were significantly slower than sulp and halo2.5, t(14) > 2.1, p < .04, irrespective of trial type. However, there were no significant differences between plac and levo, t(14) = 1.1, p < .30, or sulp and halo2.5, t(14) = 0.94, p = .36.
To account for these global differences and investigate the dynamic changes in RT during the PC, we subtracted the average RT for the UC from all RTs during the PC (ΔRT; Galea et al., 2012). Participants displayed a selective slowing in RT to unexpected trials during halo2.5 (Figure 2B). Participants' ΔRTs were compared across trial type (predictable-expected, predictable-unexpected) and session (plac, levo, sulp, halo2.5). There was a significant main effect of Trial type, rmANOVA: F(1, 14) = 99, p = .0005, ηp2 = 0.87, Session, F(3, 42) = 3.8, p = .016, ηp2 = 0.22, and interaction between Trial type and Session, F(3, 42) = 3.9, p = .015, ηp2 = 0.22 (Figure 2B). There was no significant difference between sessions for expected trials (plac = −0.11 ± 0.01 LOG ΔRT), levo = −0.11 ± 0.02, sulp = −0.12 ± 0.01, halo2.5 = 0.10 ± 0.02), F(3, 42) = 0.05, p = .98, ηp2 = 0.003. In contrast, there was a significant difference for unexpected trials, F(3, 42) = 20.4, p = .0005, ηp2 = 0.59. ΔRT was significantly slower during halo2.5 relative to the three other sessions, t(14) > 5.8, p < .0005.
To investigate the halo2.5 effect on unexpected trials in greater detail, trial-by-trial surprise was used as a proxy for the level of prediction error. We obtained β-values by regressing surprise and post-error trials against RT separately for the PC and UC within each session. Across all sessions, surprise was associated with slower RTs during the PC, as reflected by a larger β-value. Crucially, during the PC, this RT deficit to surprise was exacerbated during the halo2.5 session (Figure 2C). In contrast, RT was unaffected by post-error trials for the PC (plac = 0.02 ± 0.02 β-value, levo = 0.02 ± 0.01, sulp = −0.02 ± 0.02, halo2.5 = 0.02 ± 0.01) or UC (plac = −0.02 ± 0.01, levo = −0.001 ± 0.03, sulp = −0.01 ± 0.01, halo2.5 = −0.0042 ± 0.02). We tested how RTs were influenced by surprise and post-error by obtaining β-values from trial-by-trial regression analysis. There was a statistically significant difference in β-values between Context (PC,UC), MANOVA: F(2, 13) = 87.7, p = .0005, Wilk's λ = 0.07, ηp2 = 0.1, but not Session, F(6, 84) = 2.0, p = .07, Wilk's λ = 2.0, ηp2 = 0.70. However, the interaction between context and session was significant, F(6, 84) = 2.3, p = .02, Wilk's λ = 2.3, ηp2 = 0.78. There were no significant differences for the β-values pertaining to post-error trials for session, F(3, 42) = 0.15, p = .93, ηp2 = 0.075, context, F(1, 14) = 3.9, p = .066, ηp2 = 0.46, or interaction between session and context, F(3, 42) = 0.45, p = .98, ηp2 = 0.06. In contrast, there was a significant difference for the β-values relating to surprise between session, F(3, 42) = 4.2, p = .01, ηp2 = 0.83, context, F(1, 14) = 182, p = .0005, ηp2 = 1, and interaction between session and context, F(3, 42) = 4.8, p = .006, ηp2 = 0.88. This was driven by surprise having a significantly larger influence on RT in halo2.5 relative to the three other sessions specifically during the PC, t(14) > 3.34, p < .004 (Figure 2C).
We assumed that surprise should be estimated for the PC and UC by separately combining these block types across a session (model1). However, this assumption was tested by estimating surprise across all blocks irrespective of its explicit context (model2). For each main Drug session (plac, levo, sulp, halo2.5), we performed a formal model comparison between these scenarios using Bayesian model selection to detect which model explained a greater amount of the RT variance (Stephan et al., 2009). For all sessions, the exceedance probabilities show that model1 provides the most parsimonious explanation of the data (model1: plac = 0.71 ± 0.2, levo = 0.73 ± 0.007, sulp = .73 ± 0.008, halo2.5 = 0.72 ± 0.02). Importantly, this analysis provides credence to our model assumption that participants were able to effectively switch between contexts using the explicit cues provided.
2.5 mg versus 1 mg Haloperidol
Within a subset of participants, we compared halo2.5 and halo1 and found that only the higher dose of haloperidol caused an increased RT impairment to unexpected events during the PC. A similar proportion of participants believed that they had taken an active drug during halo2.5 (80%) and halo1 (75%), whereas participants' ratings of attention and fatigue were not significantly different, t(7) < 1.5, p > .17. Participants' RT deficit to unexpected events and surprise during the PC was significantly greater in halo2.5 relative to halo1, t(7) > 3.4, p < .01 (Figure 3).
The present results provide novel evidence of the importance of dopamine for motor flexibility. We show that during a task that used overlearned probabilistic contexts and signaled changes in these explicitly to remove any potential role of dopamine in probabilistic learning, the dopamine antagonist haloperidol reduced motor flexibility. Specifically, participants were selectively impaired in reacting to unexpected events that elicited large prediction errors and thus required a prepared action to be replaced with an unprepared one. This effect was not observed with specific D2 receptor blockade, a general increase in tonic dopamine levels, or during an overall unpredictable context.
Several lines of evidence have pointed to the importance of dopamine in both motor and cognitive flexibility. For example, dopamine-depleted Parkinson disease patients display deficits in tasks which examine motor (Galea et al., 2012; Cools et al., 1984) and cognitive flexibility (Cools et al., 2001a), with performance being restored by dopaminergic medication (Cools et al., 2001b; Galea et al., 2012). Previous animal and human work identified a specific role for dopamine D2 receptor signaling in cognitive flexibility (Haluk & Floresco, 2009; Cools, Sheridan, Jacobs, & D'Esposito, 2007). For instance, the D2 agonist bromocriptine reduces the error cost associated with set-shifting (van Holstein et al., 2011). These results have been explained using the dual-state theory (Durstewitz & Seamans, 2008) in which D2 receptor stimulation favors fast flexible behavioral switching. However, animal work also shows the importance of D1 receptor signaling (Floresco et al., 2006; Ragozzino, 2002), with suggestions that cognitive flexibility relies on a cooperative interaction of both D1 and D2 receptors (Floresco, 2013). Yet in humans there is a paucity of attempts to isolate the relative importance of D1 and D2 receptors for behavioral switching. We propose that the current results could be explained by a specific D1 or a D1/D2 receptor combination account. Neither sulpiride nor low-dose haloperidol had an effect on motor flexibility, which suggests that D2 receptor inhibition alone is not sufficient to impair motor flexibility. By contrast, high-dose haloperidol, which blocks both D1 and D2 receptors, did impair this ability, which suggests that either D1 receptor function or a combination of both D1/D2 receptor signaling is required for efficient motor flexibility. It is important to emphasize that the forthcoming discussion of the dissociable roles of D1 and D2 receptors during motor flexibility must be taken with caution, simply because of the widespread effects these drugs can have in terms of both receptor affinity and their complex effects within different brain regions.
Although it is the case that systematic pharmacological manipulations do not allow for direct testing of the involvement of a specific brain area, we believe that a discussion of possible circuit-level mechanisms is warranted. Our previous work (Galea et al., 2012) showed Parkinson disease patients off dopaminergic medication exhibited a similar impairment in motor flexibility (patients were selectively impaired in reacting to unexpected events that elicited large prediction errors and thus required a prepared action to be replaced with an unprepared one) as observed in this study with haloperidol. Both results could be explained by the role dopamine plays in facilitating a “focusing function” of the BG (Frank, 2005; Cools et al., 2001a; Redgrave, Prescott, & Gurney, 1999b; Mink, 1996). Specifically, during behavioral switching, dopamine acting on D1 receptors is thought to exert a “premotor bias” that promotes the selection of the unprepared action, via the BG's direct pathway (Hikosaka & Isoda, 2010; Gerfen, 1992). As the BG appear to be more concerned with switching from an automatic to a more difficult task than vice versa (Cameron, Coe, Watanabe, Stroman, & Munoz, 2009), D1 receptor activation could be particularly important in boosting weaker response signals to overcome dominant response signals. This suggests that the relative weighting of these signals would be important in terms of the switching cost associated with D1 receptor blockade. In fact, our results support this view because the deficit observed with haloperidol was highly specific to the unexpectedness (surprise) of the upcoming action.
On a more theoretical level, our results point at a role for dopamine to convey contextual confidence in situations when unexpected sensory information requires fast corrections of the prepotent but incorrect action. Put differently, when participants have strong prior expectations about a given context and the sequence of events therein, dopamine depletion leads to overreliance on top–down predictions and an inability to react to bottom–up sensory information. First, when a violation occurs in such a situation of strong prior expectation, a low-level sensory prediction error occurs. To respond to this prediction error, one has to reprogram the selected action based on the correct sensory information. It appears as if D1 depletion diminishes the value of this information, which in turn delays motor flexibility because there is a need for a greater accumulation of sensory information before bottom–up information overcomes the overreliance on top–down predictions (Friston et al., 2012; Galea et al., 2012). As the sensory prediction error increases with the surprise of the upcoming action, so would the motor flexibility deficit associated with it. Physiologically, this scheme is compatible with the current notion that dopamine bursts in the BG are not related solely to unexpected rewards but also occur at short latency after any salient event, whether rewarding or not (Redgrave & Gurney, 2006; Redgrave, Prescott, & Gurney, 1999a). For the present results, we propose that such bursts would occur in response to unexpected cues and that the size of the burst would be proportional to the prediction error. Animal experiments show that high levels of dopamine shift striatal neurons into an “up state” in which they respond more readily to corticostriatal inputs (Plotkin, Day, & Surmeier, 2011). We suggest that this is a way in which the system can highlight the relevance of the cortical inputs that occur during surprising events (Galea et al., 2012).
The brain's sensitivity to prediction errors has been conceptualized through models that implement optimal Bayesian inference (Iglesias et al., 2013; Friston et al., 2012; Kording & Wolpert, 2006). These models assume that the brain continuously updates a hierarchical generative model of its sensory inputs to predict future events and infer on the causal structure of the world. This updating process involves multiple, hierarchically related prediction errors that are weighted by their precision (perceived reliability). The prediction errors concern low-level sensory events as well as their probabilistic associations and how these change in time (Behrens et al., 2007). Interestingly, it has been suggested that brain areas associated with dopamine release are sensitive to low-level prediction errors about stimulus outcome, albeit on indirect grounds (Iglesias et al., 2013). In contrast, brain regions tied to acetylcholine (Ach) release are thought to be sensitive to prediction errors at a high or abstract level regarding stimulus probabilities (Iglesias et al., 2013). More generally, these neuromodulators are proposed to play a role in encoding the precision of these hierarchical prediction errors (Friston et al., 2012). This study provided participants with two levels of expected uncertainty, in the form of an overall predictable and unpredictable context (Galea et al., 2012). Through the overlearning of stimulus probabilities and providing explicit context feedback, we were able to control for prediction errors associated with estimation and unexpected uncertainty, which are linked to high-level volatility and the neuromodulator Ach (Iglesias et al., 2013; Behrens et al., 2007; Yu & Dayan, 2002, 2005). Therefore, participants knew the uncertainty associated with each context (controlling for estimation uncertainty) and which context they were currently experiencing (controlling for unexpected uncertainty). Even under these conditions, there is irreducible low-level stimulus uncertainty because of the probabilistic nature of each context. Consequently, the prediction errors experienced in the predictable context specifically examined the role of dopamine in responding to low-level stimulus prediction errors whose behavioral outcome is termed motor flexibility.
The current results indicate that the blockade of D2 receptor activation is not sufficient in altering motor flexibility. However, why has the apparent importance of D1 receptor activation during behavioral switching not been suggested by previous human work? One explanation is that preceding human work has focused on D2 receptors (Stelzel et al., 2013; van Holstein et al., 2011). Although it has been shown that bromocriptine alters the neural activity associated with motor flexibility (Stelzel et al., 2013), it had a nonsignificant effect on behavior. Therefore, this previous work is in support of our conclusion that inhibition of D2 receptor activity alone does not impair motor flexibility.
A surprising result was the global reduction in RTs observed in both the haloperidol and sulpiride sessions. Generally, tonic dopamine is thought to enhance behavioral vigor (Niv et al., 2007), so one would expect a relative decrease in RT under levodopa, yet an increase in RT for dopamine antagonists. We have no definitive theoretical explanation to these findings. It is important to highlight RTs during the haloperidol, and sulpiride sessions were only identical for trials without a prediction error, that is, all trials except the unexpected trials in a predictable context. In addition, because this effect was observed with sulpiride, it can be assumed that it was dependent on D2 receptor signaling. As D2 receptor function is crucial for behavioral exploration (Galea, Ruge, Buijink, Bestmann, & Rothwell, 2013), it is probable that D2 receptor activation would lead to greater competition between alternative actions. For trials without a prediction error, this would provide greater competition from alternative actions during the predictable context and a general increase in competition during the unpredictable context. If the BGs were required to bias selection toward the correct action and away from other response alternatives, then such increased competition could slow RTs. D2 blockade would hypothetically decrease competition and allow the prepotent and correct response to be expressed at a faster rate.
We observed no behavioral effect with levodopa. This inability to show opponent behavioral results with levodopa and haloperidol are a common experimental finding (Pine, Shiner, Seymour, & Dolan, 2010; Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006). This may appear at odds with our previous results where levodopa restored motor flexibility in Parkinson disease patients (Galea et al., 2012). Although levodopa causes an increase in tonic dopamine, we previously suggested that it could restore phasic dopamine function in mild Parkinson disease patients (Galea et al., 2012; Grace, 1991). Yet, this effect may only occur in clinical populations who have initial low levels of phasic dopamine function (Grace, 1991). Therefore, in healthy individuals, levodopa might not lead to increased phasic dopamine activity, the mechanism proposed to be crucial for behavioral flexibility (Redgrave et al., 2010). Alternatively the task employed was very simple. Thus, following training, it is likely that participants were at a performance ceiling in terms of their ability to switch behaviors. If this was true then it would not have been possible for levodopa to improve motor flexibility.
Haloperidol not only blocks D1/D2 receptors but also acts as a serotonin and adrenergic receptor antagonist (Kroeze et al., 2003). Our previous results showed that Parkinson disease patients display an identical motor flexibility deficit as haloperidol (Galea et al., 2012). This result lends additional support for a critical role of dopamine during motor flexibility. Nonetheless, it is possible that the behavioral consequences of haloperidol are a result of its inhibitory effect on any of these neuromodulators. For instance, serotonin plays an important role in cognitive flexibility (Clarke, Walker, Dalley, Robbins, & Roberts, 2007; Clarke, Dalley, Crofts, Robbins, & Roberts, 2004), and levels of serotonin are affected in Parkinson disease patients (Politis et al., 2012). Therefore, it is feasible that these effects on serotonin contributed to both the haloperidol results reported here and previous patient results (Galea et al., 2012). However, to counter this, the Parkinson patient's action reprogramming ability improved with levodopa. While levodopa is known to cause dopamine to be released from serotonin terminals, it does not produce serotonin (Carta, Carlsson, Munoz, Kirik, & Bjorklund, 2010), thus suggesting that dopamine provided the dominant contribution to our results. Future research could examine the influence of other serotonin and adrenergic antagonists. However, a general caveat of human pharmacology studies is the unspecific nature of many available neuroactive drugs. Thus, we can only speculate that motor flexibility in humans is particularly sensitive to D1 receptor activity.
In conclusion, high-dose haloperidol impairs the ability to react to unexpected events that elicit large prediction errors and thus requires motor flexibility. This effect is not observed with either specific D2 receptor blockade or a general increase in tonic dopamine levels. To reiterate, as this task was overlearned, such effects were independent of the general role of dopamine in learning. We propose that dopamine D1 receptor function or a combination of D1/D2 receptor function could play an important role in responding to low-level prediction errors about stimulus outcome.
This work was supported by the European Union framework 7 initiative REPLACES (J. M. G., J. C. R.), Birmingham Fellows scheme (J. M. G.), Tourette Syndrome Association (D. R., J. C. R.), The Dorothy Feiss Scientific Research Grant (D. R.), Biotechnology and Biological Sciences Research Council (S. B.), and European Research Council (ActSelectContext, 260424, S. B.). We thank Louise Marshall for helpful comments.
Reprint requests should be sent to Joseph M. Galea, School of Psychology, University of Birmingham, UK, or via e-mail: firstname.lastname@example.org.