Using Primary Reinforcement to Enhance Translatability of a Human Affect and Decision-Making Judgment Bias Task

An important goal in cognitive neuroscience, psychopharmacology, and affective science is the development of translational tasks that can be used to assess affective (emotional) states. For example, the development of novel treatments for affective disorders in humans depends on the use of animal models of affect (Rupniak, 2003). Because we cannot simply ask animals to report their affective state, proxy indicators such as tests of anhedonia (Van der Harst & Spruijt, 2007) or cognitive bias (Mendl, Burman, Parker, & Paul, 2009) are often designed to simulate and assess behavioral characteristics observed in humans experiencing affective states such as depression or anxiety (American Psychiatric Association, 2013; Williams, Mathews, & MacLeod, 1996; Wright & Bower, 1992; MacLeod, Mathews, & Tata, 1986). The judgment bias task has been demonstrated to provide a measure of affective valence (positivity or negativity) in a range of nonhuman animals with relatively “optimistic” (risk-seeking) decision-making being associated with environmental or pharmacological manipulations designed to induce positive affect and relatively “pessimistic” (risk-averse) decision-making being associated with environmental or pharmacological manipulations designed to induce more negative affect (Lagisz et al., 2020; Neville, Nakagawa, et al., 2020). Since its conception in 2004 by Harding, Paul, and Mendl (2004), several studies have used judgment bias in nonhuman animals as a translational tool to investigate human affective disorders and pharmacological treatments for such disorders (Hales, Robinson, & Houghton, 2016; Anderson, Munafò, & Robinson, 2013; Papciak, Popik, Fuchs, & Rygula, 2013; Enkel et al., 2010).

Conducting judgment bias studies with humans, who can report how they feel, may help to provide a better insight into how performance in judgment bias tasks reflects affective state. It might also help to elucidate the putative adaptive function of affect-modulated decision-making, specifically, leading to an understanding of precisely how and why rewarding experiences might lead to positive affect and then to “optimistic” decision-making, and vice versa. However, human judgment bias studies to date have painted a mixed picture. Whereas some studies have found that “pessimism” correlates with subjective reports of more negative affect (Positive and Negative Affect Schedule [Paul et al., 2010], State–Trait Anxiety Inventory [Aylward, Hales, Robinson, & Robinson, 2020; Anderson, Hardcastle, Munafò, & Robinson, 2012], Beck Depression Inventory [Daniel-Watanabe, McLaughlin, Gormley, & Robinson, in press], Visual Analog Scale for anxiety [Anderson et al., 2012]), other studies have found no relationship between reported affect and judgment bias (affect grid and Positive and Negative Affect Schedule [Iigaya et al., 2016], State–Trait Anxiety Inventory [Daniel-Watanabe et al., in press]) or even that “optimistic” decision-making is associated with more negative reported affect (affect grid [Neville, Dayan, Gilchrist, Paul, & Mendl, 2021]).

There are several differences between the human and nonhuman animal judgment bias studies that may have led to these inconsistencies; humans are required to learn the task in a shorter period (i.e., within an hour-long session as opposed to over several days for most nonhuman animals), humans do not live under the highly controlled conditions typical of laboratory animals, and there may be social factors influencing human decision-making (e.g., wanting to perform well to satisfy the experimenter). However, one difference that might be of particular significance is that judgment bias studies in nonhuman animals are typically conducted using primary reinforcers such as food or electric shocks, whereas all judgment bias testing in humans has used secondary reinforcers such as monetary gain or loss.

Many studies have identified differences between the neural processing of primary and second reinforcers (Sescousse, Caldú, Segura, & Dreher, 2013; Delgado, Jou, & Phelps, 2011; Beck, Locke, Savine, Jimura, & Braver, 2010; Grimm & See, 2000). For example, it has been demonstrated that primary rewards are more strongly represented in evolutionary older brain regions, such as the anterior insula, whereas secondary rewards are more strongly represented in evolutionary newer brain regions, such as the anterior OFC (Sescousse et al., 2013; Delgado et al., 2011). Given these differences, it is important to develop tasks for humans that utilize primary reinforcers and hence more closely resemble animal tasks and to investigate whether they yield similar findings to the typical secondary reinforcer human task. The use of primary reinforcers in this way is particularly pertinent to our understanding of judgment bias from a functional and evolutionary perspective, given that affect is hypothesized both to reflect ongoing and prior experience of rewards and punishers and also to mediate the way this experience guides decision-making (Bach & Dayan, 2017; Marshall, Trimmer, Houston, & McNamara, 2013; Nettle & Bateson, 2012; Mendl, Burman, & Paul, 2010). Moreover, primary reinforcers may tap into fundamental affective processes and pathologies that underlie affective disorders more reliably than secondary reinforcers, given their more direct relevance to our ability to survive and reproduce. Consequently, to better understand judgment bias as a measure of affect, we need a task for humans in which we administer primary reinforcers and in which we can obtain a fine-scale picture of sequential effects that might be influencing decision-making and affective state.

To this end, we aimed to develop a translation of the automated rat judgment bias task (Neville, King, et al., 2020; Jones et al., 2018) for humans that uses primary reinforcers: apple juice and cold salty tea (Pauli et al., 2015). Using this novel variant of the judgment bias task alongside computational modeling, we additionally aimed to elucidate the extent to which latent processes underlying decision-making might relate to subjective experiences of affect, reward, and punishment within the task. We hypothesized that more positive reported affect would be associated with model parameters characterizing biases toward the “optimistic” response and that, consistent with previous research (Neville et al., 2021; Rutledge, Skandali, Dayan, & Dolan, 2014), the absolute favorability (i.e., average earning rate) of within-task experience would inform decision-making and the relative favorability (i.e., reward prediction error) of within-task experience would inform reported affective valence.

Thirty-three people from the Bristol Veterinary School community participated in the study. All participants provided written informed consent, and the study was approved by the Faculty of Science Research Ethics Committee at the University of Bristol. This sample size was based on a previous human judgment study (39 participants, across two conditions [Neville et al., 2021]) and a previous study using primary reinforcers as part of a conditioning paradigm (29 participants [Pauli et al., 2015]).

The inclusion criteria were that the participant enjoyed apple juice; was not allergic to apple juice, salt, or black tea; was not hypertensive or had any medical condition that meant that he or she must limit his or her salt intake; and was over the age of 18 years. Participants were asked to abstain from drinking anything (except water) or eating in the hour before the study. To cover their time and expenses, participants were paid £7 for the hour-long session. To encourage full engagement with the study, participants were informed that the top three ranking participants in terms of accuracy would receive an additional £7 bonus.

The task was conducted on a laptop (Dell Latitude), which was connected to two syringe pumps (SPLG100, World Precision Instruments) that were set to pump liquid at a rate of 2 mL per minute. Sterilized food-grade PVC tubing (outer diameter: 11 mm, inner diameter: 8 mm) were attached to syringes (Becton Dickinson; 50-mL Plastipak) that were driven by the syringe pumps. This tubing was connected via a tube connector to smaller sterilized food-grade PVC tubing (outer diameter: 6 mm, inner diameter: 4 mm) and held in place in front of the participant using a retort clamp and stand, the height of which could be adjusted by the participant at the start of the experiment. The participants placed the end of these tubes in their mouth. Individuals made responses on a keyboard connected to the laptop. The code for the task was written in MATLAB (MathWorks, Inc.) using the PsychToolBox extensions (Kleiner et al., 2007; Brainard, 1997).

The methodology was adapted from that of the human monetary judgment bias task (Neville et al., 2021), which itself was a translation of a rodent judgment bias task (Jones et al., 2018). On each trial of the task, participants were instructed to press and hold the enter key before being presented with a fixation cross for 1000 msec followed by a random dot kinematogram (RDK) displayed for 2000 msec, which across trials varied in direction of motion (leftward or rightward) and coherence (proportion of dots moving in a coherent direction: 0.01, 0.02, 0.16, or 0.32). Participants had two options when the RDK was presented: to release the key before the end of the 2-sec RDK presentation (“leave”) or continue to hold the key for 2 sec (“stay”). The outcome associated with either response depended on the stage of training and the direction of the RDK, with one direction being favorable and requiring a “stay” response to gain reward and the other being unfavorable and requiring a “leave” response to avoid punishment (either leftward or rightward, counterbalanced across participants).

Participants were provided with written instructions about the rules of the task and were then provided with practice trials that were composed of two blocks of 48 trials. The aim of the first practice block was to introduce the participants to the task and train them on the correct responses to the RDK. In this first practice block, the word “correct” was shown on screen for correct responses (i.e., those where “stay” was executed when the RDK direction was favorable, and “leave” was executed when the RDK direction was unfavorable), and “incorrect” was shown on screen for incorrect responses. The direction of motion of the RDK was very easy to detect (coherence = 0.32) on all trials, and the direction of motion was leftward on 50% of trials and rightward on the remaining 50%.

The aim of the second practice block was to acquaint participants with the delivery and taste of the apple juice (Apple Juice from Concentrate, Morrisons) and salty tea. Salty tea was prepared each morning as per Pauli et al. (2015) with two black tea bags (Bettys & Taylors of Harrogate; Yorkshire Tea) and 29 g of salt per liter of boiling water, which was chilled before data collection. We opted to use salty tea as opposed to electric shocks as we considered this to be a milder punisher (hence less of an ethical concern) and so that the modality of the punisher was the same as the reward (i.e., both gustatory). Importantly, liquid reinforcers such as juice and salty tea have successfully been used in fMRI studies (Pauli et al., 2015; Metereau & Dreher, 2013; Kim, Shimojo, & O'Doherty, 2011), hence their use in our task paves the way for future investigations of the neural underpinnings of judgment bias in humans. In this block, the direction of the RDK was easy to detect (coherence = 0.16) on 50% of trials, of which half were leftward and half were rightward, and moderately difficult to detect (coherence = 0.02) on the remaining 50%, of which half were leftward and half were rightward. The volume of juice received for “stay” responses when the direction of the RDK was favorable was 0.457 mL, and likewise, the volume of salty tea received for “stay” responses when the RDK was unfavorable was 0.457 mL. While juice was delivered and also for 3000 msec after delivery, the words “Juice delivered” were displayed on screen, whereas during salty tea delivery and for 3000 msec after this, the words “Salty tea delivered” were displayed on screen. The potential volume of juice on each trial (i.e., “0.457 mL”) was shown above an orange-colored bar with a height proportional to the potential juice volume, similarly the volume of salty tea (i.e., “0.457 mL”) was displayed below a brown bar positioned directly below the orange-colored bar with a height proportional to the potential tea volume. These bars were shown before the instruction to press and hold the enter key. The participant received nothing for making the “leave” response, and the words “Nothing delivered” were displayed on screen for 3000 msec. Across all blocks, the directions and coherence levels of the RDKs were randomized across trials before the start of the study so that the order of these was identical for all participants.

The test session (see Figure 1) comprised 90 trials on which the RDK moved leftward and 90 trials on which it moved rightward. For each direction, 30 trials had coherence levels of low (0.01), moderate (0.02), and high (0.16). RDKs with low and moderate coherence levels are the ambiguous probe cues, whereas those with a high coherence level are the reference cues. Hence, each possible stimulus was shown on 16.7% of trials. The potential volume of juice fluctuated across trials according to a noisy sine wave with a mean volume of 0.457 mL and a standard deviation of 0.216 mL, ranging from a minimum of 0.021 mL to a maximum of 0.781 mL. As a result of ethical concerns about the effect of large quantities of salty tea, the potential volume of salty tea remained at 0.457 mL throughout testing. As in the second practice block, the potential volume of juice and salty tea was displayed on screen both as text and graphically using two colored bars with heights proportional to the volume of juice and salty tea. Before the first trial and every subsequent 10 trials, participants were asked to report how they were feeling using a computerized affect grid (Killgore, 1998). To complete the affect grid, participants had to move a cross that was initially central in the grid to the location that best described their current affective state using the arrow keys on a keyboard. Horizontal movements represented changes in affective valence, with movements to the right reporting a more positively valenced affect. Vertical movements represented arousal, with upward movement reporting higher levels of arousal.

Two participants were excluded from data analysis because of poor performance at the reference cues, which we defined as not making the correct response significantly above chance across both reference cues according to a one-tailed binomial test (Participant 12, p = .18; Participant 29, p = .99).

We conducted both a model-dependent and model-agnostic analysis of the data. The aim of the model-dependent analysis was to investigate the latent processes involved in decision-making and how these might be modulated by different aspects of reward and punisher experience. Judgment bias RT data (“stay”: 2 sec; “leave”: <2 sec) were fitted to the partially observable Markov decision process (POMDP) model described in full by Neville et al. (2021). Briefly, we consider that participants transition through a 2-D state space in which they accumulate evidence about the true direction of the presented stimulus as informed by their observations (Dimension 1) across the discretized duration of the trial (Dimension 2). The probability on a given trial that an individual will opt for the safe “leave” response, and the speed with which they do so, will depend on their transitions through the state space and the value of occupying each state.

In the model, the movement through the state space and the values are determined by a number of parameters including those characterizing the psychometric function, namely, σ, which reflects the ability of the participant to discriminate between the stimuli, and λ_ref and λ_amb, which are lapse rates for the reference and ambiguous stimuli, respectively; those characterizing the hazard function (probability of “timeout,” i.e., time elapsed > 2 sec, given “timeout” has not already occurred) to account for the possibility of making the “stay” response by default because of inaccurate timekeeping (φ and ζ); and an inverse temperature parameter (B) to reflect decision stochasticity. In addition, we included a set of parameters that allowed for biases toward or away from the “optimistic” response. This set of parameters was composed of a baseline bias parameter (⁠$β0δ$⁠) capturing overall tendencies for risk-seeking or risk aversion, as well as the overall dislike for the salty tea and enjoyment of the apple juice, and parameters that encompassed the potential effect of the average earning rate (⁠$βR¯δ$⁠), most recent outcome (⁠$βOδ$⁠), weighted prediction error (⁠$βwPEδ$⁠), and squared weighted prediction error (⁠$βwPE2δ$⁠) on bias, as well as a parameter that allowed for decision-making to vary as a function of the number of trials completed (β_n) to account for fatigue. The average earning rate reflects the learnt value of the test session from previous juice and salty tea intake and updates according to a Rescorla–Wagner learning model, with learning rate $αR¯$ (following Neville et al., 2021; Guitart-Masip, Beierholm, Dolan, Duzel, & Dayan, 2011), whereas the weighted reward prediction error is the difference between the model-predicted outcome and the actual outcome across trials weighted such that the influence of past prediction errors attenuates over trials, with forgetting factor γ_wPE (following Neville et al., 2021; Rutledge et al., 2014).

Models were fitted to the RT data using maximum likelihood, with multiple starting values (including values found to provide the best fit for the core model as starting values) because of nonconvexity. Parameters that characterized decision-making in the absence of biases (i.e., B, λ_amb, λ_ref σ, φ, and ζ) were included in all models to account for their potential influence on behavior. We then added parameters in a stepwise manner, first assessing whether the parameter that characterized constant biases in decision-making improved model parsimony and then those that characterized within-task variation in decision-making. We then assessed whether the addition of single parameters to the best-fitting model would further improve the model fit. Models were compared using Bayesian information criterion (BIC) values, and we additionally compared the final set of models using the Akaike information criterion (AIC). A stepwise model-fitting procedure was employed as fitting all possible models was not feasible because of the large number of possible models and the computational intensiveness of model fitting. Model fitting was carried out using the computational facilities of the Advanced Computing Research Centre at the University of Bristol.

The parameter estimates from the most parsimonious model were analyzed using permutation tests to assess whether they varied significantly from zero, where this was meaningful (i.e., for $β0δ$ and κ).

The aim of the model-agnostic analysis was to investigate the relationship, first, between primary reward and punisher experience and reported affect and, second, between the parameters characterizing decision-making and reported affect. These analyses were conducted in R (R Core Team, 2015) using the nlme package (Pinheiro, Bates, DebRoy, Sarkar, & R Core Team, 2017). Likelihood ratio tests were used to assess whether the difference in model deviance was significant after removal of a parameter from a model.

The reported valence (affect grid x coordinate) and reported arousal (affect grid y coordinate) were analyzed using general linear mixed models (GLMMs) in which both the intercept and slope were allowed to vary among participants, with the predictor variables for which a random slope was included determined using BIC comparison (i.e., comparing a model with the variable included in the random effects structure to the model without the variable included in the random effects structure). The predictor variables were those encompassing reward and punisher experience, included those derived from the best-fitting POMDP model: the most recent volume of offered juice, weighted reward prediction error (wPE), squared weighted reward prediction error (wPE²), the previous outcome O, total juice consumed, the number of trials completed, and the average earning rate $R¯$⁠. Because of correlations between wPE, wPE², and O, and also between total juice consumed, number of trials completed, and $R¯$⁠, these predictor variables were not included in the same GLMM but instead were included separately, and each model was then compared using their BIC values. The GLMMs that provided the best fit were analyzed further.

In addition, the parameter estimates from the POMDP model were analyzed using a general linear model with the mean reported arousal and mean reported valence as fixed effects, except ζ and φ, in which the mean timeout probability that was jointly determined by these two parameters was instead used for a more intuitive interpretation of the results, as in Neville et al. (2021). The values of B were log-transformed because of the presence of extreme outliers exerting undue influence on the model; the GLMM was also run after removal of these outliers.

The most parsimonious model of judgment bias RT, according to the BIC values, included the following core parameters: B, λ_amb, λ_ref σ, φ, and ζ, which characterized decision-making in the absence of biases, in addition to $β0δ$⁠, characterizing baseline biases, and κ, characterizing the rate of satiation (see Table 1). This model fit the observed data well (see Figure 2). The AIC values indicated that inclusion of $βR¯δ$ or $βwPEδ$ outperformed the BIC best model. However, permutation tests revealed that the estimates of these parameters did not differ significantly from zero (⁠$βR¯δ$⁠: mean = −0.13, SE = 0.11, p = .28; $βwPEδ$⁠: mean = 0.005, SE = 0.01, p = .72). Hence, there was no strong evidence for the inclusion of these parameters, and we selected the BIC best model as our final model for further analysis.

Visual inspection of the parameter estimates revealed several potential outliers. These were formally defined as estimates that were either above the third quartile or below the first quartile by more than 1.5 times the interquartile range (i.e., below Q1–1.5 × IQR or above Q3 + 1.5 × IQR). All analyses were conducted both before and after the removal of these outliers. The presented results are from the analyses including the outliers, except where the removal of outliers changed the results qualitatively, in which case both results are presented.

The parameter characterizing satiation was significantly lower than zero (κ: mean = −5.84, SE = 4.23, p < .001), reflecting the fact that participants valued juice less with increasing juice consumption. Overall, the subjective value of the juice depreciated across the test session and, for much of the test session, remained less appetitive than the salty tea was aversive (Figure 3). This decrease in the subjective value of the juice exerted a strong influence on behavior; participants were more risk-averse than would be expected if there was no effect of satiation (Figure 4). The effect of satiation was also reflected in the observed decrease in RTs and proportion of “stay” responses made as the number of trials increased (Figure 5). The estimates of κ were not associated with either reported valence (beta weight = −5.72, SE = 4.64, likelihood ratio test [LRT] = 2.03, p = .16) or arousal (beta weight = 0.98, SE = 4.64, LRT = 0.075, p = .79).

The bias parameter was significantly greater than zero (⁠$β0δ$⁠: mean = 0.05, SE < 0.01, p < .001), reflecting the participants' overall bias toward persisting with the “stay” response for a longer duration on a trial than would otherwise be anticipated (Figure 4), but this parameter was not predicted by reported affective valence (beta weight < 0.01, SE < 0.01, LRT < 0.01, p = .95) or arousal (beta weight < 0.01, SE < 0.01, LRT = 0.55, p = .46). Thus, although the value of the juice depreciated considerably over trials (Figure 3), participants occasionally continued to risk making the “stay” response, which was driven by this fixed bias. There was no evidence that the extent of this bias, or the speed of satiation, was influenced by affective state.

Participants who reported lower arousal tended to make more errors to the ambiguous stimulus; the association between the lapse rate and reported arousal was marginally nonsignificant (λ_amb: beta weight = −0.02, SE = 0.01, LRT = 3.77, p = .052). The reference lapse rate did not depend on reported arousal (λ_ref: beta weight < 0.01, SE < 0.01, LRT = 2.04, p = .15). Neither λ_amb (beta weight < 0.01, SE = 0.01, LRT < 0.01, p = .93) nor λ_ref (beta weight < 0.01, SE < 0.01, LRT = 2.13, p = .15) depended on reported valence. The slope parameter, σ, did not significantly depend on reported affective valence (beta weight = 0.04, SE = 0.04, LRT = 1.10, p = .30) or arousal (beta weight = 0.03, SE = 0.04, LRT = 0.78, p = .38). Likewise, the mean timeout probability (determined by ζ and φ) was not significantly associated with reported affective valence (beta weight < 0.01, SE = 0.03, LRT < 0.01, p = .95) or arousal (beta weight = 0.01, SE = 0.03, LRT = 0.41, p = .52).

Participants who reported more positive valence had significantly higher inverse temperature parameter values (B: beta weight = 1.05, SE = 0.48, LRT = 4.92, p = .027; Figure 6), and there was no significant relationship between affective arousal and B (beta weight = 0.22, SE = 0.48, LRT = 0.23, p = .63). However, after outlier removal, no significant effect of valence (beta weight = −0.21, SE = 0.30, LRT = 0.52, p = .47) or arousal (beta weight = −0.46, SE = 0.28, LRT = 2.97, p = .085) on inverse temperature was observed (Figure 6). The median value of the inverse temperature parameters was 336.4297 with an interquartile range of 1224.997, and any value over 3172.95 was deemed to be an outlier. The six outliers identified, whose removal altered the results qualitatively, had values of 4099.67, 5309.31, 82248.43, 15654.76, 121346.67, and 2779806.37 (Figure 6).

The best-fitting GLMM of reported valence included the potential outcome as solely a fixed effect (ΔBIC = 12.27 vs. inclusion as a random linear effect), number of trials completed with a slope coefficient that varied among participants (ΔBIC = 26.91 vs. solely a fixed effect; as opposed to the average earning rate − ΔBIC = 26.75, and total juice consumed − ΔBIC = 1.10), and weighted prediction error as solely a fixed effect (ΔBIC = 14.98 vs. inclusion as a random linear effect; as opposed to the squared weighted prediction error − ΔBIC = 54.25, or previous outcome ΔBIC = 38.14). Reported valence was positively correlated with both the prediction error (beta weight = 40.70, SE = 5.34, LRT = 52.14, p < .001) and potential outcome (beta weight = 14.30, SE = 4.24, LRT = 11.33, p < .001). The random slope coefficients for trial did not differ significantly from zero according to a permutation test (mean = −0.03, SE = 0.12, p = .81).

The best-fitting GLMM of reported arousal included potential outcome as solely a fixed effect (ΔBIC = 15.34 vs. inclusion as a random linear effect), number of trials completed with a slope coefficient that varied among participants (ΔBIC = 71.70 vs. solely a fixed effect; as opposed to the average earning rate − ΔBIC = 83.80, or total juice consumed − ΔBIC = 0.95), and squared prediction error as solely a fixed effect (ΔBIC = 18.37 vs. inclusion as a random linear effect; as opposed to the weighted prediction error − ΔBIC = 1.11, or previous outcome − ΔBIC = 1.12). A higher potential outcome tended to be associated with greater reported arousal (beta weight = 7.06, SE = 4.66, LRT = 2.89, p = .09). The squared weight prediction error had no significant effect of affective arousal (beta weight = 4.96, SE = 4.66, LRT = 1.14, p = .26). The random slope coefficients for trial were overall significantly lower than zero (mean = −0.52, SE = 0.17, p = .004), suggesting a decrease in reported arousal as the test session progressed.

In this study, we developed a human judgment bias task that closely mirrored nonhuman animal versions of the task by using primary, rather than secondary, reinforcers. We did this to increase the translatability of the task and to allow more effective use of results from the human task to shed light on judgment bias as a measure of affect in nonhuman animals. To achieve this, we modified a human judgment bias task, which was itself originally designed as a translation of a rodent judgment bias task, by replacing monetary gain with apple juice and monetary loss with salty tea. We then sought to investigate the relationship between prior experience, affect, and decision-making using computational modeling. The task itself was largely successful, with more than 90% of participants performing better than chance in their responses to the reference cues to which they had been trained. Furthermore, the data produced were highly amenable to computational modeling.

Our initial analyses demonstrated that the favorability of the potential outcome of participants' responses modulated self-reported affective valence, with a greater volume of offered juice leading to more positive affective valence. There was also a tendency for a greater volume of offered juice to lead to greater reported affective arousal. This supports the definition of “emotion” (i.e., short-term affect) as a state elicited by (anticipated) rewards or punishers (Mendl & Paul, 2020; Rolls, 2013). It also indicates that participants were sensitive to the fluctuating volume of potential apple juice and were, indeed, perceiving larger volumes of apple juice as more rewarding. This demonstrates that the manipulation of reward experience worked well, having induced the anticipated shifts in affective states. The finding that positive affective valence was related to recently experiencing rewards that were greater than anticipated (positive reward prediction errors) aligns with the results of a number of other studies (Neville et al., 2021; Otto & Eichstaedt, 2018; Rutledge et al., 2014).

Our results provide evidence for a complex relationship between affect and decision-making that could only be teased apart through use of computational modeling. Contrary to our expectations, there was no evidence in this study that an “optimistic” bias was directly associated with more positive reported affect. This conflicts with some (but not all) previous findings from judgment bias studies in humans and with the meta-analytic findings in animal studies that relatively positive affective states are associated with more “optimistic” decision-making (Lagisz et al., 2020; Neville, Nakagawa, et al., 2020). Importantly, the meta-analyses identified a small-to-moderate effect size with high heterogeneity, using manipulations including those designed to induce substantial shifts in affective valence (e.g., enriched vs. barren housing; administration of pharmacological substances; restraint stress). It is also possible that the primary reinforcers used in these animal studies may have been far more salient to their nonhuman subjects than the juice/salty tea was for the humans in this study (e.g., animals were food restricted; animals were maintained on a nonvaried diet). Given this, it is perhaps not entirely unsurprising that no significant relationship was found between the parameter estimates encompassing “optimism” and affective valence in our nonclinical population of human participants. These findings, first, raise the question of whether judgment bias can be sufficiently sensitive to detect more subtle variation in baseline affect and, second, highlights that contextual differences in how humans and animals experience the tasks may present significant hurdles to the development of truly translational and directly comparable tests, even given the explicit attempt.

We did find that, when looking at the data as a whole, the inverse temperature parameter was associated with reported valence; lower decision stochasticity was associated with more positive reported affect. This result is consistent with findings that depression, a clinically negatively valenced affective state, has been associated with more stochastic decision-making (Harlé, Guo, Zhang, Paulus, & Yu, 2017; Huys, Pizzagalli, Bogdan, & Dayan, 2013). However, this result appeared to be driven by a subset of participants whose data revealed large outlying estimates of the inverse temperature parameter and who tended to report affective valence at the more positive end of the scale. The cause of the outlying values is unclear; speculatively, it could reflect a trait characterized by heightened sensitivity to rewards and punishers and overall more positive affect. It could also reflect that the model fits particularly well to these participants (given that the inverse temperature parameter can also capture model misfit), although the reasons for this are not apparent. Nonetheless, the results of our study imply a relationship between affective valence and the cognitive processes underlying judgment bias. The potential link between decision stochasticity and affect requires further investigation.

We also identified a weak relationship between affective arousal and the cognitive processes underlying judgment bias. In particular, a higher ambiguous stimulus lapse rate tended to be associated with lower arousal, which corroborates results from the monetary version of the task (Neville et al., 2021) and likely reflects that higher arousal may lead to higher engagement with the task and result in better performance. It would therefore be useful to assess whether lapse rates, extracted using computational modeling, could provide a measure of arousal in judgment bias tasks for nonhuman animals.

Our results also highlighted the influence of reward and punisher experience on both affect and decision-making. First, the total juice consumed by the participant was the largest experiential contributor to their decision-making. The value of the juice decreased as a function of increasing juice intake, which is consistent with participants becoming satiated. In accordance with the findings of the monetary version of the task (Neville et al., 2021), we also found decreasing arousal as more trials had been completed, which may indicate that participants found the task to be tiring. Greater consideration should thus be given to the potential for satiation and potential fatigue (and individual differences that lead to variation in how rapidly an individual becomes sated) when conducting nonhuman animal judgment bias tasks.

The degree of satiation indicated in our human participants was arguably extreme; the absolute subjective value (i.e., subjective magnitude) of the juice remained largely below the value of the salty tea after approximately 20 trials. Yet, despite this, and even after accounting for errors that were characterized by a lapse rate, participants continued to make the “optimistic” response more often than would be anticipated; they exhibited an overall “optimistic” bias. The precise nature of this bias is unclear. One possibility is that this bias, particularly at the favorable and near-favorable cue, reflects the intrinsic reward associated with making accurate (correct) responses. Similarly, it may reflect that the feedback provided by the “stay” response was itself rewarding, given that the “leave” response provides no feedback about the correct action. This would be in line with studies that have shown that being correct is in itself rewarding (Satterthwaite et al., 2012; Han, Huettel, Raposo, Adcock, & Dobbins, 2010). An alternative is that the “gamble” of the “stay” may have elicited a rewarding frisson of excitement, particularly given the repetitive and likely dull nature of the task. Studies have shown that risky decision-making can induce excitement and that this increases under boredom (Kılıç, van Tilburg, & Igou, 2020; Binde, 2013; Mercer & Eastwood, 2010). This further highlights issues in developing truly translational versions of the judgment bias task; nonhuman animals may not experience an intrinsic reward associated with being correct, and judgment bias tasks for nonhuman animals have been suggested to be enriching for them (Krakenberg et al., 2021; Mallien et al., 2016).

The extent to which recent outcomes were better or worse than anticipated was found to determine self-reported affect. This is in agreement with previous studies demonstrating that prediction error is a key determinant of subjectively experienced affective valence in humans (Neville et al., 2021; Otto & Eichstaedt, 2018; Rutledge et al., 2014), at least in relation to short-term rewarding experiences.

As outlined above, several findings were consistent between this task and the monetary version from which it was adapted: A higher lapse rate is associated with lower affective arousal, more positive prediction errors are associated with more positive reported affective valence, and reported affective arousal decreases as a function of number of trials completed. However, there were also a couple of inconsistencies. First, we found no evidence for a relationship between time estimation and affective valence, and second, we found no effect of the average earning rate or previous outcome on judgment bias (Neville et al., 2021). It is unclear whether this might stem from differences in the processing of primary and secondary rewards, or other factors such as satiation having an overriding effect on behavior. This therefore warrants further examination.

In summary, we have developed a novel version of the judgment bias task for humans that uses primary as opposed to secondary reinforcers and, as a result, is more comparable to versions of the task designed for nonhuman animals. We identified a relationship between decision stochasticity and affective valence. Specifically, we generally observed higher decision stochasticity in participants who reported more negatively valenced affect, although only when a cluster of outlying participants were included. We also found that lower reported affective arousal tended to be associated with a greater propensity for errors. The decision-making processes underlying judgment bias on this task are hence linked to both reported affective valence and arousal. In addition, we confirmed some, but not all, previous findings that had been observed in a monetary version of the task, such as the positive association between prediction errors and reported valence. We conclude that our novel version of the judgment bias task provides a means to investigate the relationship between affect and decision-making in greater depth and in a manner that is more comparable to animal versions of the task.

Reprint requests should be sent to Vikki Neville, Bristol Veterinary School, University of Bristol, Langford BS40 5DU, United Kingdom, or via e-mail: [email protected].

Vikki Neville: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Writing—Original draft; Writing—Review & editing. Peter Dayan: Conceptualization; Formal analysis; Methodology; Supervision; Writing—Original draft; Writing—Review & editing. Iain D. Gilchrist: Conceptualization; Methodology; Supervision; Writing—Review & editing. Elizabeth S. Paul: Conceptualization; Methodology; Supervision; Writing—Review & editing. Michael Mendl: Conceptualization; Methodology; Supervision; Writing—Original draft; Writing—Review & editing.

Vikki Neville, Biotechnology and Biological Sciences Research Council (https://dx.doi.org/10.13039/501100000268), grant number: BB/M009122/1. Peter Dayan, Alexander von Humboldt-Stiftung (https://dx.doi.org/10.13039/100005156). Peter Dayan, Max-Planck-Gesellschaft (https://dx.doi.org/10.13039/501100004189). All authors, Biotechnology and Biological Sciences Research Council (https://dx.doi.org/10.13039/501100000268), grant number: BB/T002654/1.

A retrospective analysis of the citations in every article published in this journal from 2010 to 2020 has revealed a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .408, W(oman)/M = .335, M/W = .108, and W/W = .149, the comparable proportions for the articles that these authorship teams cited were M/M = .579, W/M = .243, M/W = .102, and W/W = .076 (Fulvio et al., JoCN, 33:1, pp. 3–7). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.