Diffusion models have become essential for describing the performance and statistics of reaction times in human decision making. Despite their success, it is not known how to evaluate decision confidence from them. I introduce a broader class of models consisting of two partially correlated neuronal integrators with arbitrarily time-varying decision boundaries that allow a natural description of confidence. The dependence of decision confidence on the state of the losing integrator, decision time, time-varying boundaries, and correlations is analytically described. The marginal confidence is computed for the half-anticorrelated case using the exact solution of the diffusion process with constant boundaries and compared to that of the independent and completely anticorrelated cases.
In most interesting situations, detection of a signal involves discriminating a meaningful from a random energy fluctuation. A standard procedure employed to discriminate between these two alternatives is to integrate over time such fluctuations and decide that there is a signal embedded in the noise if the accumulated sum reaches a predetermined threshold and that there is no signal otherwise. Although this is just the way some photosensors are used to detect dim stars, humans seem to employ a remarkably similar strategy to make decisions. Human performance and reaction time statistics in two-alternative forced-choice tasks are accurately fit by models consisting of a diffusion process of an underlying variable to decision boundaries (Ratcliff, 1978; Ratcliff & Smith, 2004; Luce, 1986; Usher & McClelland, 2001; Bogacz et al., 2006; Mazurek, Roitman, Ditterich, & Shadlen, 2003; Ratcliff, Hasegawa, Hasegawa, Smith, & Segraves, 2007; Wang, 2002; Beck et al., 2008). Moreover, diffusion to bound processes seem to be neurally hardwired in several areas of the brain, where the neurons' firing rate reflects integration of noisy information for alternatives and determines choices and decision times when it reaches a fixed threshold (Hanes & Schall, 1996; Shadlen & Newsome, 1996; Platt & Glimcher, 1999; Ratcliff et al., 2007; see Schall, 2001, for a review).
Upon a decision, a crucial variable is the degree of confidence about the choice. In detecting a new particle or star, the estimated confidence will determine if we can believe in the result. Humans and animals are able to reliably estimate confidence or uncertainty about their decisions and use it to change or abort behaviors (Vickers, 1979; Juslin & Olsson, 1997; Vickers & Lee, 1998; Landy, Goutcher, Trommershuser, & Mamassian, 2007; Whiteley & Sahani, 2008; Kepecs, Uchida, Zariwala, & Mainen, 2008). Human subjects can be asked directly to report their confidence, but it is also possible to use choice behaviors in humans and animals to estimate their decision confidence. For instance, a subject can be presented with two choices: one that involves high but uncertain reward and another with low but certain reward. The confidence that the subject has about the uncertain reward at any trial will determine which option is taken, and therefore decision confidence can be estimated from its behavior at any trial. Despite the importance of decision confidence, very little is known about how it is encoded in the nervous system (but see Kepecs et al., 2008; Beck et al., 2008). Furthermore, the dependence of confidence on essential parameters of the decision process like reaction time or distance to boundary of the losing integrator has not been established. Knowing such dependencies on simple models would allow a quantitative comparison between theory and both behavioral and neurophysiological data.
This letter describes analytically decision confidence for a decision diffusion model (DDM) consisting of two neuronal integrators receiving partially correlated inputs. The main results are as follows: (1) decision confidence depends on the difference between the traveled distances by the two integrators in a general way, regardless of the correlation in the inputs and on the time dependence of the decision thresholds; (2) accumulation of evidence over time occurs without any leak or loss, that is, the final state of the integrators and elapsed time fully specify the a posteriori probability distribution over the parameters on which the decision is based; (3) marginal confidence is computed for the independent, half-anticorrelated, and completely anticorrelated cases as a function of the decision time, being at a maximum for independent integrators; and (4) the distributions of reaction times for the above cases are characterized. Models with several integrators similar to the one proposed here have been introduced before to describe qualitatively decision confidence (Vickers, 1979), but analytical expressions for confidence are in general not known for these simple models (Juslin & Olsson, 1997; Vickers & Lee, 1998; Van Zandt, 2000; Pleskac & Busemeyer, 2007). It is worth mentioning that the DDM considered here reduces to the classical DDM (Ratcliff & Smith, 2004; Luce, 1986) when the two integrators receive completely anticorrelated inputs, for which decision confidence as a function of the decision time is also completely characterized for arbitrary time-dependent boundaries.
A two-alternative forced-choice task is modeled as follows. A decision for choice k (k = 1, 2) is made whenever the variable xk reaches its threshold Θk(t) first. It can be assumed that the decision is correct if the boundary first reached corresponds to that of the integrator with the highest drift. This is a general situation that, for instance, could correspond to a task in which the goal is to detect the brighter of two lights. It also arises naturally in experiments in which the direction of motion of a random-dot motion display has to be determined during short presentations (Shadlen & Newsome, 1996; Mazurek et al., 2003). Here, the direction of motion can be modeled by the sign of the drift rates of the two integrators (i.e., μ = μ1 = −μ2>0 indicates rightward motion, while μ = μ1 = −μ2 < 0 indicates leftward motion). The difficulty of this task is controlled by the motion coherence, that is, the percentage of dots that move in the same direction as opposed to random directions, and it can be modeled by the magnitude of the drift rates, |μ| = |μ1| = |μ2|. It will be assumed that the variance of the noise and correlation coefficient remains constant across all stimulation conditions. I assume that when a decision has been made, the state of the losing integrator can be read out. Figure 1 illustrates the model for the independent (ρ = 0 in Figure 1A) and half-anticorrelated (ρ = 1/2, ν = −1, in Figure 1C) cases with time-varying thresholds.
When ρ = 0 in equations 2.1, the two integrators become independent, and then the system becomes a race model (see, e.g., Vickers, 1979; Luce, 1986). Therefore, the model defined in equations 2.1 covers a wide spectrum of classical models of decision making, ranging from the classical DDM for ρ = 1 and ν = −1, to the race model for ρ = 0, as shown in Figure 2. However, the model considered here does not include interactions or connections between the neuronal integrators, as in Usher & McClelland (2001). A discussion about these models has been recently presented in Bogacz (2007).
3.1. Independence of the Probability Distributions over the Drift Rates on Decision Boundaries.
A decision about which drift is higher (μ1>μ2 or μ2>μ1?) in the system defined in equations 2.1 will depend on the drift estimates at decision time t given the observed trajectories of the integrators up to that time, denoted and , with initial states x1(0) = a and x2(0) = b at t = 0. The critical quantity is then the probability density of having the drifts μ1 and μ2 given the trajectories and up to time t, . Instead of working directly with this density, it is more convenient to introduce the new variables u = x1 + x2 and v = x1 − x2 and work with the probability density of having drifts μu = μ1 + μ2 and μv = μ1 − μ2 given the trajectories and , with initial conditions u0 = a + b and v0 = a − b. In the new variables, equations 2.1 transform into two independent Wiener processes, with means μu and μv, and variances σ2u = 2σ2(1 + ρν) and σ2v = 2σ2(1 − ρν), making the calculations easier.
Now I consider the case in which there are boundaries Θk(t) (k = 1, 2). In this case, only trajectories that do not cross the boundaries at any time before t (i.e., such that u + v < 2Θ1(τ) and u − v < 2Θ2(τ) for all τ < t) are realizable, while the trajectories that cross the boundaries before time t are absorbed. In what follows, I show the central result that the probability distribution of the drift rates given the traveled distances by the integrators and the elapsed time when boundaries are present is identical to the probability distribution of the drift rates when there are not boundaries, equation 3.2.
Two steps will be followed in the proof. First, it is shown that the probability density of having the state (u, v) at time t when arbitrary boundaries are present can be factorized in two terms: one that depends on the drift rates but not on the boundaries and another that depends on the boundaries but not on the drift rates. Second, it is shown that this factorization and Bayes' theorem naturally lead to equation 3.2 for uniform a priori distributions over the drift rates. This proof is similar to that derived by Beck et al. (2008) for Poisson-like distributions.
Finally, one needs to show that all information contained in the trajectory is preserved in the final state of the integrators and elapsed time in the presence of boundaries. Since for any trajectory that does not cross the bound at any time before t, , one has , then using equations 3.2 and 3.6, . Therefore, the integrators accumulate evidence with any leak or loss. This interestingly means that, as in the case without boundaries, the shape of the trajectory does not need to be remembered, since the final state of the integrators and elapsed time contain all necessary information in order to perform inference about the drift rates. More formally, the distances traveled by the integrators along with the elapsed time are sufficient statistics for the drift rates.
The equivalence between probability distributions with and without boundaries can be understood intuitively as follows. Note, first, that any trajectory that does not cross the boundaries at any time before t and has the same initial and final points leads to the same probability distribution over the drift rates, equation 3.2. Note, second, that the effect of the boundaries is to reduce the number of trajectories that can reach a given ending point from the same initial condition compared to the case with no boundaries. However, since any of the remaining realizable trajectories (i.e., those that do not cross the boundaries) with the same initial and ending points lead to the same probability distribution of the drift rates, the shape of the boundaries does not affect the information available about the drifts at time t.
The result that the probability distribution over the drift rates does not depend on the temporal profile of the decision boundaries before the decision time is nevertheless surprising. Let us assume, for instance, that the decision boundaries drop sharply to very negative values at short times and then recover back to some finite value and that the final state of one integrator is positive, x1>0. Then one could argue that the a posteriori probability distribution over the drift rate μ1 of that integrator has to be skewed to very large negative values (i.e., negative drifts are more likely than positive ones) because the trajectory of the integrator first needed to reach very low values before ending at the final state x1>0. Then, in general, one would conclude that the probability distribution over the drift rates depends not only on the final state of the integrators, but also on the temporal shape of the boundaries. Equation 3.6 shows that the above reasoning is incorrect and that in fact the probability distribution over the drift μ1 has a larger probability mass at positive values. The paradox can be resolved by noticing that the traveled distances by the integrators and elapsed time are sufficient statistics for the drift rates μk (k = 1, 2) in the decision process, while the shape of the paths traveled provides no additional information about the drift rates. Therefore, the state of the integrators, along with the elapsed time, summarizes all information one needs to know to make a decision that involves any inequality over their drift rates.
Figure 3 shows the probability density function over the drift rate μ computed from numerical simulations of a single integrator that obeys equation 2.2 in the presence of a nonlinear boundary Θ(t), a case for which there are no generally valid analytical solutions. Sample trajectories with a broad distribution of drift rates were generated, and from the trajectories that ended at threshold at times within a short window t and t + Δt, the distribution of drift rates at decision time was computed. As the theory predicts, equation 3.8 should be valid in the presence of any threshold, and therefore the distribution of drift rates should be a gaussian with mean Θ(t)/t and variance σ2/t when the decision is made at time t regardless of the shape of the boundary. The simulations are in perfect agreement with the theory.
A special feature of the DDM considered in equations 2.1 is that all information is readily available in the final state of the integrators and the elapsed time, equation 3.6. Indeed, this property is violated in neuronal integrator models with leak terms (negative linear xks), as considered, for example, in Usher and McClelland (2001). It can be shown that in these models, the a posteriori probability density distribution over the drift rates depends not only on the final states of the integrators but also on the mean value of the trajectories. Furthermore, if expressed only as a function of the final state of the integrators (along with the initial state and elapsed time), the probability density over the drift rate depends on the temporal profile of the decision boundaries Θk(t). It is interesting to note, then, that brain areas like LIP and FEF show ramping activity that resembles perfect, leakless integration of evidence.
3.2. Decision Confidence.
Figure 4A shows decision confidence as a function of decision time for a fixed state of the losing integrator and as a function of the state of the losing integrator, x2, for fixed decision time. Constant bounds, Θ(t) = Θ, for the integrators are used in this example. As a function of decision time, confidence decays monotonically from one to one-half. In the limit of long t, equation 3.9 converges to . The monotonic decay is due to the fact that fast responses are likely to happen in trials in which large drift rates are used, which leads to better performance and, hence, increased confidence. As a function of the state of the losing integrator, decision confidence decreases from one at large and negative x2 and approaches one-half when x2 is close to threshold.
Experimentally, human verbal reports of confidence are close but typically lower than the actual performance shown by the subjects (Vickers, 1979); in general, the slope of the confidence versus difficulty curve is lower than that of the probability correct versus difficulty curve. One could argue that verbal reports provide biased estimates of the true internal confidence because of their discretized nature. A second, and more interesting, possibility is that subjects' belief does not fully correspond to their performance because the subjects have not completely learned the task's contingencies. In either case, confidence would not correspond to the probability of being correct. The last situation can be described with the DDM presented here if, for instance, the variance σ in equation 2.1 is only approximately known by the subject and is taken to be larger than the real one. For the problem corresponding to equation 3.16, probability correct and confidence will be correlated and have the same functional dependences on μ0 (which measures the difficulty of each trial), but the slope of the performance versus difficulty curve will be higher than that of the confidence-versus-difficulty curve.
3.3. Marginal Confidence.
Figure 4B shows that marginal confidence (computed using equations 3.20 to 3.22, for two independent and half-anticorrelated integrators and using equation 3.11 for the classical DDM with the same threshold and noise values in the three cases) is at a maximum for the whole range of decision times when the integrators are independent, nearly so when they are half-anticorrelated, and at a minimum for the classical DDM. Therefore, decision confidence in integrators with even strong correlations is close to that in independent integrators and higher than that in the classical DDM. The large difference in marginal confidence between the classical DDM and the half-anticorrelated or independent integrators is due to the fact that as decision time increases, the losing integrator in the latter two cases separates from its boundary (a reminder that the losing integrator has negative drift). Using the fact that confidence increases as the distance between the integrators increases, equation 3.9, it is now clear why marginal confidence becomes higher for the independent and half-anticorrelated cases than for the classical DDM.
3.4. Reaction Time Distributions.
The time evolution of the probability density function of finding the neuronal integrators in a given state can be used to gain additional insight into the decision process. Figure 5 shows three snapshots of the time evolution of the probability density distribution over for two independent (see Figure 5A) and two half-anticorrelated integrators (see Figure 5B). In both cases, I use μ1>μ2 ⩾ 0. The distribution broadens over time and moves toward the boundaries. When the distributions get close to the boundaries, their peaks remain approximately stationary. For the parameter values used, the distributions approach the boundary l1 more rapidly. Therefore, most of the trajectories will escape through the semi-line l1. However, many others will cross l2 first. It can be proved that one of the two boundaries is crossed at some finite time with probability one if the drift of at least one integrator is larger than zero. The distribution spreads circularly for two independent integrators, while for two half-anticorrelated integrators, the distribution spreads more along the nonprincipal diagonal.
The first passage times-state joint probability distribution, equation 3.18, can be used to describe the state of the losing integrator given that a decision has been made. Figure 6A shows a typical example of the joint probability distribution. The mean value of x2 (the white line) diverges monotonically from threshold as a function of the crossing time of integrator k = 1. This is because, as shown in Figure 5A, trajectories tend to disperse at long times. A different picture emerges when the two integrators are half-anticorrelated, equations 3.18 and A.8, as shown in the right panel. The mean value of x2 approaches threshold at short decision times, while it diverges from threshold at long decision times. The first rise is due to the presence of negative correlations in the inputs to the two integrators and can be understood as follows. If integrator k = 1 reaches its threshold at a late time, this delay is likely due to a negative input fluctuation to that integrator. Since the two integrators have anticorrelated inputs, the integrator k = 2 has likely experienced a positive input fluctuation, explaining why the mean value of x2 approaches the threshold at short decision times. At long decision times, the diffusion of trajectories due to noise dominates over the previous mechanism, and therefore the mean value of x2 diverges from threshold in that regime.
Figure 6B shows the probability distributions of first passage times for each integrator along with the condition that it crossed its threshold first. This first passage time distribution is computed by integrating equation 3.18 with the probability distribution of the losing integrator. The integrator with the higher drift crosses first the threshold with a probability larger than one-half (full lines). The distributions are positively skewed, similar to the reaction times distributions typically found in two-alternative forced-choice tasks (Ratcliff & Smith, 2004; Luce, 1986).
A decision diffusion model (DDM) consisting of two neuronal integrators receiving independent as well as correlated inputs has been studied. Decision confidence has been shown to depend on the difference between the traveled distances of the two integrators at decision time. Importantly, leakless integrators accumulate evidence without loss, since their final state and elapsed time fully specify the probability distribution over the signals, regardless of the stopping rule used to make decisions. Different behaviors are obtained from leak integrators, since the probability distribution over the drift rates depends not only on the final states of the integrators and elapsed time. These results can have important consequences about neuronal coding and integration of information in lateral intraparietal area (LIP) and frontal eye fields (FEF), where almost linear, perfect integration of evidence is observed.
Previous models have proposed that decision confidence is encoded in the difference between the state of two integrators that accumulate evidence in favor of two alternatives (Vickers, 1979; Juslin & Olsson, 1997; Vickers & Lee, 1998; Van Zandt, 2000), like the DDM studied here. However, it was not known how confidence was mathematically related to the difference of the states (balance of evidence) of the integrators. As equation 3.9 shows, decision confidence is a cumulative gaussian whose argument is proportional to the balance-of-evidence term. In addition, it explicitly shows time dependences that were not anticipated by these preliminary works. Pleskac and Busemeyer (2007) have considered a model in which confidence is computed after the decision has been made. However, an interesting alternative, as proposed by the model presented here, is that that when a decision is made, there is immediate access to decision confidence and the evaluation of confidence does not require additional deliberation. Ratcliff and Starns (2009) recently presented a model about how verbal reports of decision confidence can be generated by human subjects. The model requires multiple additional integrators (one for each confidence rating—for example, high, medium, or low), whose role is to determine which rating will be reported. The DDM model that I have described here makes the interesting prediction that decision confidence is represented in the brain as a continuous value: if verbal reports about confidence are not forced, subjects might use confidence only implicitly without the necessity of discretizing it.
Biophysically inspired neuronal network models for decision making have been studied in Wang (2002) and Beck et al. (2008). These attractor models show almost linear ramping activity at low firing rates, like the DDM studied here, but they display nonlinearities that become evident when the firing rate approaches the decision boundary. Consistent with these models, Huk and Shadlen (2005) and Wong, Huk, Shadlen, and Wang (2007) have shown that brief motion pulses (100 ms) injected at different times from the onset of a dynamic random dot display affect the firing rate of LIP neurons in a time-dependent manner, which earlier pulses producing larger changes in firing rate than later pulses. As Huk and Shadlen (2005) pointed out, this result is also consistent with a perfect integrator model of inputs with time-decaying boundaries, like the DDM considered in this letter. Further experiments are required to test which is the correct model. These experiments would likely involve injecting motion pulses at different times conditioned to the same firing rate of the LIP neurons for all times. In this hypothetical experiment, the attractor model predicts time-shift invariance of the induced changes in firing rate, while the DDM with time-decaying boundaries predicts its violation. Further theoretical work is also required to connect realistic neuronal attractor networks with simplified integrator models and to determine to what extent nonlinearities that arise near the decision boundaries can be well approximated by hard decision boundaries (see Roxin & Ledberg, 2008).
In summary, I have provided analytical tools to describe decision confidence and uncertainty in decision making using the DDM. The analytical expressions for decision confidence in partially correlated neuronal integrators found in this letter might allow estimating confidence from multirecording neurophysiological data in decision areas like LIP and FEF (Hanes & Schall, 1996; Shadlen & Newsome, 1996; Platt & Glimcher, 1999) and compare it with behaviorally measured confidence values. The results can be extended to multiple integrators and to more general forms of noise, therefore opening a window for studying how decision uncertainty is represented in neuronal populations in the brain.
Appendix: Solution for a DDM with Half-Anticorrelated Integrators
This appendix provides details for the derivation of the solution of the FPE, equation 3.17, for the case of half-anticorrelated integrators. It also gives the derivation for the case in which the integrators are independent.
I thank Alex Pouget and Jan Drugowitsch for insightful discussions. This work has been partially supported by the Swartz Foundation.