## Abstract

Diffusion models have become essential for describing the performance and statistics of reaction times in human decision making. Despite their success, it is not known how to evaluate decision confidence from them. I introduce a broader class of models consisting of two partially correlated neuronal integrators with arbitrarily time-varying decision boundaries that allow a natural description of confidence. The dependence of decision confidence on the state of the losing integrator, decision time, time-varying boundaries, and correlations is analytically described. The marginal confidence is computed for the half-anticorrelated case using the exact solution of the diffusion process with constant boundaries and compared to that of the independent and completely anticorrelated cases.

## 1. Introduction

In most interesting situations, detection of a signal involves discriminating a meaningful from a random energy fluctuation. A standard procedure employed to discriminate between these two alternatives is to integrate over time such fluctuations and decide that there is a signal embedded in the noise if the accumulated sum reaches a predetermined threshold and that there is no signal otherwise. Although this is just the way some photosensors are used to detect dim stars, humans seem to employ a remarkably similar strategy to make decisions. Human performance and reaction time statistics in two-alternative forced-choice tasks are accurately fit by models consisting of a diffusion process of an underlying variable to decision boundaries (Ratcliff, 1978; Ratcliff & Smith, 2004; Luce, 1986; Usher & McClelland, 2001; Bogacz et al., 2006; Mazurek, Roitman, Ditterich, & Shadlen, 2003; Ratcliff, Hasegawa, Hasegawa, Smith, & Segraves, 2007; Wang, 2002; Beck et al., 2008). Moreover, diffusion to bound processes seem to be neurally hardwired in several areas of the brain, where the neurons' firing rate reflects integration of noisy information for alternatives and determines choices and decision times when it reaches a fixed threshold (Hanes & Schall, 1996; Shadlen & Newsome, 1996; Platt & Glimcher, 1999; Ratcliff et al., 2007; see Schall, 2001, for a review).

Upon a decision, a crucial variable is the degree of confidence about the choice. In detecting a new particle or star, the estimated confidence will determine if we can believe in the result. Humans and animals are able to reliably estimate confidence or uncertainty about their decisions and use it to change or abort behaviors (Vickers, 1979; Juslin & Olsson, 1997; Vickers & Lee, 1998; Landy, Goutcher, Trommershuser, & Mamassian, 2007; Whiteley & Sahani, 2008; Kepecs, Uchida, Zariwala, & Mainen, 2008). Human subjects can be asked directly to report their confidence, but it is also possible to use choice behaviors in humans and animals to estimate their decision confidence. For instance, a subject can be presented with two choices: one that involves high but uncertain reward and another with low but certain reward. The confidence that the subject has about the uncertain reward at any trial will determine which option is taken, and therefore decision confidence can be estimated from its behavior at any trial. Despite the importance of decision confidence, very little is known about how it is encoded in the nervous system (but see Kepecs et al., 2008; Beck et al., 2008). Furthermore, the dependence of confidence on essential parameters of the decision process like reaction time or distance to boundary of the losing integrator has not been established. Knowing such dependencies on simple models would allow a quantitative comparison between theory and both behavioral and neurophysiological data.

This letter describes analytically decision confidence for a decision diffusion model (DDM) consisting of two neuronal integrators receiving partially correlated inputs. The main results are as follows: (1) decision confidence depends on the difference between the traveled distances by the two integrators in a general way, regardless of the correlation in the inputs and on the time dependence of the decision thresholds; (2) accumulation of evidence over time occurs without any leak or loss, that is, the final state of the integrators and elapsed time fully specify the a posteriori probability distribution over the parameters on which the decision is based; (3) marginal confidence is computed for the independent, half-anticorrelated, and completely anticorrelated cases as a function of the decision time, being at a maximum for independent integrators; and (4) the distributions of reaction times for the above cases are characterized. Models with several integrators similar to the one proposed here have been introduced before to describe qualitatively decision confidence (Vickers, 1979), but analytical expressions for confidence are in general not known for these simple models (Juslin & Olsson, 1997; Vickers & Lee, 1998; Van Zandt, 2000; Pleskac & Busemeyer, 2007). It is worth mentioning that the DDM considered here reduces to the classical DDM (Ratcliff & Smith, 2004; Luce, 1986) when the two integrators receive completely anticorrelated inputs, for which decision confidence as a function of the decision time is also completely characterized for arbitrary time-dependent boundaries.

## 2. Model

*x*

_{1}and

*x*

_{2}receiving independent and cross-correlated inputs, where η

_{i}(

*t*),

*i*= 1, 2,

*c*are independent gaussian white noise processes with zero mean (〈η

_{i}(

*t*)〉 = 0) and unit variance (〈η

_{i}(

*t*)η

_{i}(

*t*′)〉 = δ(

*t*−

*t*′), where δ(

*t*−

*t*′) is the Dirac's delta function).

^{1}The variables

*x*

_{1}and

*x*

_{2}can correspond to averaged firing rates across a population of neurons with similar tuning, or to population-averaged voltages (Mazurek et al., 2003). The model captures the approximately linear ramping activity observed in neurons in lateral intraparietal area (LIP) and frontal eye fields (FEF) in monkeys performing simple decision tasks (Hanes & Schall, 1996; Shadlen & Newsome, 1996). The

*k*th neuronal integrator (

*k*= 1, 2) drifts with rate μ

_{k}and accumulates input white noises with total deviation σ. The correlation coefficient, ρ ∈ [0, 1], determines the degree of correlation between the two integrators, being independent if ρ = 0 and completely correlated if ρ = 1. The correlation index, ν ∈ {−1, 1}, determines the correlation sign between them, being positively correlated if ν = 1 and anticorrelated if ν = −1. The variables

*x*

_{1}and

*x*

_{2}diffuse from the initial values

*a*and

*b*until one of them reaches its threshold, Θ

_{1}(

*t*) and Θ

_{2}(

*t*), respectively, which can depend on time.

A two-alternative forced-choice task is modeled as follows. A decision for choice *k* (*k* = 1, 2) is made whenever the variable *x _{k}* reaches its threshold Θ

_{k}(

*t*) first. It can be assumed that the decision is correct if the boundary first reached corresponds to that of the integrator with the highest drift. This is a general situation that, for instance, could correspond to a task in which the goal is to detect the brighter of two lights. It also arises naturally in experiments in which the direction of motion of a random-dot motion display has to be determined during short presentations (Shadlen & Newsome, 1996; Mazurek et al., 2003). Here, the direction of motion can be modeled by the sign of the drift rates of the two integrators (i.e., μ = μ

_{1}= −μ

_{2}>0 indicates rightward motion, while μ = μ

_{1}= −μ

_{2}< 0 indicates leftward motion). The difficulty of this task is controlled by the motion coherence, that is, the percentage of dots that move in the same direction as opposed to random directions, and it can be modeled by the magnitude of the drift rates, |μ| = |μ

_{1}| = |μ

_{2}|. It will be assumed that the variance of the noise and correlation coefficient remains constant across all stimulation conditions. I assume that when a decision has been made, the state of the losing integrator can be read out. Figure 1 illustrates the model for the independent (ρ = 0 in Figure 1A) and half-anticorrelated (ρ = 1/2, ν = −1, in Figure 1C) cases with time-varying thresholds.

_{1}= −μ

_{2}, and the thresholds are identical and time independent, Θ

_{1}(

*t*) = Θ

_{2}(

*t*) = Θ. In that case, only motion parallel to the diagonal

*x*

_{1}= −

*x*

_{2}is possible, and therefore each integrator can be viewed as the antineuron of the other. Equations 2.1 reduce in this case to the system where

*x*=

*x*

_{1}= −

*x*

_{2}and the decision boundaries lie at

*x*= ±Θ (as before, η(

*t*) is a gaussian white noise process). Although very relevant to fit behavioral data, the classical DDM assumes perfect anticorrelation between the integrators, and such degree of correlation seems unrealistic in the brain (Zohary, Shadlen, & Newsome, 1994).

When ρ = 0 in equations 2.1, the two integrators become independent, and then the system becomes a race model (see, e.g., Vickers, 1979; Luce, 1986). Therefore, the model defined in equations 2.1 covers a wide spectrum of classical models of decision making, ranging from the classical DDM for ρ = 1 and ν = −1, to the race model for ρ = 0, as shown in Figure 2. However, the model considered here does not include interactions or connections between the neuronal integrators, as in Usher & McClelland (2001). A discussion about these models has been recently presented in Bogacz (2007).

## 3. Results

### 3.1. Independence of the Probability Distributions over the Drift Rates on Decision Boundaries.

A decision about which drift is higher (μ_{1}>μ_{2} or μ_{2}>μ_{1}?) in the system defined in equations 2.1 will depend on the drift estimates at decision time *t* given the observed trajectories of the integrators up to that time, denoted and , with initial states *x*_{1}(0) = *a* and *x*_{2}(0) = *b* at *t* = 0. The critical quantity is then the probability density of having the drifts μ_{1} and μ_{2} given the trajectories and up to time *t*, . Instead of working directly with this density, it is more convenient to introduce the new variables *u* = *x*_{1} + *x*_{2} and *v* = *x*_{1} − *x*_{2} and work with the probability density of having drifts μ_{u} = μ_{1} + μ_{2} and μ_{v} = μ_{1} − μ_{2} given the trajectories and , with initial conditions *u*_{0} = *a* + *b* and *v*_{0} = *a* − *b*. In the new variables, equations 2.1 transform into two independent Wiener processes, with means μ_{u} and μ_{v}, and variances σ^{2}_{u} = 2σ^{2}(1 + ρν) and σ^{2}_{v} = 2σ^{2}(1 − ρν), making the calculations easier.

*n*steps of size δ

*t*, such that

*t*=

*n*δ

*t*, and defining

*u*=

_{i}*u*(

*i*δ

*t*) (

*v*=

_{i}*v*(

*i*δ

*t*)),

*i*= 1, …,

*n*, then the probability density of having a variable step δ

*u*=

_{i}*u*−

_{i}*u*

_{i−1}(δ

*v*=

_{i}*v*−

_{i}*v*

_{i−1}) at time

*t*=

*i*δ

*t*follows a gaussian distribution with mean μ

_{u}δ

*t*(μ

_{v}δ

*t*) and variance σ

^{2}

_{u}δ

*t*(σ

^{2}

_{v}δ

*t*). Since each increment is independent of the others, the probability density of the trajectory is If

*p*(μ

_{u}, μ

_{v}) is the distribution of the drifts based on prior knowledge, Bayes' theorem states that , where the proportionality is in relation to the drift rates. If the prior distribution is uniform,

*p*(μ

_{u}, μ

_{v}) =

*p*

_{0}, Bayes' theorem and equation 3.1 give after normalization that the

*free*(i.e., without boundaries) distribution over the drift rates is where (

*u*,

*v*) is the ending point of the trajectory at time

*t*. Then the a posteriori probability distribution over the drift in the variable

*u*(

*v*) is a gaussian distribution with mean (

*u*−

*u*

_{0})/

*t*([

*v*−

*v*

_{0}]/

*t*) and variance σ

^{2}

_{u}/

*t*(σ

^{2}

_{v}/

*t*). Hence, the mean drift is just the slope of the trajectory, computed as the ratio between the distance from initial to ending points, and elapsed time. The variance of the probability distribution decays as a function of time, so a longer time results in a narrower distribution and therefore in a better estimation of the drifts. Importantly, the identity between and

*p*(μ

_{free}_{u}, μ

_{v}∣

*u*−

*u*

_{0},

*v*−

*v*

_{0},

*t*) shows that all the information in relation to the drift rates available in the trajectories is preserved in the distances traveled by the integrators (i.e., their final state) and elapsed time. This is just the description of the fact that the traveled distances and elapsed time are sufficient statistics for the distribution of drift rates.

Now I consider the case in which there are boundaries Θ_{k}(*t*) (*k* = 1, 2). In this case, only trajectories that do not cross the boundaries at any time before *t* (i.e., such that *u* + *v* < 2Θ_{1}(τ) and *u* − *v* < 2Θ_{2}(τ) for all τ < *t*) are realizable, while the trajectories that cross the boundaries before time *t* are absorbed. In what follows, I show the central result that the probability distribution of the drift rates given the traveled distances by the integrators and the elapsed time when boundaries are present is identical to the probability distribution of the drift rates when there are not boundaries, equation 3.2.

Two steps will be followed in the proof. First, it is shown that the probability density of having the state (*u*, *v*) at time *t* when arbitrary boundaries are present can be factorized in two terms: one that depends on the drift rates but not on the boundaries and another that depends on the boundaries but not on the drift rates. Second, it is shown that this factorization and Bayes' theorem naturally lead to equation 3.2 for uniform a priori distributions over the drift rates. This proof is similar to that derived by Beck et al. (2008) for Poisson-like distributions.

*u*,

*v*) of the integrators at time

*t*given initial conditions (

*u*

_{0},

*v*

_{0}) and drift rates μ

_{u}and μ

_{v}can be expressed as the integral of the probability of having the trajectory , equation 3.1, over all possible trajectories with initial point (

*u*

_{0},

*v*

_{0}) and final point (

*u*,

*v*), which do not cross the boundaries at any time before

*t*, where and (remember that a trajectory is defined in terms of the differential increments (δ

*u*, δ

_{i}*v*) for

_{i}*i*= 1, …,

*n*in time steps δ

*t*=

*t*/

*n*), and the region Ω is defined as the set of trajectories that satisfies

*u*

_{0}+

*v*

_{0}+ ∑

^{j}

_{i=1}(δ

*u*+ δ

_{i}*v*) < 2Θ

_{i}_{1}(

*j*δ

*t*) and

*u*

_{0}−

*v*

_{0}+ ∑

^{j}

_{i=1}(δ

*u*− δ

_{i}*v*) < 2Θ

_{i}_{2}(

*j*δ

*t*) for

*j*= 1, …,

*n*(i.e., they do not cross the boundaries before time

*t*), along with the constraint that ∑

^{n}

_{i=1}δ

*u*=

_{i}*u*−

*u*

_{0}and ∑

^{n}

_{i=1}δ

*v*=

_{i}*v*−

*v*

_{0}(i.e., the initial and final points are (

*u*

_{0},

*v*

_{0}) and (

*u*,

*v*), respectively). By expanding the squared terms in the exponents of equation 3.3 and using that ∑

^{n}

_{i=1}δ

*t*=

*t*, ∑

^{n}

_{i=1}δ

*u*=

_{i}*u*−

*u*

_{0}, and ∑

^{n}

_{i=1}δ

*v*=

_{i}*v*−

*v*

_{0}, it is possible to extract out from the integral the terms that depend on the drift rates as where Note that

*R*(

*u*,

*v*) depends on the boundaries but not on the drift rates.

*p*(μ

_{bound}_{u}, μ

_{v}∣

*u*,

*v*,

*t*), and this is proportional to equation 3.4 through Bayes' theorem, then only the dependencies of equation 3.4 on the drift rates are explicitly considered. This means, interestingly, that to determine the distribution over the drift rates, it is not required to compute explicitly the complicated integral in equation 3.5, a task achievable only in particular cases. Therefore, after using Bayes' theorem

*p*(μ

_{bound}_{u}, μ

_{v}∣

*u*,

*v*,

*t*) ∝

*p*(

_{bound}*u*,

*v*,

*t*∣ μ

_{u}, μ

_{v}) (proportionality is in relation to the drift rates; here a uniform a priori distribution over the drifts rates is assumed, but a similar result holds for any arbitrary a priori distribution) and normalizing the expression, it is found that regardless of the boundary conditions, which is identical to equation 3.2. This proves that

*p*(μ

_{bound}_{u}, μ

_{v}∣

*u*−

*u*

_{0},

*v*−

*v*

_{0},

*t*) =

*p*(μ

_{free}_{u}, μ

_{v}∣

*u*−

*u*

_{0},

*v*−

*v*

_{0},

*t*).

Finally, one needs to show that all information contained in the trajectory is preserved in the final state of the integrators and elapsed time in the presence of boundaries. Since for any trajectory that does not cross the bound at any time before *t*, , one has , then using equations 3.2 and 3.6, . Therefore, the integrators accumulate evidence with any leak or loss. This interestingly means that, as in the case without boundaries, the shape of the trajectory does not need to be remembered, since the final state of the integrators and elapsed time contain all necessary information in order to perform inference about the drift rates. More formally, the distances traveled by the integrators along with the elapsed time are sufficient statistics for the drift rates.

The equivalence between probability distributions with and without boundaries can be understood intuitively as follows. Note, first, that any trajectory that does not cross the boundaries at any time before *t* and has the same initial and final points leads to the same probability distribution over the drift rates, equation 3.2. Note, second, that the effect of the boundaries is to reduce the number of trajectories that can reach a given ending point from the same initial condition compared to the case with no boundaries. However, since any of the remaining realizable trajectories (i.e., those that do not cross the boundaries) with the same initial and ending points lead to the same probability distribution of the drift rates, the shape of the boundaries does not affect the information available about the drifts at time *t*.

The result that the probability distribution over the drift rates does not depend on the temporal profile of the decision boundaries before the decision time is nevertheless surprising. Let us assume, for instance, that the decision boundaries drop sharply to very negative values at short times and then recover back to some finite value and that the final state of one integrator is positive, *x*_{1}>0. Then one could argue that the a posteriori probability distribution over the drift rate μ_{1} of that integrator has to be skewed to very large negative values (i.e., negative drifts are more likely than positive ones) because the trajectory of the integrator first needed to reach very low values before ending at the final state *x*_{1}>0. Then, in general, one would conclude that the probability distribution over the drift rates depends not only on the final state of the integrators, but also on the temporal shape of the boundaries. Equation 3.6 shows that the above reasoning is incorrect and that in fact the probability distribution over the drift μ_{1} has a larger probability mass at positive values. The paradox can be resolved by noticing that the traveled distances by the integrators and elapsed time are sufficient statistics for the drift rates μ_{k} (*k* = 1, 2) in the decision process, while the shape of the paths traveled provides no additional information about the drift rates. Therefore, the state of the integrators, along with the elapsed time, summarizes all information one needs to know to make a decision that involves any inequality over their drift rates.

*t*) =

*A*+

*Bt*. Then the probability distribution at time

*t*over the state

*x*of the integrator with starting point

*x*= 0 and drift rate μ follows, with

*D*= 2

*A*(

*B*− μ)/σ

^{2}. (It is easy to show that

*p*(

*x*,

*t*∣ μ) vanishes at the time-dependent threshold, and since it is composed of a sum of two free solutions of the associated Fokker-Planck equation, it is the solution with a time-dependent absorbing boundary Θ(

*t*); Cox & Miller, 1965.) Finally, using Bayes' theorem with uniform a priori distribution over the drift rate,

*p*(μ ∣

*x*,

*t*) ∝

*p*(

*x*,

*t*∣ μ), one finds that that is, the a posteriori probability distribution over the drift rate μ is a gaussian with mean

*x*/

*t*and variance σ

^{2}/

*t*, and it does not depend on the parameters

*A*and

*B*that define the time-dependent boundary. (Note that equation 3.8 is a particular case of equation 3.6 when restricted to the dynamics of a single integrator.) The same result holds when there are two side time-independent boundaries, a case for which analytical expressions are also known (Cox & Miller, 1965), and in fact, as it has been shown before, equation 3.8 is valid for any decision boundaries.

Figure 3 shows the probability density function over the drift rate μ computed from numerical simulations of a single integrator that obeys equation 2.2 in the presence of a nonlinear boundary Θ(*t*), a case for which there are no generally valid analytical solutions. Sample trajectories with a broad distribution of drift rates were generated, and from the trajectories that ended at threshold at times within a short window *t* and *t* + Δ*t*, the distribution of drift rates at decision time was computed. As the theory predicts, equation 3.8 should be valid in the presence of any threshold, and therefore the distribution of drift rates should be a gaussian with mean Θ(*t*)/*t* and variance σ^{2}/*t* when the decision is made at time *t* regardless of the shape of the boundary. The simulations are in perfect agreement with the theory.

A special feature of the DDM considered in equations 2.1 is that all information is readily available in the final state of the integrators and the elapsed time, equation 3.6. Indeed, this property is violated in neuronal integrator models with leak terms (negative linear *x _{k}*s), as considered, for example, in Usher and McClelland (2001). It can be shown that in these models, the a posteriori probability density distribution over the drift rates depends not only on the final states of the integrators but also on the mean value of the trajectories. Furthermore, if expressed only as a function of the final state of the integrators (along with the initial state and elapsed time), the probability density over the drift rate depends on the temporal profile of the decision boundaries Θ

_{k}(

*t*). It is interesting to note, then, that brain areas like LIP and FEF show ramping activity that resembles perfect, leakless integration of evidence.

### 3.2. Decision Confidence.

*t*. Given the final state of the integrators, the join probability density of the drift rates is given by equation 3.6. From this probability density, it is easy to compute whether the probability of μ

_{1}>μ

_{2}is larger or lower than the probability of μ

_{2}>μ

_{1}, and the choice μ

_{1}>μ

_{2}or μ

_{2}>μ

_{1}will be made accordingly. In order to associate uniquely one decision boundary with one choice, the initial states of the integrators and the bounds are taken to be identical for the two integrators,

*a*=

*b*and Θ(

*t*) = Θ

_{1}(

*t*) = Θ

_{2}(

*t*). Therefore, if the boundary of integrator

*k*= 1 is reached first, then at decision time,

*x*

_{1}(

*t*) = Θ(

*t*)>

*x*

_{2}(

*t*). This implies that the case μ

_{1}>μ

_{2}has higher probability, and hence it is chosen (and, conversely, if integrator

*k*= 2 reaches its bound first). Without loss of generality, in the following, integrator

*k*= 1 is assumed to reach its boundary first,

*x*

_{1}= Θ(

*t*), and then the choice μ

_{1}>μ

_{2}has been made. Decision confidence can then be written using equation 3.6 as where Δ(

*x*

_{2}) = Θ(

*t*) −

*x*

_{2}and

*x*

_{2}is the final state of integrator 2 at decision time

*t*. Equation 3.9 is a cumulative normal gaussian with the argument being proportional to the ratio between the difference in distance traveled by the two integrators and the squared root of the decision time. The same expression holds if the prior distribution of the drifts is

*p*(μ

_{1}, μ

_{2}) =

*p*(μ

_{1})δ(μ

_{1}− μ

_{2}), where

*p*(μ

_{1}) =

*p*

_{0}(

*p*

_{0}is a constant), which states that μ

_{1}is distributed uniformly along with the constraint μ

_{1}= −μ

_{2}. Decision confidence depends, then, on the difference between the states of the two integrators at decision time. The correlation between the integrators affect only σ

_{v}, and hence the slope of the confidence function, but it does not change the functional dependence on the states of the integrators and elapsed time.

Figure 4A shows decision confidence as a function of decision time for a fixed state of the losing integrator and as a function of the state of the losing integrator, *x*_{2}, for fixed decision time. Constant bounds, Θ(*t*) = Θ, for the integrators are used in this example. As a function of decision time, confidence decays monotonically from one to one-half. In the limit of long *t*, equation 3.9 converges to . The monotonic decay is due to the fact that fast responses are likely to happen in trials in which large drift rates are used, which leads to better performance and, hence, increased confidence. As a function of the state of the losing integrator, decision confidence decreases from one at large and negative *x*_{2} and approaches one-half when *x*_{2} is close to threshold.

*p*(μ

_{1}, μ

_{2}) =

*p*(μ

_{1})δ(μ

_{1}− μ

_{2}) with

*p*(μ

_{1}) =

*p*

_{0}(

*p*

_{0}is a constant), which ensures the constraint μ = μ

_{1}= −μ

_{2}; in addition, constant bounds Θ(

*t*) = Θ are used. Then the state of one integrator is determined by the state of the remaining integrator, since when integrator

*k*= 1 reaches its threshold,

*x*

_{1}= Θ, integrator

*k*= 2 is at

*x*

_{2}= −Θ. At decision time, the probability distribution over the drift rate μ can be written using equation 3.6 as and from this equation or from the general expression in equation 3.9 with σ

_{v}= 2σ, one gets that decision confidence is which depends on only decision time (the dependence on the threshold and noise variance is not made explicit). For long decision times, decision confidence converges to . Experimental measurements of decision confidence have been typically fit by

*a*/

*t*+

*b*functions (Vickers, 1979), which are mathematically different from the dependences reported here. However, the two expressions are not very different numerically one from each other when they are used in a limited range of decision times. It should be noted that distinguishing between the two expressions requires large amounts of data for very long reaction times, which are not typically available. Nevertheless, it would be desirable to design experiments that allow the estimation of confidence for a wide range of decision times to test the analytical expressions found in this letter.

*t*). In this case, the probability distribution over the drift rate μ at decision time is written using equation 3.6 as and decision confidence then becomes

_{1}= −μ

_{2}= ±μ

_{0}with probability one-half, then following similar steps as in equations 3.6 and 3.9, it can be shown that At decision time, decision confidence depends on the difference between the traveled distances by the integrators, Δ(

*x*

_{2}) = Θ(

*t*) −

*x*

_{2}, but not explicitly on decision time. As in equations 3.6 and 3.9, confidence is independent of the boundaries used to generate decisions. For the classical DDM with time-dependent boundaries, Θ(

*t*), decision confidence becomes which depends on only decision time through the boundary time dependence. It is interesting to note that for this case, decision confidence equals the expression for the probability of reaching first the correct boundary (Cox & Miller, 1965). Therefore, confidence and performance are identical here. Although this result can be shown to be rather general, it is not true for all boundary conditions.

Experimentally, human verbal reports of confidence are close but typically lower than the actual performance shown by the subjects (Vickers, 1979); in general, the slope of the confidence versus difficulty curve is lower than that of the probability correct versus difficulty curve. One could argue that verbal reports provide biased estimates of the true internal confidence because of their discretized nature. A second, and more interesting, possibility is that subjects' belief does not fully correspond to their performance because the subjects have not completely learned the task's contingencies. In either case, confidence would not correspond to the probability of being correct. The last situation can be described with the DDM presented here if, for instance, the variance σ in equation 2.1 is only approximately known by the subject and is taken to be larger than the real one. For the problem corresponding to equation 3.16, probability correct and confidence will be correlated and have the same functional dependences on μ_{0} (which measures the difficulty of each trial), but the slope of the performance versus difficulty curve will be higher than that of the confidence-versus-difficulty curve.

### 3.3. Marginal Confidence.

*x*(

_{k}*k*= 1, 2) at decision time, which satisfies a Fokker-Planck equation (FPE) with absorbing boundaries. The FPE associated with equations 2.1 is where is the joint probability density of finding the integrators in state at time

*t*given that the initial condition at time

*t*= 0 was . The FPE has to be solved along with the condition that at the boundaries, defined as the semi-lines

*x*

_{1}= Θ(

*t*) for all

*x*

_{2}⩽ Θ(

*t*) (

*l*

_{1}) and

*x*

_{2}= Θ(

*t*) for all

*x*

_{1}⩽ Θ(

*t*) (

*l*

_{2}),

*P*has to be zero at all times. This condition defines the semi-lines

*l*

_{1}and

*l*

_{2}as absorbing boundaries. Solutions to the FPE, equation 3.17, are known for the independent and completely anticorrelated (classical DDM) cases with time-independent boundaries (Cox & Miller, 1965). In the appendix, I provide details for the derivation of the solution for two half-anticorrelated integrators with time-independent boundaries. The above three solutions allow a direct comparison of marginal confidence between different cases. In the following, Θ = 0 and

*a*< 0. Then |

*a*| measures the initial distance to threshold.

*k*reaches its threshold at time

*t*and integrator

*j*≠

*k*is at state

*x*(

_{j}*x*⩽ 0) at that time, along with the condition that no integrator has crossed its threshold before, given the initial condition . This quantity, denoted , is written as which can be obtained from the probability flux vector expression associated with the FPE (see equation 3.17; Risken, 1989). The dependence on the drift rates has been made explicit for later convenience. Its integral over

_{j}*x*is the conditional first passage times distribution for integrator

_{j}*k*. For instance, using equation A.9 in the appendix, for two independent integrators equation 3.18 takes the expression and similarly for .

*t*when the state of the losing integrator and the values of the drift rates are unknown or, more formally, where decision confidence

*p*(μ

_{1}>μ

_{2}∣

*x*

_{2},

*t*) is given in equation 3.9 and is the probability distribution of

*x*

_{2}at time

*t*given that the first integrator has reached threshold at time

*t*averaged over the distribution of drift rates. The latter is defined as , where

*C*(

*t*) is a normalization constant; , which includes the average over the distribution of drift rates; and is the a priori distribution over the drift rates. One could also define marginal confidence for a fixed set of values of drift rates, as they can be controlled experimentally, but here I focus on the averaged quantity. For two half-anticorrelated integrators, using a uniform prior along with the constraint that μ

_{1}= −μ

_{2}, and equations A.8 and 3.16, it is found that the probability density of finding the second integrator at state

*x*

_{2}given that the first integrator has reached its boundary at time

*t*takes the expression where the proportionality is in relation to the variable

*x*

_{2}. For two independent integrators, using equations A.9 and 3.17, with the constraint that μ

_{1}= −μ

_{2}, it is obtained

Figure 4B shows that marginal confidence (computed using equations 3.20 to 3.22, for two independent and half-anticorrelated integrators and using equation 3.11 for the classical DDM with the same threshold and noise values in the three cases) is at a maximum for the whole range of decision times when the integrators are independent, nearly so when they are half-anticorrelated, and at a minimum for the classical DDM. Therefore, decision confidence in integrators with even strong correlations is close to that in independent integrators and higher than that in the classical DDM. The large difference in marginal confidence between the classical DDM and the half-anticorrelated or independent integrators is due to the fact that as decision time increases, the losing integrator in the latter two cases separates from its boundary (a reminder that the losing integrator has negative drift). Using the fact that confidence increases as the distance between the integrators increases, equation 3.9, it is now clear why marginal confidence becomes higher for the independent and half-anticorrelated cases than for the classical DDM.

### 3.4. Reaction Time Distributions.

The time evolution of the probability density function of finding the neuronal integrators in a given state can be used to gain additional insight into the decision process. Figure 5 shows three snapshots of the time evolution of the probability density distribution over for two independent (see Figure 5A) and two half-anticorrelated integrators (see Figure 5B). In both cases, I use μ_{1}>μ_{2} ⩾ 0. The distribution broadens over time and moves toward the boundaries. When the distributions get close to the boundaries, their peaks remain approximately stationary. For the parameter values used, the distributions approach the boundary *l*_{1} more rapidly. Therefore, most of the trajectories will escape through the semi-line *l*_{1}. However, many others will cross *l*_{2} first. It can be proved that one of the two boundaries is crossed at some finite time with probability one if the drift of at least one integrator is larger than zero. The distribution spreads circularly for two independent integrators, while for two half-anticorrelated integrators, the distribution spreads more along the nonprincipal diagonal.

The first passage times-state joint probability distribution, equation 3.18, can be used to describe the state of the losing integrator given that a decision has been made. Figure 6A shows a typical example of the joint probability distribution. The mean value of *x*_{2} (the white line) diverges monotonically from threshold as a function of the crossing time of integrator *k* = 1. This is because, as shown in Figure 5A, trajectories tend to disperse at long times. A different picture emerges when the two integrators are half-anticorrelated, equations 3.18 and A.8, as shown in the right panel. The mean value of *x*_{2} approaches threshold at short decision times, while it diverges from threshold at long decision times. The first rise is due to the presence of negative correlations in the inputs to the two integrators and can be understood as follows. If integrator *k* = 1 reaches its threshold at a late time, this delay is likely due to a negative input fluctuation to that integrator. Since the two integrators have anticorrelated inputs, the integrator *k* = 2 has likely experienced a positive input fluctuation, explaining why the mean value of *x*_{2} approaches the threshold at short decision times. At long decision times, the diffusion of trajectories due to noise dominates over the previous mechanism, and therefore the mean value of *x*_{2} diverges from threshold in that regime.

Figure 6B shows the probability distributions of first passage times for each integrator along with the condition that it crossed its threshold first. This first passage time distribution is computed by integrating equation 3.18 with the probability distribution of the losing integrator. The integrator with the higher drift crosses first the threshold with a probability larger than one-half (full lines). The distributions are positively skewed, similar to the reaction times distributions typically found in two-alternative forced-choice tasks (Ratcliff & Smith, 2004; Luce, 1986).

## 4. Discussion

A decision diffusion model (DDM) consisting of two neuronal integrators receiving independent as well as correlated inputs has been studied. Decision confidence has been shown to depend on the difference between the traveled distances of the two integrators at decision time. Importantly, leakless integrators accumulate evidence without loss, since their final state and elapsed time fully specify the probability distribution over the signals, regardless of the stopping rule used to make decisions. Different behaviors are obtained from leak integrators, since the probability distribution over the drift rates depends not only on the final states of the integrators and elapsed time. These results can have important consequences about neuronal coding and integration of information in lateral intraparietal area (LIP) and frontal eye fields (FEF), where almost linear, perfect integration of evidence is observed.

Previous models have proposed that decision confidence is encoded in the difference between the state of two integrators that accumulate evidence in favor of two alternatives (Vickers, 1979; Juslin & Olsson, 1997; Vickers & Lee, 1998; Van Zandt, 2000), like the DDM studied here. However, it was not known how confidence was mathematically related to the difference of the states (balance of evidence) of the integrators. As equation 3.9 shows, decision confidence is a cumulative gaussian whose argument is proportional to the balance-of-evidence term. In addition, it explicitly shows time dependences that were not anticipated by these preliminary works. Pleskac and Busemeyer (2007) have considered a model in which confidence is computed after the decision has been made. However, an interesting alternative, as proposed by the model presented here, is that that when a decision is made, there is immediate access to decision confidence and the evaluation of confidence does not require additional deliberation. Ratcliff and Starns (2009) recently presented a model about how verbal reports of decision confidence can be generated by human subjects. The model requires multiple additional integrators (one for each confidence rating—for example, high, medium, or low), whose role is to determine which rating will be reported. The DDM model that I have described here makes the interesting prediction that decision confidence is represented in the brain as a continuous value: if verbal reports about confidence are not forced, subjects might use confidence only implicitly without the necessity of discretizing it.

Biophysically inspired neuronal network models for decision making have been studied in Wang (2002) and Beck et al. (2008). These attractor models show almost linear ramping activity at low firing rates, like the DDM studied here, but they display nonlinearities that become evident when the firing rate approaches the decision boundary. Consistent with these models, Huk and Shadlen (2005) and Wong, Huk, Shadlen, and Wang (2007) have shown that brief motion pulses (100 ms) injected at different times from the onset of a dynamic random dot display affect the firing rate of LIP neurons in a time-dependent manner, which earlier pulses producing larger changes in firing rate than later pulses. As Huk and Shadlen (2005) pointed out, this result is also consistent with a perfect integrator model of inputs with time-decaying boundaries, like the DDM considered in this letter. Further experiments are required to test which is the correct model. These experiments would likely involve injecting motion pulses at different times conditioned to the same firing rate of the LIP neurons for all times. In this hypothetical experiment, the attractor model predicts time-shift invariance of the induced changes in firing rate, while the DDM with time-decaying boundaries predicts its violation. Further theoretical work is also required to connect realistic neuronal attractor networks with simplified integrator models and to determine to what extent nonlinearities that arise near the decision boundaries can be well approximated by hard decision boundaries (see Roxin & Ledberg, 2008).

*x*

_{1},

*x*

_{2}) =

*x*

_{1}−

*x*

_{2},

*x*

_{1}, and

*x*

_{2}are the states of the integrators at time

*t*, and it is assumed that the two integrators are allowed to evolve freely until the stopping time is externally forced (i.e., bounds are not present, as in Vickers, 1979, and the state of the integrators is read out at the forced stopping time). Since the difference between the states of the integrators grows in mean linearly with time and proportionally to the difference in their drift rates, that is, 〈Δ(

*x*

_{1},

*x*

_{2})〉> = (μ

_{1}− μ

_{2})

*t*, the argument of equation 4.1grows in mean as a function of time, and therefore confidence is expected to increase. Figure 7 shows that decision confidence decreases as a function of decision time for the classical DDM when decision time is internally generated, equation 3.11, and that decision confidence in the same model without boundaries for any given drift rate μ

_{v}= μ

_{1}− μ

_{2}= 2μ

_{0}>0, approximated as increases as a function of the externally forced stopping time. As expected, the curves move downward with increasing input noise σ, since the difficulty of the task increases with it. Mathematically, this is because both equation 3.11 and equation 4.2 are decreasing functions of σ.

In summary, I have provided analytical tools to describe decision confidence and uncertainty in decision making using the DDM. The analytical expressions for decision confidence in partially correlated neuronal integrators found in this letter might allow estimating confidence from multirecording neurophysiological data in decision areas like LIP and FEF (Hanes & Schall, 1996; Shadlen & Newsome, 1996; Platt & Glimcher, 1999) and compare it with behaviorally measured confidence values. The results can be extended to multiple integrators and to more general forms of noise, therefore opening a window for studying how decision uncertainty is represented in neuronal populations in the brain.

## Appendix: Solution for a DDM with Half-Anticorrelated Integrators

This appendix provides details for the derivation of the solution of the FPE, equation 3.17, for the case of half-anticorrelated integrators. It also gives the derivation for the case in which the integrators are independent.

*u*=

*x*

_{1}+

*x*

_{2}and

*v*=

*x*

_{1}−

*x*

_{2}. The free solutions satisfy the FPE, equation 3.17, for any initial condition. However, none of them satisfies the vanishing boundary condition at the semi-lines

*l*

_{1}and

*l*

_{2}.

*x*

_{1}= 0 is solved. I seek a solution with the form consisting of a source at the initial condition (here it is not assumed that

*a*=

*b*, as in section 3.3) and an image source at weighted with the coefficient

*r*. By imposing the boundary condition

*P*(0,

*x*

_{2}) = 0 for all

*x*

_{2}(i.e., the line

*x*

_{1}= 0 is an absorbing boundary), I find that the image coordinates are with α = 2(1 − σ

^{2}

_{u}/σ

^{2}

_{v})/(1 + σ

^{2}

_{u}/σ

^{2}

_{v}), and where

*u*

_{0}=

*a*+

*b*,

*v*

_{0}=

*a*−

*b*,

*u*

_{1}=

*a*

_{1}+

*b*

_{1}and

*v*

_{1}=

*a*

_{1}−

*b*

_{1}. The solution satisfies the initial condition, . If the absorbing boundary is on the line

*x*

_{2}= 0, the solution instead is with the image at and where

*u*

_{2}=

*a*

_{2}+

*b*

_{2}and

*v*

_{2}=

*a*

_{2}−

*b*

_{2}. In both cases, the image is located on the ellipse,

*i*= 1, 2, which includes the source at the initial condition. It is convenient to introduce the matrices which define a mapping from the initial condition to the image positions through the lines

*x*

_{1}= 0 and

*x*

_{2}= 0, respectively, ,

*i*= 1, 2. Repeated and alternated applications of Π

_{1}and Π

_{2}to the initial condition lead to points that lie on the same ellipse, equation A.6.

*l*

_{1}and

*l*

_{2}, for any values of

*c*

_{1}and

*c*

_{2}. This is because the source at the initial condition plus one of the images can vanish on one semi-line, but not simultaneously on the other, implying the necessity of additional images. One possibility is to augment the number of images to try to match the boundary condition at the two semi-lines simultaneously. In fact, I find that a finite number of images is enough for a DDM with two half-anticorrelated integrators, corresponding to (ρ = 1/2 and ν = −1 in equation 2.1). The solution consists of five images, with σ

^{2}

_{u}= σ

^{2}and σ

^{2}

_{v}= 3σ

^{2}; positions , , , , and ; and coefficients , , , , and . The solution is schematically represented in Figure 1D. The images are located on an ellipse, equation A.6. Their positions are obtained by applying iteratively and alternatively the matrices Π

_{i}(α),

*i*= 1, 2, with α = 1 to the initial condition. After defining ,

*i*= 1, 2, we have , , , , and . Images 1 and 2 are the images of the source at the initial condition through lines

*x*

_{1}= 0 and

*x*

_{2}= 0, respectively. Images 3 and 4 can be thought of as the images of the first and second images through the lines

*x*

_{2}= 0 and

*x*

_{1}= 0, respectively, and image 5 as the image of images 3 and 4 through the lines

*x*

_{1}= 0 and

*x*

_{2}= 0. The coefficients have been found by using the previous results for single line boundaries, which allow imposing for each pair of image and its image through one semi-line that the boundary condition is met on that semi-line.

^{2}

_{u}= σ

^{2}

_{v}= 2σ

^{2}; positions , , and ; and coefficients , , and

*c*

_{3}=

*c*

_{1}

*c*

_{2}. A schematic representation of the solution is shown in Figure 1B. All images and initial source lie on a circle, equation A.6. The positions of the two first images are obtained by application of the matrices Π

_{i}(0),

*i*= 1, 2 (α = 0) to the initial condition, . Their coefficients can be obtained from the coefficients of the images required to solve the FPE, equation 3.17, with absorbing boundaries on the lines

*x*

_{1}= 0 or

*x*

_{2}= 0, equations A.4 and A.5. The third image can be thought of as the image of images 1 and 2 through the lines

*x*

_{2}= 0 and

*x*

_{1}= 0, respectively. It is uniquely defined because of the transitivity of the matrix product Π

_{1}(0)Π

_{2}(0), that is, . The coefficient of the third image has been chosen in order to match the boundary condition on both semi-lines simultaneously. The solution for this case can also be obtained by multiplying the known individual solutions for two one-dimensional Wiener processes with absorbing boundaries (Cox & Miller, 1965).

## Acknowledgments

I thank Alex Pouget and Jan Drugowitsch for insightful discussions. This work has been partially supported by the Swartz Foundation.

## Notes

^{1}

*t*as where δ

*x*(i = 1, 2) are the variable increments in each step and

_{i}*w*(

_{i}*i*= 1, 2,

*c*) are independent across time and follow normal gaussians.