Mechanism by which confirmation bias tends to increase reward. (a) Average reward and reward distributions for different levels of confirmation bias. The heat map represents the per trial average reward of the confirmation model for all learning rate combinations (confirmatory learning rates are represented on the $y$-axis whereas disconfirmatory learning rates are represented on the $x$-axis) associated with a softmax policy with $β=0.1$⁠. The rewards concern the stable condition with 128 trials and asymmetric contingencies (⁠$p-=0.35$ and $p+=0.65$⁠) and are averaged across agents. The three signs inside the heat map (⁠$Δ$⁠, $×$⁠, and $+$⁠) represent the three learning rate combinations used in the simulations illustrated in panels b and c. The histograms show the distribution across agents of the average per trial reward for the three different combinations. (b) Estimated values. The line plots represent the evolution of the best option value $V+$ across trials. The large plot represents the agents-averaged value of the best option across trials for three different learning rate combinations: “unbiased” (⁠$αC=αD=0.25$⁠), “biased (low)” (⁠$αC=0.35$ and $αD=0.15$⁠), and “biased (high)” (⁠$αC=0.45$ and $αD=0.05$⁠). The lines represent the mean and the shaded areas, the SEM. The small plots represent the value of the best option across trials plotted separately for the three combinations. The thick lines represent the average across agents and the lighter lines the individual values of 5% of the agents. (c) Choice accuracy. The line plots represent the evolution of the probability to select the best option across trials. The large plot represents the agents-averaged probability to select the best option across trials for three different learning rates combinations: “unbiased” (⁠$αC=αD=0.25$⁠), “biased (low)” (⁠$αC=0.35$ and $αD=0.15$⁠), and “biased (high)” (⁠$αC=0.45$ and $αD=0.05$⁠). The lines represent the mean and the shaded areas, the SEM. The small plots represent the probability of selecting the best option across trials plotted separately for the three combinations. The thick lines represent the average across agents and the lighter lines the individual probability for 5% of the agents.