A summary of belief updating and behavior of our simulated affective agent over 64 trials. Probabilistic beliefs are plotted using a blue-yellow gradient (corresponding with high-low certainty). As shown in the graphic that connects panels c and d, the dynamics of this simulation can be divided into four quarters: two periods within each of the 32 trials before and after the context reversal, each comprising an initial phase of negative valence (anxiety; quarters 1 and 3), followed by a phase of positive valence (confidence; quarters 2 and 4). (a) The context changed midway through the experiment (indicated in all panels with a vertical green line): food was on the left for the first 32 trials (L) and on the right for the subsequent 32 trials (R). (b, c) These density plots show the subject's beliefs about the best course of action, both before (panel b) and after the trial (panel c). Prior beliefs were based purely on baseline priors and her action model, which entailed high ambiguity (yellow) during quarters 1 and 3 of the trial series (corresponding with cue-checking policies V and high certainty (blue) during quarters 2 and 4 (corresponding to shortcut policies V. After perceptual evidence was accumulated (after the trial), posterior beliefs about policies always converged to the best policy, except in the first trial after context reversal (trial 33, when the rat receives a highly unexpected shock), which explains her initial confusion. Whenever prior certainty about policies was high, expectations agreed with posterior beliefs about policies (again, except for trial 33). (d) This density plot illustrates affective inference in terms of beliefs about her valence state (confident positive or anxious negative states ). Roughly speaking, our rat experienced a negatively valenced state during quarters 1 and 3 and a positively valenced state during quarters 2 and 4. (e) We plot lower-level expected precision (), overlaid on a density plot of valence beliefs (grayscale version of panel d).
