Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie [2019. Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience , 19 (3), 523–536]. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus–outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

Recent interest in the role of the hippocampus in temporal aspects of cognition has been fueled, in part, by the observation of “time” cells in the rodent hippocampus—that is, cells that have differential firing patterns depending on how long ago an event occurred. Such cells are thought to provide an internal representation of elapsed time. Yet, the hippocampus is not needed for processing temporal duration information per se, at least on the order of seconds, as evidenced by intact duration judgments in rodents and humans with hippocampal damage. Rather, it has been proposed that the hippocampus may be essential for coding higher order aspects of temporal mnemonic processing, such as those needed to temporally organize a sequence of events that form an episode. To examine whether (1) the hippocampus uses duration information in the service of establishing temporal relations among events and (2) its role in memory for duration is unique to sequences, we tested amnesic patients with medial-temporal lobe damage (including the hippocampus). We hypothesized that medial-temporal lobe damage should impair the ability to remember sequential duration information but leave intact judgments about duration devoid of a sequential demand. We found that amnesics were impaired in making judgments about durations within a sequence but not in judging single durations. This impairment was not due to higher cognitive load associated with duration judgments about sequences. In convergence with rodent and human fMRI work, these findings shed light on how time coding in the hippocampus may contribute to temporal cognition.

Medial-temporal lobe (MTL) lesions are associated with severe impairments in episodic memory. In the framework of the temporal context model, the hypothesized mechanism for episodic memory is the reinstatement of a prior experienced context (i.e., “jump back in time”), which relies upon the MTL [Howard, M. W., Fotedar, M. S., Datey, A. V., & Hasselmo, M. E. The temporal context model in spatial navigation and relational learning: Toward a common explanation of medial temporal lobe function across domains. Psychological Review, 112, 75–116, 2005]. This hypothesis has proven difficult to test in amnesia because of the floor-level performance by patients in recall tasks. To circumvent this issue, in this study, we used a “looped-list” format, in which a set of verbal stimuli was presented multiple times in a consistent order. This allowed for comparison of statistical properties such as probability of first recall and lag-conditional response probability (lag-CRP) between amnesic patients and healthy controls. Results revealed that the lag-CRP, but not the probability of first recall, is altered in amnesia, suggesting a selective disruption of temporal contiguity. To further characterize the results, we fit a scale-invariant version of the temporal context model [Howard, M. W., Shankar, K. H., Aue, W. R., & Criss, A. H. A distributed representation of internal time. Psychological Review, 122, 24–53, 2015] to the probability of first recall and lag-CRP curves. The modeling results suggested that the deficit in temporal contiguity in amnesia is best described as a failure to recover temporal context. These results provide the first direct evidence for an impairment in a jump-back-in-time mechanism in patients with MTL amnesia.