Needing: An Active Inference Process for Physiological Motivation

Our bodies preserve their stability against the many disturbing forces through homeostasis and its mechanisms (Cannon, 1939). Among the mechanisms of homeostasis and its adaptive form, allostasis (Sterling & Eyer, 1988), there is a particular one that we will refer to as “needing.” Generally, needs intensify and exert a pervasive influence on various aspects and dimensions of our lives. For example, when experiencing hunger, its effect persists until satisfied by food. This principle extends to other needs such as thirst and sleep. Therefore, states of need become pervasive, shaping our perceptions, decisions, and more.

Needing is a process related to internal states (Livneh et al., 2020; Craig, 2003) characterized by a deprivation of essential elements crucial for life or survival (Bouton, 2016; Baumeister & Leary, 1995; MacGregor, 1960). Here, we refer to those (internal) states as need states. Such states (through the needing mechanism) have a directional effect on motivation (Bosulu et al., 2022; Dickinson & Balleine, 1994; Balleine, 1992). Needing also interacts with other subsystems that process external stimuli, such as wanting, liking, or interception. Thus, needing is both separate from (e.g., see Watson, Wiers, Hommel, & de Wit, 2014; Hogarth & Chase, 2011), but also interacts with (see Berridge, 2004), different reward-related subsystems. This happens in an independent way; for instance, need states tend to amplify the incentive salience of relevant Pavlovian cues that generate “wanting” (Berridge, 2004; Toates, 1994) independently of “liking” and (re)learning (Berridge, 2012, 2023). Yet needing can, independently of wanting (i.e., of Pavlovian cues) amplify “liking” or pleasure (Becker et al., 2019; Cabanac, 2017; Berridge & Kringelbach, 2015) and influence learning (Salamone, Correa, Yang, Rotolo, & Presby, 2018; Wassum, Ostlund, Balleine, & Maidment, 2011; Dickinson & Balleine, 1994; Balleine, 1992) and interoceptive prediction (Bosulu et al., 2022; Livneh et al., 2020), and needing can also directly activate relevant actions (Passingham & Wise, 2012) or explorative behavior (Panksepp, 2004) as well as behavior related to autonomic and neuroendocrine levels (Swanson, 2000).

Being related to internal states (Sterling & Laughlin, 2015), needing generates the value attributed to rewards “from within” and is thus separate from the external prediction of reward. Indeed, need states can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation/wanting (Berridge, 2012, 2023). In the same sense, needing can also modify the perception (e.g., the pleasure) of relevant stimuli independently of their original sensory valence (Cabanac, 1971, 2017). Water tends to acquire a pleasing taste when one is thirsty, and a warm environment or object feels comforting when experiencing cold. Conversely, when too hot, seeking out a cool place can be equally satisfying. These instances highlight how the directional impact of needing plays a crucial role in shaping motivation, ultimately influencing what is perceived as rewarding within a given need state. This directional influence of needing has the ability to alter the perception of pertinent stimuli, irrespective of the predicted or “actual” value of the reward.

Nevertheless, a comprehensive formal framework that encompasses the aforementioned findings and specifically elucidates the following aspects remains elusive: (1) the mechanism by which needing steers motivation, independently of the external world, and (2) the intricate interplay between needing and external rewards processed within other subsystems, such as wanting associated with relevant Pavlovian cues, or liking linked to relevant hedonic sensation. In the following sections, we will first tackle these points/questions conceptually, using formal methodologies from active inference (Parr, Pezzulo, & Friston, 2022) and introduce the concept of pervasiveness of a need state. Subsequently, we will present three simulations that delve into the functioning of the needing system and its interactions with wanting (and liking).

In the next two sections, we present a conceptual perspective aiming to elucidate two fundamental aspects of needing: (1) its directional impact and (2) its interplay with other subsystems.

A fundamental objective of organisms is to regulate and maintain their internal states within specific narrow limits (Barrett, 2017; Sterling & Laughlin, 2015; Friston, Kilner, & Harrison, 2006; Cannon, 1939). For instance, the average normal body temperature for humans is generally between 36.1°C (97°F) and 37.2°C (99°F), which is a very small range compared with the range of possible temperatures in the universe, from the absolute zero to trillions of degrees. The same is true for levels of glucose or the balance between water and salt in the body. The underlying principle is that the spectrum of “states” conducive to life is exceedingly limited in contrast to the immensely large number of alternative combinations that would not sustain life. Therefore, to ensure a living organism remains within its normal physiological boundaries, natural evolution might have established these boundaries as innate preferred states—possibly encoded genetically—that the organism consistently endeavors to attain. From the formal standpoint of active inference, these preferred states (which might correspond, for instance, to physiological bounds) are referred to as (empirical) priors. They carry a higher probability of realization from the organism's perspective, meaning they are less surprising (Friston, 2010).

The perception of a need state translates into a “goal” of reducing surprise by reaching the preferred states, for example, states that represent adaptive physiological conditions (Friston, 2010). Such a tendency could activate an action or a policy (i.e., a sequence of actions) that compels creatures to seek out the (valuable) preferred states. Note that the actions or policies that resolve a state of need could in some cases correspond to (fixed) regulatory actions, such as autonomic reflexes, as opposed to action courses determined by the circumstances of the external environment (Sajid, Ball, Parr, & Friston, 2021). With time, the states that the creature occupies when pursuing a policy that resolves a need can become valued per se (Friston & Ao, 2012). In other words, when the creature pursues a course of actions toward the preferred state, all the intermediate states (here intended in a broad sense that encompasses situations, actions, stimuli, etc.) can become valued and needed, through a Pavlovian mechanism (Berridge, 2018; Bouton, 2016). For instance, when moving from a state of hunger to a state of satiety, some intermediary states, such as the gustatory stimulus associated to having food and the act of eating, would become valued, because they are in the path toward the preferred (satiety) state (Pezzulo, Rigoli, & Friston, 2015). Because of its connection with prior preferences, possibly genetically encoded, which may or may not be cognitive, needing influences both reflexive and cognitive (higher level) goals, like shivering for warmth or deciding to buy winter clothing, extending an organism's control over its states (Cisek, 2022). In summary, a creature that is far from preferred states would experience a need (e.g., for food)—and then start to prefer the valued states (here, intended in a broad sense that encompasses stimuli, actions, etc.) that secure the relevant reward (e.g., food) or are part of the experience of having food (e.g., food cues or relevant actions). In this sense, the tendency to occupy preferred states confers to need states the possibility to influence—and give value to—stimuli or actions that are either costly states (noted S(h_n)) that lead to surprise, or in the path toward the preferred state (noted π(p)). In other words, when one experiences needing, any state (stimulus or action) that is in the path to the preferred state will become something one needs (and hence valued) because it reduces need-related entropy. Hence, the directional effect of need states on motivation could come from the tendency to occupy these preferred states.

A need state could be defined as a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is surprising with regard to prior preferences. Needing can then be defined as an active inference process that aims at reducing such pervasive surprise by inducing a tendency to transition, from states to states, toward the preferred state. The effect of needing does not have to be learned anew, but it is “actively inferred”—reflecting the fact that need states (hunger, thirst) and needed stimuli (food, water) are related to fundamental priors that are potentially sculpted by natural selection. Indeed, even very simple creatures that have limited (or no) learning capacities are able to navigate their environment adaptively to fulfill their current needs, for example, by following food gradients (Sterling & Laughlin, 2015). The implications of this definition of needing is that it allows shifts in need states to direct motivation toward relevant stimuli even without learning, for instance through alliesthesia (the natural change in sensation/perception of a relevant stimulus induced by a need state). A striking illustration of this phenomenon occurs when animals would suddenly have an increase in “wanting” associated with a salt cue, when depleted of salt even in absence of (re)learning (Berridge, 2012, 2023). This happens even when that cue was learned to be predictive of negative outcome and without the animals having (re)learned about the cue (or the salt) through tasting (and liking) it in the newly induced need (salt depleted) state. Our article provides a plausible explanation through the lenses of active inference and pervasiveness. The switching from learned negative outcome to wanting happens through pervasiveness, which progate surprise to all other states (stimuli, cues, goals, etc.) except the states on the path to the priori preferred state. Notably, this category encompasses the (memories of) salt and its cue, as these stimuli still co-occurred with salt in the body, in a multisensory-like representation, despite being learned to be negative (Smith & Read, 2022). Thus by reducing such pervasive surprise through an active inference process, the animal would tend to have an increase in wanting associated with such salt cues without (re)learning. This adjustment is achieved through precision increase as elaborated below.

The effect of needing on wanting (and on other phenomena such as pleasure and liking) could be conceptualized by appealing to the formal notion of precision in active inference. Mathematically, precision is a term used to express the inverse of the variance of a distribution, which in our context can be seen (loosely speaking) as the inverse of entropy (Holmes, 2022; Friston, 2010)—in the sense that the higher the entropy, the lower the precision. In predictive coding and active inference, precision acts as a multiplicative weight on prediction errors: Prediction errors that are considered more precise have a greater impact on neural computations (Parr et al., 2022).

With respect to need states, precision can be interpreted as a higher salience attributed to the most relevant stimulus given the need state. There are different precisions associated with different subsystems, such as those related to interoceptive streams, rewards, policies, and so forth (Parr et al., 2022). At the neurophysiological level, policy precision, or the confidence that a policy should be followed, is typically associated with the dopaminergic subsystem in active inference (Holmes, 2022; Parr et al., 2022; FitzGerald, Dolan, & Friston, 2015). Therefore, Pavlovian cues that enhance policy precision and confidence that the policy should be followed would trigger dopamine bursts, which will attribute incentive salience, that is, wanting, to such cues (Berridge, 1996, 2007). This is in line with the idea that dopamine is linked with incentive salience and wanting, but also with reward cues and behavioral activation as they typically co-occur (Hamid et al., 2016, but see Berridge, 2023). Rather, precisions regarding hedonic contact with the reward (to ask questions such as: Is it good?) or the state of satiety (to ask questions such as: Am I well?) might be mediated by the opioid subsystem (Berridge & Kringelbach, 2015) and the serotonin subsystem (Parr et al., 2022; Liu, Lin, & Luo, 2020; Luo, Li, & Zhong, 2016). The impact of needing on stimuli that are processed within these subsystems are “natural” increases in precision because of prior preferences. The need-induced increase in precision implies more certainty that the state to which the stimulus or policy leads to is the least surprising. It is this certainty that amplifies wanting, liking, and so forth, and it is an active inference process that may be separated from learned reward prediction (also see Berridge, 2023).

This discussion helps appreciate the deep interdependence between needing (which acts on the system as a whole) and wanting as well as other subsystems such as the hedonic/liking and interoceptive ones. When one is in a surprising (need) state, the presence of a cue (e.g., a traffic or restaurant sign) might reduce uncertainty about goal/reward achievement by improving policy precision via the dopamine subsystem (wanting). The presence of, or contact with, the reward itself might reduce entropy by enhancing precision through the opioid subsystem (pleasure/liking). Finally, being in a preferred state or moving toward the preferred state determines an increase of precision—which is because of the fulfillment of prior preferences—via the serotonin subsystem (well-being).

In all these cases, rewards and cues are useful information sources: They reduce entropy by signaling the availability of a path to preferred states (π(p)), or equivalently a path away from surprising states (S(h_n)), given some prior preference (C_p). Indeed, from the point of view of the organism in a need state, both encountering either a relevant reward or a cue that leads to that reward would reduce uncertainty about how to move to the preferred state and alleviate the need. This is despite making contact with a reward and encountering a cue that predicts reward might be treated in different subsystems: the liking and the wanting subsystems, respectively.

Our discussion so far has highlighted two crucial aspects of needing and its interactions. First, need states exert a directional influence on choices separately from reward prediction. This is because of the pervasiveness of need states that make states surprising with regard to prior preferences and to the animals' tendency to reduce surprise, which naturally increases the value of rewards, cues, and actions that lead to the preferred state. This tendency exists irrespective of reward prediction (Berridge, 2023; Smith & Read, 2022; Zhang, Berridge, Tindell, Smith, & Aldridge, 2009). Second, when there is a path to the preferred state, such as a reward or a Pavlovian cue, needing would increase the value of reward or Pavlovian cues within the subsystems that process them. This translates into a lowering of entropy about which state to occupy (i.e., entropy of the probability distribution of the to-be-reached states) to transition to the preferred state, and thus an increase in the precision of relevant stimuli (e.g., liking) or goal-achieving policies (e.g., wanting). Such high precision could be viewed as need-induced salience, which, for instance, in the wanting subsystem translates into higher incentive salience. With these insights in mind, we now shift from a conceptual discourse to the practical implementation of needing through an active inference framework.

In the following section, we present the mathematical formulation by introducing an agent endowed with needing. This agent is utilized in simulations to illustrate the functionality of the needing system and its influence on reward subsystems, such as wanting and liking. In this section, we adopt a parallel structure to that of section Needing System: A Conceptual Perspective. Concerning the directional impact of needing (section The Directional Effect of Needing), we delve into mathematical specifics of the subsections (from sections To Remain Within Their Physiological Boundaries, Organisms are Endowed with Apriori Preferred States That They Tend to Occupy to Needing Is an (Active) Inference Process That Aims at Reducing Pervasive Surprise). In addition, with respect to the interaction between needing and other subsystems (section Interaction between Needing and Other Subsystems), we explore the mathematical intricacies associated with both the need-related entropy, and need-induced precision increase. We then map the mathematical formulation to the conceptual perspective in sections The General and Directional Motivation of Needing and Interaction between Needing and Other Subsystems Such as Wanting, Liking, and So Forth.

The conditional probability of the states Y given prior preference will decrease proportionally to the amount of need (n) they embed, and this decrease in probability will increase the surprise of being in those states. Because of the pervasive effect of a need state, the value of any state y_i with regard to (the amount of) need (noted y_i,n) that one has while being in that state depends on its co-occurrence, that is, joint probability, with the state that generated the need, noted y_n.

Assuming that animals have prior preferences over levels of satiety/need (e.g., a preferred level of sugar in the body), the prior preference probability over satiety is equal to p(C_p,n) = 1.

Given the pervasiveness of the need state, almost all of the y_i become surprising because their joint probability, that is, co-occurrence, with the preferred state decreases. Hence, only the states “food in stomach” and the states that lead to it have high probability as they (eventually) co-occur with the priori preferred state. Crucially, as it will become clear in our first simulation, this probability increases as need increases, because the sum of probability of p(y_i,n|C_p,n) has to sum to 1.

Because the creature expects to occupy (or to move toward) these a priori probable states, the prior over states also translates into priors over actions or action sequences (policies) that achieve such states. In this simplified setting, action (and policy) selection simply corresponds to inferring a distribution of states that it prefers to occupy and policies to reach (sequences of) these states. In other words, the active inference agent tends to select policies that lead it to achieve goal states—which in Bayesian terms correspond to maximizing model evidence (Parr et al., 2022).

In this context, y_i,n represents the need state, whereas minimizing G(π) denotes needing, the process facilitating the fulfillment/satisfaction of the need state. This later process is contingent upon the path or policy leading to the preferred state (while both y_i,n and G(π) depend on prior preference). For instance, hunger on its own represents a need state where one is far from the preferred state of satiety. When there is a path to that preferred state, the tendency within different subsystems to value states along that path (e.g., food cues, taste, interoception) is what we describe as “needing.”

Needing is thus the minimization of G(π), or to expressing this in a “positive” way, needing is the maximization of E_Q(y|π) ln[P(y_i,n|C_p,n)]. In other words, needing induces a tendency to seek states in which the expected log-probability given the prior preference is maximized under that specific (pervasive) need state.

Now, we consider the effect of a need state on (1) the entropy over the states that a creature plans to occupy in the future by following its inferred policy and (2) the inverse of the entropy, that is, the precision, which is a measure of certainty about which states to occupy next.

Here, H (i.e., H_n or H_n,p) denotes the entropy and it can be calculated on two sets of states. When a reward (or its cue) state is available, the entropy is over which states to occupy by the creature when some of the states are on a path π(p) to the preferred state (p) whereas the rest of states are not (S(h_n)). Alternatively, when there is no reward, then the entropy is over which states S(h_n) to occupy.

In the specific context of the interaction between needing and “wanting,” we can draw a connection between our precision equation and the computational models pertaining to wanting and incentive salience proposed by Smith and Read (2022) and Zhang and colleagues (2009). In both models, a variable denoted as “k” represents the impact of physiological and other brain states on the dopaminergic state. This “k” factor alters the reward value “r,” subsequently leading to an amplification of the reward cues, expressed as the function $r_rt,k$⁠. When such a variable “k” is influenced by need states, the function $r_rt,k$ becomes associated with precision over policies that guide the attainment of the preferred state, denoted here as P_n,p.

Illustrating the connection between needing and wanting (Figure 2).

A need state could be defined as a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is (biologically) surprising with regard to prior preferences. Such surprise is noted: h_n(y) = −lnP(y_i,n|C_p,n). The pervasiveness of need states, given into prior preferences, propagates such surprise depending on the joint probability between a need state y_n and another state y_i given by p(y_i, y_n) [n]. Prior preferences then “judges” how any state y_i should be preferred given the preferred state in which that need is alleviated, noted C_p,n, and in which the system expects to be in, and this is given by: P(y_i,n|C_p,n). Based on that, the pervasiveness of need states make many states less probable, that is, surprising, noted S(h_n). Hence the system, expressing states in terms of surprise, that is, −ln P(y_i,n|C_p,n), can “infer” the trajectory, that is, the couple of states which are noted π(p), that lead to the preferred state, by choosing states (actions, cues or rewards) from which one expects minimization of such surprise: −E_Q(y|π) ln[P(y_i,n|C_p,n)]. Needing is this active inference process that aims at reducing such pervasive surprise by seeking states that maximize E_Q(y|π) ln[P(y_i,n|C_p,n)], that is, states whose expected log-probability given the prior preference is maximized under that specific (pervasive) need state.

Need states can amplify wanting or liking and thus assign a high precision P_n,p to Pavlovian cues or hedonic contacts with rewards that lead to the preferred state. Indeed, by following policies that minimize −E_Q(y|π) ln[P(y_i,n|C_p,n)] toward the preferred state and away from surprising states S(h_n), when there is a possibility/path to reach such preferred state, the states on that trajectory π(p) become more and more probable given the prior preference. This is because, because of pervasiveness, the states on π(p) have a lower joint probability with the need state, and this decreases entropy over which state to occupy. In other words when one is in need (surprising) state that becomes pervasive, and there is a path to the preferred state, the entropy H_n,p (Y = S(h_n), π(p)) = H_n,p decreases by following the policies that minimize −E_Q(y|π) ln[P(y_i,n|C_p,n)]. Thus, the inverse of that entropy $Hn,p−1$⁠, that is, the precision (or neuronal gain) P_n,p assigned to states (stimuli/events/reward cues) π(p) that leads to the preferred state is increased. This increase in precision happens within any subsystem (wanting, liking, interoception, etc.) if such π(p) states (stimuli/events/reward cues) happen to be processed by that subsystem. So, the interaction between needing and, for instance, wanting happens when needing enhances mesolimbic dopamine reactivity, which assigns higher precision to Pavlovian cues that are relevant under the need state. The enhancement of dopamine reactivity amplifies/generates “wanting” associated with relevant Pavlovian cues by acting as neuronal gain expressed as P_n,p.

In the next sections, we subsequently illustrate the functioning of the model in two simulations, which exemplify how being in need states influences the tendency to reach preferred states independently of reward prediction (Simulation 1), and how the simultaneous presence of a state of need and the presence of a path to the preferred (reward or goal) state implies low entropy and high precision over which state to occupy (Simulation 2).

We designed a 3 × 3 grid world with nine states, in which only one state (State 2) is rewarding/preferred, one is a need state (State 8), that is, it is (biologically) costly/surprising, and the other seven states are “neutral”; see Figures 1 and 3. We can draw a parallel between the State 2 and 8 and human physiological states, such as hunger or temperature: The preferred state (2) corresponds to the optimal interval of sugar level in the bloodstream, or the temperature range (between 36.1°C [97°F] and 37.2°C [99°F]), and the State 8 is a deviation from that.

In the grid world, each state corresponds to a box in Figure 3. In our simulations below, we will assume that the agents are in a need state (e.g., hunger) that can be low, mild, or severe. Accordingly, a negative reward of −1 (slightly hungry), −2 (hungry), or −5 (starving) is assigned to the “costly” state (State 8). State 2 represents a reward/preferred state that gives a reward of 1 and resolves the need state. Note that the agents that dwell in the simulated environment sense this biological reward/cost and will have to compute the expected biological costs, or values, by themselves. Of note, the grid world is used here for visual presentation purposes and the proposed framework can indeed be generalized beyond a grid world.

Our simulations will consider two agents (see the Appendix and the Appendix table). Agent 1 incorporates the main ideas discussed in this article about the needing system. It is a simplified active inference model that bases action selection on prior preferences about states and policies (Parr et al., 2022). Agent 2 is used as a comparison, to show that reward prediction alone is not sufficient to explain the directional effect of needing and its interplay with other subsystems such as wanting. It is a simplified reinforcement learning system that based action selection on learned action values computed by reward prediction (Sutton & Barto, 2018) and perceives a “need” without the pervasiveness of need states as described here. The goal of Simulation 1, illustrated below, is to assess the effects of increasing need states on the action selection mechanisms of the two agents.

The results illustrated in Figure 4 show that increasing the cost of the need state, when pervasiveness occurs, significantly increases the probability assigned to policies that reach the rewarding state in Agent 1. This is evident when considering that the probability increases from (about) 0.5, 0.8, and 1 in the three left rows. However, increasing the costs of the need state without need-related pervasiveness does not affect reward prediction as shown in Agent 2. This is evident when considering that the softmax of state-action (Q) values assigned by Agent 2 to the rewarding state is always relatively the same in the three right rows. These results help illustrate the idea that costly or need states might exert directional effects and impact on the probability (or tendency) to reach preferred states through pervasiveness, irrespective of reward prediction.

In Simulation 2 (using the same grid-world environment), we ask if being in a state of greater need increases precision associated with rewards, that is, states that are on the path to the preferred state. We compare two conditions, namely, a condition when a reward is present (i.e., the reward State 2 is baited with a reward of 1) or absent (i.e., the reward State 2 has the same cost as all the other neutral states). The results of the simulations of entropy and precision can be appraised graphically in Figure 5. These results shown indicate that compared with the case with no reward, the condition where a reward is present implies a significant decrease of the entropy over which states the Agent 1 plans to occupy in the path to the reward (Figure 5, left) and a significant increase of its precision, which is a measure of certainty about which states to occupy in the future (Figure 5, right). This is because the availability of a reward makes the agent more confident about the states to occupy and the policy to select, whereas in the absence of a reward, all states are equally costly/surprising and the agent has no strong preference about which states to occupy (i.e., high entropy and low precision). The presence of a reward (which in this simulation is known by the agent) is a path that makes it possible to pursue a preferred course of action toward the preferred state, reducing the entropy about the states to occupy and increasing the certainty (precision) about the states to visit in the path toward the preferred state.

Given the importance of more directly comparing our theoretical framework to empirical data and demonstrating how and why needing arises as pervasive surprise from prior preferences, we adjusted our parameters and devised a (third) simulation closely resembling Robinson and Berridge's (2013) experiment. They showed that needing can increase the wanting associated with a cue even if that cue predicts disgust sensation, demonstrating that needing can amplify wanting independently of prediction (Robinson & Berridge, 2013) also discussed in Berridge (2023).

Oral infusions of NaCl solution at Dead Sea concentration into the mouth of a rat typically elicit strong “disgust” reactions and Pavlovian cues that predict such stimuli quickly become repulsive and elicit retreat after a few pairings (Robinson & Berridge, 2013). In Robinson and Berridge (2013), each rat quickly learned to shrink away from the conditioned stimulus that was paired with Dead Sea salt—the CS + salt lever—whenever it appeared, retreating to a far wall as if trying to escape from the Pavlovian cue and its predicted salty infusion (Berridge, 2023). However, on a particular test day, the rats suddenly found themselves for the first time in their lives in a new state of physiological sodium deficiency through an injection of drugs that mimic the natural brain hormonal signals of salt appetite (Berridge, 2023). This was a never before encountered state because these (modern laboratory) rats, like most modern humans, have never experienced a salt appetite of that intensity: Their food, like ours, contains more than enough salt (Berridge, 2023). Crucially, on the test day, the rats re-encountered their Pavlovian CS + salt cue before ever experiencing (e.g., tasting) new positive hedonic value of Dead Sea saltiness as ‘liked’ (the liking part was tested later and was also shifted (see Robinson & Berridge, 2013)). They had only their past memories of Dead Sea disgust to guide their learned value of the CS + salt cue. However on the test day, before ever retasting the salty stimulus, the sodium-deficient rats immediately ran to their CS + salt lever as soon as it appeared, jumping onto and avidly nibbling the metal lever that had previously repulsed them (Berridge, 2023). The previously learned negative value of the salt cue was now discarded and the CS + salt cue instantly elicited positive wanting in their novel sodium-deficient state (Berridge, 2023; Robinson & Berridge, 2013), as shown in Figure 6A. This indicates that needing can, instantaneously, increase the wanting of a cue independently of learned prediction afforded by that cue, as we show in the simulation below.

To replicate the Robinson and Berridge (2013) experiment, we conducted a “one-step decision” simulation. We created a simulation (similar to simulation one) where there was a negative state, the Dead Sea salt, which had a negative value, whereas all other states were positive. We compared an active inference agent (Agent 1) built with needing system, and an agent that learned through temporal difference (TD; Sutton & Barto, 2018) over hundreds of trials (Agent 2). Both agents (1 and 2) initially evaluated a state called “Dead Sea salt cue” negatively under satiety, similar to the conditioning step in the Robinson and Berridge (2013) experiment. Then, we tested them with a single-step decision, after inducing a need state assumed to be related to salt, where only the Dead Sea salt cue could satisfy the need. That is, only the Dead Sea salt cue would become positive. The “depletion test” was a single-step decision, directly after changing their state into a “need state.” As can be seen in Figure 6, on their last two decisions before “salt depletion,” both Agents 1 and 2 (i.e., under needing and under learned values) had a negative valuation for the state “Dead Sea salt” before induction of the need state. At the one-step immediately after need induction, Agent 1 shifted the value of the Dead Sea salt cue, similar to the findings of Robinson and Berridge (2013), whereas Agent 2 retained learned value in the one time-step decision, as shown in Figure 6B.

In this scenario, attributing positive value to the Dead Sea salt/cue does not depend on learning (not even the “incentive learning,” i.e., learning of the need-state/reward pair; see Balleine & Dickinson, 1998), because the agents or the rats have never experienced salt depletion, and came to “want” the Dead Sea salt/cue before ever (re)experiencing it in the salt-depleted state (Robinson & Berridge, 2013). Rather, the Dead Sea salt has a positive value because salt level is encoded as a prior preference. The depletion of salt (i.e., being far from prior preference) generates a pervasive need, and the Dead Sea salt is the only way to fulfill the prior preference—hence, it is sought after and valued. In other words, the sudden positive valuation of the Dead Sea salt reflects the needing process described in this article. In this sense, the sensory representation linking the Dead Sea salt cue to salt level becomes the path/policy under which the state (Dead Sea salt) that reduces pervasive surprise (deviation from salt level) is more probable. This translates to the increase in precision described in Simulation 2.

We do not imply that animals solely rely on needing and not learning; rather, they utilize both mechanisms. A needing-based agent lacking proper learning mechanisms may survive in a stable environment as a simple creature, but not as effectively as adaptive animals do in dynamic environments with numerous states and events. Needing and learning can indeed work together (for instance in the case of incentive learning; see Balleine & Dickinson, 1998). However, in scenarios like Robinson and Berridge's (2013) experiment, the needing mechanism likely accounts for the immediate increase in wanting associated with the dead salt cue in the unlearned salt depletion state. If salt depletion persists, learning will eventually assign high value to the dead salt cue over multiple trials. However, this gradual update contrasts with the instant change observed in Robinson and Berridge's (2013) experiment, suggesting an active inference process in determining which sensory representation to value, driven by the pervasiveness that renders all other sensory representations surprising except the Dead Sea salt cue. The fact that needing influenced both the Dead Sea salt lever cue (wanting) and (later) the hedonic contact with the Dead Sea salt (liking) indicates that needing increases different forms of precision within different subsystems, depending on whether the available stimulus is processed by that subsystem. Moreover, all this shows that, under need state, the valuation of states (stimuli or actions) has to do with both minimization of surprise and availability as a path to the preferred state, which are represented via y∣C and y∣π, respectively, in the −E_Q(y|π) ln[P(y_i,n|C_p,n)] equation. The tendency or process under wich π is selected in a way that minimizes the aforementioned equation is what we describe as “needing.”

The role of needing and the way it interacts with other reward subsystems are not completely understood. Here, we aimed to provide a computationally guided analysis of the mechanisms of needing from the normative perspective of active inference and the hypothesis of (embodied) pervasiveness of need states.

We firstly defined a need in terms of pervasiveness and surprise; that is, a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is surprising with regard to prior preferences. We then defined needing as an active inference process that aims at reducing such pervasive surprise by inducing a tendency to transition from states to states, toward the preferred state. Here, the term “surprise” is used in the sense prescribed by theories like predictive coding and active inference. The key idea is that living creatures strive to remain within tight physiological boundaries (i.e., preferred states) and are surprised outside them—like a fish out of water. This perspective suggests that being in a need state is intrinsically costly (where the cost is related to surprise), and thus a state of need may exert a directional effect on action selection and motivation, because creatures would have an automatic tendency to select policies that avoid surprises and lead to preferred states. Importantly, this automatic tendency would be present independently of any reward or reward cue. Moreover, the pervasive effect of a need state acts on the system as a whole, making all states surprising with regard to prior preferences, except the relevant rewarding states, that is, those on the path to the preferred state. Hence, it is the embodiment of that pervasiveness into prior preferences that allows needing to activate prior policies that direct the system, in a gradient-like manner, toward the relevant rewarding states.

We further defined the interaction between needing and other subsystems, specifically the wanting (or liking) subsystem, where needing increases the precision of relevant stimuli or action processed within those subsystems. In the case of wanting, this precision is that of policies that achieve preferred (goal or reward) states in active inference, consistent with previous work that linked policy precision with dopaminergic reactivity (Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2017; FitzGerald et al., 2015) and thus incentive salience (Berridge, 2004). From this perspective, Pavlovian cues that signal that there is a path to preferred state acquire incentive salience (Berridge, 2004) and generate “wanting.” Needing amplifies such wanting, as a state of greater need can amplify policy precision by amplifying the value of reward cues that do not co-occur with the pervasive surprise (because they lead to the preferred state). Hence, the higher the initial state of need, the greater the wanting associated with Pavlovian cues related to relevant rewards.

This simulation demonstrates that for an organism that uses prior preferences to embody the pervasiveness effect, need states can have directional effects independently of reward prediction. Needing governs directional motivation because of the inherent tendency of living beings to move toward preferred states. This propensity activates policies leading to those preferred states, which in turn elevates the preference (and value) of states within their trajectory. Homeostasis and allostasis, which assist animals in maintaining viable physiological boundaries (Holmes, 2022; Demekas, Parr, & Friston, 2020; Barrett, 2017; Sterling, 2004; Sterling & Eyer, 1988; Cannon, 1939), mediate this tendency. Indeed, from the active inference perspective, a living organism continuously strives to reach or remain in its preferred states (which could be sometimes evolutionarily defined) through homeostatic or allostatic regulation, at the somatic, autonomic, and neuroendocrine levels (Parr et al., 2022; Swanson, 2000). These preferred states act as drives or goals that, via homeostasis and allostasis, direct action (Barrett, 2017); hence, the directional effect of need states. This directional influence, contingent on the propensity to occupy preferred states, also underlies the amplifying effect of need states on wanting, liking/pleasure, interoceptive prediction, choice, and so forth, by increasing the precision of their related states (stimuli or actions) located on the path toward the preferred state. This leads to Simulation 2 below.

This simulation shows that the presence (vs. the absence) of (a path to) a reward decreases the entropy of the states that Agent 1 plans to occupy and increases the associated precision, that is, the certainty that it should occupy these states. Importantly, our Simulation 2 also shows that the decrease in entropy over which state to occupy, and the increase of associated precision, are magnified when Agent 1 is in a more severe state of need (i.e., when the costs of the nonrewarded states are increased) and there is a path to the preferred state. As an example, if this path to the preferred state is signaled by a Pavlovian reward cue processed within the dopaminergic subsystem, then the wanting associated with that cue will be magnified. In other words, the more costly (surprising) these states are, the more the agent is certain that it needs to go to the preferred state. This illustrates how need states amplify the wanting (and/or the liking) of stimuli: by reducing entropy and making the agent more confident about what stimulus, response or course of action to select. Need states have cascading effects also on the stimuli and actions in the path toward goal or reward states. When in a severe need state, relevant stimuli, reward cues, and actions have a greater role in reducing entropy and increasing the confidence in the selected course of actions (Holmes, 2022; Parr et al., 2022). These relevant stimuli, reward cues, and actions are therefore assigned a greater value and a greater “need-generated” salience, which, within the wanting subsystem, would neurophysiologically correspond to increased dopaminergic activity that attribute higher incentive salience, that is, higher wanting, to reward cues (Berridge, 2004).

The proposed interplay of surprise, precision, and pervasiveness may offer a normative framework for creatures to navigate situations of competition between needs or between needs and other goals. For example, when faced with hunger and thirst simultaneously, the decision may hinge on the degree of deviation (surprise) or the intensity of the need state. If hunger greatly outweighs thirst, seeking food is likely. In addition, the presence or absence of intermediary states (precision) fulfilling one need over the other plays a role. If food is more accessible than water, eating may be chosen; conversely, if water is more available, drinking may be pursued. Furthermore, pervasiveness factors into this choice: If pursuing one need inadvertently exacerbates the other more than vice versa, the latter may be prioritized because of pervasiveness, as it would “translate” its impact and make it more noticeable. For instance, if obtaining food increases thirst significantly because of exertion, whereas getting water only affects hunger minimally, opting for water first may be preferable, because thirst would become more “felt” when trying to resolve hunger. Similarly, when a non-need goal is in competition with a need state, the decision process considers need level, precision, and pervasiveness. Strong needs and high pervasiveness, as well as a path to the preferred state (precision), tend to amplify the difficulty of bypassing the need. For instance, mild hunger in winter may not deter meeting someone outside, but a cold-related need state will make the decision to meet outside more challenging as the need will increase and be pervasive, even more so if there is a warm place where one could meet. All in all, behavior balances goals, reflexes, and sensory inputs to control one's state in the environment (Cisek, 2022).

One advantage of using active inference is its natural incorporation of the pervasiveness of need states through the concept of prior preferences. Conversely, the choice of a reinforcement learning model is motivated by the ability to capture the independent nature of reward prediction and needing. In our study, we used a relatively simple TD model, which may limit the interpretation of our results. Future studies could broaden the results, by considering more advanced reinforcement learning models, such as those that consider more directly homeostatic needs and the notion of progress to goal (Juechems & Summerfield, 2019; Keramati & Gutkin, 2014).

In summary, our simulations show that our model using active inference provides a natural way to model the needing as its own system. Traditional models that discuss need-related motivation often assume an automatic link between needing and reward prediction (Keramati & Gutkin, 2014; also see Berridge, 2023). The proposed model presents a more nuanced view, acknowledging the association and dissociation of needing with other subsystems, such as the wanting subsystem, that process external cues. This broader perspective has the potential to explain a wider range of experimental findings, both at the neural and behavioral levels. Indeed, the increase in precision, which, in active inference, is interpreted as neuronal gain (Friston, 2010), could possibly be viewed as the (need induced) salience conferred by need states to relevant stimuli (see Chen, Papies, & Barsalou, 2016). In this sense, if we interpret the increase in precision (in our results) as a need-induced neuronal gain toward stimuli/events that are treated within some subsystem, it becomes clear how needing can function independently while interacting with other subsystems like wanting, liking, and interoception. For instance, needing can increase precision of stimuli/events treated within other subsystems, such as the liking or interoceptive ones (Bosulu et al., 2022; Becker et al., 2019; Cabanac, 2017; Berridge & Kringelbach, 2015), even in absence of a Pavlovian cue that triggers the wanting subsystem (Salamone et al., 2018; Wassum et al., 2011), and this fits well with some recent meta-analytic results of human fMRI data distinguishing between needing and wanting (Bosulu et al., 2022). Conversely, a specific need can intensify wanting when a relevant Pavlovian reward cue is present, possibly through this increase in (policy) precision. Notably, this can occur through “active inference,” that is, without (re)learning (or liking) the consequence of that reward in that specific need state (Berridge, 2023; Zhang et al., 2009; Berridge, 2007). We acknowledge that although simulations illustrate the benefits and predictions of the proposed active inference framework of needing, empirical data testing is still required.

The current discussion focuses on how needs influence perception of the external environment and their impact on the wanting (and liking, etc.) associated with relevant cues. However, a pertinent question arises: Could external cues (e.g., wanting) potentially heighten the sense of needing? Indeed, research suggests that needing and wanting are distinct constructs (Bosulu et al., 2022; Berridge, 2004). Because of pervasiveness, need states persist with the creature regardless of its location or time. This differs from usual external rewards (and usual aversive states), which are tied to specific states, that is, places or times. Our definition of pervasiveness, however, means that from the point of view of the animal, certain forms of expectations or wanting may be perceived as needing, if they are sustained over time and impact other states. This would essentially happen for more complex creatures, for example, humans, that are capable of holding in their mind the cues that cause the desire/wanting. This article does not discuss cases in which wanting or expectations influence needing, and these could be covered in future research.

Regarding pervasiveness, it is worth noticing that the joint probability between a state and a need state does not depend on the level of need. Another perspective is that prior preferences, particularly those possibly genetically encoded, persist with the creature regardless of its location or time. Therefore, when a need state arises, the deviation from prior preference is felt regardless of the situation, proportionally to the pervasiveness. For the sake of simplicity, we did not fully consider how different need states might map to different levels of pervasiveness.

The proposed model of needing has potential to be extended to broader psychological phenomena in humans (Stussi & Pool, 2022; Pool, Sennwald, Delplanque, Brosch, & Sander, 2016; Maner, DeWall, Baumeister, & Schaller, 2007; Baumeister & Leary, 1995; Maslow, 1943). That being said, it is important to recognize that the term “need” can occasionally be interpreted as an expression of personal requirement, as in “I need more information.” There is a growing body of literature showing that animals show information-seeking and curiosity driven behavior (Gottlieb, Cohanpour, Li, Singletary, & Zabeh, 2020), and dopamine reflects information-seeking dynamics, not just reward prediction dynamics (Blanco-Ponzo, Akam, & Walton, 2024). Although this aspect lies beyond the scope of the current article, it presents an intriguing avenue for further investigation. This framework could also be applied to drug dependence/addiction, where a crucial question arises: Does drug consumption stem from needing, where the drug state becomes embedded as a prior preference over internal states (Turel & Bechara, 2016; O'Brien et al., 2006), or from wanting, driven by excessive dopamine sensitization to drug-related policies leading to the drug (Berridge & Robinson, 2016; O'Brien et al., 2006)? Depending on the case, different brain regions and behavioral therapy approaches may be targeted.

Overall, this study aimed to provide a conceptual model for needing and its interaction with reward subsystems, based on the active inference framework and the embodied pervasiveness hypothesis. However, further work is needed to fully clarify and empirically test the relationships between the abstract notions introduced here and their underlying biological substrates. A systematic mapping between the information-theoretic concepts used and neurobiological evidence remains an open objective for future research.

Agent 1 incorporates the main ideas and equations discussed in this article about needing and wanting systems. It is a simplified version of active inference, in which the perceptual part is kept as simple as possible, by assuming that all the states of the grid world are observable. Technically, this means that we are dealing with a Markov Decision Process and not a Partially Observable Markov Decision Process as more commonly done in active inference (see Friston et al., 2017; Friston, Daunizeau, & Kiebel, 2009). This simplifying assumption is motivated by the fact that our focus in this work is on action selection and not perceptual discrimination.

In Agent 1, the need state has a pervasive effect on all the states of the grid world (except the reward state) as described above. Following active inference, this pervasive effect is reflected in the prior preferences for the states to occupy (denoted as C, to follow the usual notation of active inference in discrete time), which is greater for the rewarding state than for any other state. The prior preferences followed a normal distribution centered on the value of the preferred state with a degree of sensitivity regarding deviations from that centered value. The center, that is, prior preference, is the mean; the sensitivity is the variance; and their values were 1 and 0.5, respectively. (Note that the agent may have behaved differently if the variance/sensitivity was different.) This followed the logic of the “homeostatic reinforcement learning” model of Keramati and Gutkin (2014), which is based on minimizing the sum of discounted deviations from a setpoint.

To model the pervasive effect of the need state, we assume a joint probability of 1 between the (increasing) need state and other states, except the preferred/satiety state, which had a joint probability of 0 with the need state. (Please note that the behavior of the agent may have been different if the joint probability with the other states was significantly less than one.)

Given the pervasiveness of the need state, almost all of the y_i acquire the negative valence and their conditional probability based on prior preference (of satiety) decreases, because their joint probability, that is, co-occurrence, with satiety decreases. Hence, only the state (or group of states) on the path to the preferred state has high probability as it co-occurs with the prior preference.

Because the agent expects to occupy (or to move toward) these a priori probable states, the prior over states also translates into priors over actions or action sequences (policies) that achieve such states. In this simplified setting, action (and policy) selection simply corresponds to inferring a distribution of states that it prefers to occupy and policies to reach (sequences of) those states. In other words, Agent 1 tends to select policies that lead it to achieve goal states—which in Bayesian terms corresponds to maximizing model evidence (Parr et al., 2022).

We should keep in mind that Agent 2 is used only to illustrate prediction in its simplest form and does not include advanced reinforcement learning techniques.

We thank Kent C. Berridge for providing valuable insights and feedback during the development of this article.

Corresponding author: Juvenal Bosulu, University of Pennsylvania, Richards Medical Research, Laboratories, Philadelphia, PA, or via e-mail: [email protected].

All data are available upon request.

Juvenal Bosulu: Conceptualization; Data curation; Methodology; Visualization; Writing—Review & editing. Giovanni Pezzulo: Writing—Review & editing. Sébastien Hétu: Funding acquisition; Writing—Review & editing.

The research was supported in part by NSERC Discovery grant #RGPIN-2018-05698 to S. H. and UdeM institutional funds to J. B. and S. H. This research is supported by funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the specific grant agreement no. 945539 (Human Brain Project SGA3) to G. P. and no. 952215 (TAILOR) to G. P., and the European Research Council under the grant agreement no. 820213 (ThinkAhead) to G. P.

Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.