Need states are internal states that arise from deprivation of crucial biological stimuli. They direct motivation, independently of external learning. Despite their separate origin, they interact with reward processing systems that respond to external stimuli. This article aims to illuminate the functioning of the needing system through the lens of active inference, a framework for understanding brain and cognition. We propose that need states exert a pervasive influence on the organism, which in active inference terms translates to a “pervasive surprise”—a measure of the distance from the organism's preferred state. Crucially, we define needing as an active inference process that seeks to reduce this pervasive surprise. Through a series of simulations, we demonstrate that our proposal successfully captures key aspects of the phenomenology and neurobiology of needing. We show that as need states increase, the tendency to occupy preferred states strengthens, independently of external reward prediction. Furthermore, need states increase the precision of states (stimuli and actions) leading to preferred states, suggesting their ability to amplify the value of reward cues and rewards themselves. Collectively, our model and simulations provide valuable insights into the directional and underlying influence of need states, revealing how this influence amplifies the wanting or liking associated with relevant stimuli.

Our bodies preserve their stability against the many disturbing forces through homeostasis and its mechanisms (Cannon, 1939). Among the mechanisms of homeostasis and its adaptive form, allostasis (Sterling & Eyer, 1988), there is a particular one that we will refer to as “needing.” Generally, needs intensify and exert a pervasive influence on various aspects and dimensions of our lives. For example, when experiencing hunger, its effect persists until satisfied by food. This principle extends to other needs such as thirst and sleep. Therefore, states of need become pervasive, shaping our perceptions, decisions, and more.

Needing is a process related to internal states (Livneh et al., 2020; Craig, 2003) characterized by a deprivation of essential elements crucial for life or survival (Bouton, 2016; Baumeister & Leary, 1995; MacGregor, 1960). Here, we refer to those (internal) states as need states. Such states (through the needing mechanism) have a directional effect on motivation (Bosulu et al., 2022; Dickinson & Balleine, 1994; Balleine, 1992). Needing also interacts with other subsystems that process external stimuli, such as wanting, liking, or interception. Thus, needing is both separate from (e.g., see Watson, Wiers, Hommel, & de Wit, 2014; Hogarth & Chase, 2011), but also interacts with (see Berridge, 2004), different reward-related subsystems. This happens in an independent way; for instance, need states tend to amplify the incentive salience of relevant Pavlovian cues that generate “wanting” (Berridge, 2004; Toates, 1994) independently of “liking” and (re)learning (Berridge, 2012, 2023). Yet needing can, independently of wanting (i.e., of Pavlovian cues) amplify “liking” or pleasure (Becker et al., 2019; Cabanac, 2017; Berridge & Kringelbach, 2015) and influence learning (Salamone, Correa, Yang, Rotolo, & Presby, 2018; Wassum, Ostlund, Balleine, & Maidment, 2011; Dickinson & Balleine, 1994; Balleine, 1992) and interoceptive prediction (Bosulu et al., 2022; Livneh et al., 2020), and needing can also directly activate relevant actions (Passingham & Wise, 2012) or explorative behavior (Panksepp, 2004) as well as behavior related to autonomic and neuroendocrine levels (Swanson, 2000).

Being related to internal states (Sterling & Laughlin, 2015), needing generates the value attributed to rewards “from within” and is thus separate from the external prediction of reward. Indeed, need states can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation/wanting (Berridge, 2012, 2023). In the same sense, needing can also modify the perception (e.g., the pleasure) of relevant stimuli independently of their original sensory valence (Cabanac, 1971, 2017). Water tends to acquire a pleasing taste when one is thirsty, and a warm environment or object feels comforting when experiencing cold. Conversely, when too hot, seeking out a cool place can be equally satisfying. These instances highlight how the directional impact of needing plays a crucial role in shaping motivation, ultimately influencing what is perceived as rewarding within a given need state. This directional influence of needing has the ability to alter the perception of pertinent stimuli, irrespective of the predicted or “actual” value of the reward.

Nevertheless, a comprehensive formal framework that encompasses the aforementioned findings and specifically elucidates the following aspects remains elusive: (1) the mechanism by which needing steers motivation, independently of the external world, and (2) the intricate interplay between needing and external rewards processed within other subsystems, such as wanting associated with relevant Pavlovian cues, or liking linked to relevant hedonic sensation. In the following sections, we will first tackle these points/questions conceptually, using formal methodologies from active inference (Parr, Pezzulo, & Friston, 2022) and introduce the concept of pervasiveness of a need state. Subsequently, we will present three simulations that delve into the functioning of the needing system and its interactions with wanting (and liking).

In the next two sections, we present a conceptual perspective aiming to elucidate two fundamental aspects of needing: (1) its directional impact and (2) its interplay with other subsystems.

The Directional Effect of Needing

To Remain within Their Physiological Boundaries, Organisms Are Endowed with A Priori Preferred States That They Tend to Occupy

A fundamental objective of organisms is to regulate and maintain their internal states within specific narrow limits (Barrett, 2017; Sterling & Laughlin, 2015; Friston, Kilner, & Harrison, 2006; Cannon, 1939). For instance, the average normal body temperature for humans is generally between 36.1°C (97°F) and 37.2°C (99°F), which is a very small range compared with the range of possible temperatures in the universe, from the absolute zero to trillions of degrees. The same is true for levels of glucose or the balance between water and salt in the body. The underlying principle is that the spectrum of “states” conducive to life is exceedingly limited in contrast to the immensely large number of alternative combinations that would not sustain life. Therefore, to ensure a living organism remains within its normal physiological boundaries, natural evolution might have established these boundaries as innate preferred states—possibly encoded genetically—that the organism consistently endeavors to attain. From the formal standpoint of active inference, these preferred states (which might correspond, for instance, to physiological bounds) are referred to as (empirical) priors. They carry a higher probability of realization from the organism's perspective, meaning they are less surprising (Friston, 2010).

Not Being within Prior Preferred States Is “Surprising” and Not Having a Path (i.e., a Relevant Reward) to Get Back to the Preferred State Creates Entropy

Here, the surprise associated with a state, denoted h(y), is the “negative” of being probable and simply means less probable. In addition, this surprise is not cognitive as the word “surprise” is commonly used. Anecdotally, for a fish, being out of water would count as a surprising state, as would a very thirsty human. A key claim of active inference is that any self-organizing system must minimize such surprise to resist a natural tendency to disorder (Friston, 2010; Friston, Kilner, & Harrison, 2006) and, in the case of our fish, death. Formally, the notion of surprise is closely related to the notion of entropy. Entropy, denoted as H, is the long-term average of the surprise, and here, it expresses the uncertainty related to which state must be occupied. If an organism is endowed with priors about the (preferred) states to occupy, these states have a high prior probability to be occupied from the perspective of the organisms and achieving them reduces the organism's surprise and its long-term average: entropy (Parr et al., 2022; Friston, 2010). Hence, there is a negative correlation between the availability of a path to the preferred state (i.e., rewards, with their related cues and actions, that lead to the preferred states) and the need-induced entropy, because of the prior tendency to visit (more often) those states, which results in lower entropy. Importantly, the notion of being in a surprising state (or in other words, being far from preferred states) links well to the concept of “needing” discussed in the Introduction section. In the same way being in a surprising state entails an (informational) cost, a state of need entails a (biological) cost if a creature does not respond to the need (see Baumeister & Leary, 1995; MacGregor, 1960). When a living organism moves away from its preferred state, it is in a state of “need”—which is related to a tendency to occupy (again) the preferred states (which we refer to as “needing”). Formally, if
(1)
represents the probability that a state (y) should occur, given prior preferences (denoted as C), then a need state can be represented as:
(2)
where hn represents the “need-related” surprise of a sensation or state y, which is equal to the negative log probability of being in (or observing) a state Y given the distribution of prior preferences, C. Note that for simplicity, in this article, we will collapse the notions of “state” and of “observation that can be obtained in the state,” which are typically distinct in active inference and, more broadly, in Partially Observable Markov Decision Processes.

Pervasiveness of Need States: Need States Are Pervasive Over Time and to Other States, Except to the One (or Few) State That Alleviates That Need

Pervasiveness is the key hypothesis that relates “needing” to prior preferences over states to occupy as well as to the surprise and entropy of not being in such states. By “pervasiveness,” we refer to the potential increase, “as time goes,” in a need state's impact on “other states,” that is, other dimensions of life, unless one transitions to the (only) state that satisfies that need. Formally, we represent the pervasiveness of the need state, or the fact that the need state propagates a negative valence to any other state i of the organism, in the following way:
(3)
where yi is any state, yn is the need state, and p(yi, yn) is the joint occurrence (conjoint probability) of the ith state and the need state. Looking at Equation 1, it becomes clear that the surprise, noted as −ln[p(yi|Cp)], gets greater for almost all states. So, when the organism is in a need state, all states eventually become surprising, that is, paired with negative valence, because they can jointly occur with the need state. The only exception is the rewarding state (and the fewer states that lead to it) that alleviates that specific need, because the probability that it jointly occurs with the need state is zero (or close to zero, eventually). This allows animals to follow a gradient of surprise minimization. Practically speaking, if one is in a need state, for example, a state of “hunger,” this need state would propagate a negative valence and (need related) surprise to all the other states (e.g., the states of “sleep,” “run,” “play”). The only exception to this is the state “having food in the stomach.” Hence, pervasiveness increases entropy (the long-term average surprise) because almost all states become surprising when one is in a need state. The concept of pervasiveness is illustrated in the figure below.
Figure 1.

The impact of pervasiveness of a need state on other states. The expansion of the red states illustrates the pervasiveness effect. (A) There is no pervasiveness and hunger is more like an aversive state (State 8). (B) Pervasiveness occurs and hunger impacts all other states, except the (preferred) state that alleviates it. This increases preference for stimuli/events or actions that lead to the preferred state because there is no other choice. By doing so, it increases the precision (neuronal gain) or weight assigned to relevant stimuli/events or actions within the subsystems that process them.

Figure 1.

The impact of pervasiveness of a need state on other states. The expansion of the red states illustrates the pervasiveness effect. (A) There is no pervasiveness and hunger is more like an aversive state (State 8). (B) Pervasiveness occurs and hunger impacts all other states, except the (preferred) state that alleviates it. This increases preference for stimuli/events or actions that lead to the preferred state because there is no other choice. By doing so, it increases the precision (neuronal gain) or weight assigned to relevant stimuli/events or actions within the subsystems that process them.

Close modal

Needing Induces a Tendency to Transition, From States to States, Toward the Preferred State That Alleviates It

The perception of a need state translates into a “goal” of reducing surprise by reaching the preferred states, for example, states that represent adaptive physiological conditions (Friston, 2010). Such a tendency could activate an action or a policy (i.e., a sequence of actions) that compels creatures to seek out the (valuable) preferred states. Note that the actions or policies that resolve a state of need could in some cases correspond to (fixed) regulatory actions, such as autonomic reflexes, as opposed to action courses determined by the circumstances of the external environment (Sajid, Ball, Parr, & Friston, 2021). With time, the states that the creature occupies when pursuing a policy that resolves a need can become valued per se (Friston & Ao, 2012). In other words, when the creature pursues a course of actions toward the preferred state, all the intermediate states (here intended in a broad sense that encompasses situations, actions, stimuli, etc.) can become valued and needed, through a Pavlovian mechanism (Berridge, 2018; Bouton, 2016). For instance, when moving from a state of hunger to a state of satiety, some intermediary states, such as the gustatory stimulus associated to having food and the act of eating, would become valued, because they are in the path toward the preferred (satiety) state (Pezzulo, Rigoli, & Friston, 2015). Because of its connection with prior preferences, possibly genetically encoded, which may or may not be cognitive, needing influences both reflexive and cognitive (higher level) goals, like shivering for warmth or deciding to buy winter clothing, extending an organism's control over its states (Cisek, 2022). In summary, a creature that is far from preferred states would experience a need (e.g., for food)—and then start to prefer the valued states (here, intended in a broad sense that encompasses stimuli, actions, etc.) that secure the relevant reward (e.g., food) or are part of the experience of having food (e.g., food cues or relevant actions). In this sense, the tendency to occupy preferred states confers to need states the possibility to influence—and give value to—stimuli or actions that are either costly states (noted S(hn)) that lead to surprise, or in the path toward the preferred state (noted π(p)). In other words, when one experiences needing, any state (stimulus or action) that is in the path to the preferred state will become something one needs (and hence valued) because it reduces need-related entropy. Hence, the directional effect of need states on motivation could come from the tendency to occupy these preferred states.

Needing Is an (Active) Inference Process That Aims at Reducing Pervasive Surprise

A need state could be defined as a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is surprising with regard to prior preferences. Needing can then be defined as an active inference process that aims at reducing such pervasive surprise by inducing a tendency to transition, from states to states, toward the preferred state. The effect of needing does not have to be learned anew, but it is “actively inferred”—reflecting the fact that need states (hunger, thirst) and needed stimuli (food, water) are related to fundamental priors that are potentially sculpted by natural selection. Indeed, even very simple creatures that have limited (or no) learning capacities are able to navigate their environment adaptively to fulfill their current needs, for example, by following food gradients (Sterling & Laughlin, 2015). The implications of this definition of needing is that it allows shifts in need states to direct motivation toward relevant stimuli even without learning, for instance through alliesthesia (the natural change in sensation/perception of a relevant stimulus induced by a need state). A striking illustration of this phenomenon occurs when animals would suddenly have an increase in “wanting” associated with a salt cue, when depleted of salt even in absence of (re)learning (Berridge, 2012, 2023). This happens even when that cue was learned to be predictive of negative outcome and without the animals having (re)learned about the cue (or the salt) through tasting (and liking) it in the newly induced need (salt depleted) state. Our article provides a plausible explanation through the lenses of active inference and pervasiveness. The switching from learned negative outcome to wanting happens through pervasiveness, which progate surprise to all other states (stimuli, cues, goals, etc.) except the states on the path to the priori preferred state. Notably, this category encompasses the (memories of) salt and its cue, as these stimuli still co-occurred with salt in the body, in a multisensory-like representation, despite being learned to be negative (Smith & Read, 2022). Thus by reducing such pervasive surprise through an active inference process, the animal would tend to have an increase in wanting associated with such salt cues without (re)learning. This adjustment is achieved through precision increase as elaborated below.

Interaction between Needing and Other Subsystems

Needing Modifies Perception of Rewarding States Processed within Subsystems through Increase in Precision: Such Precision Is Signaled by Neurotransmitters of Each Subsystems

The effect of needing on wanting (and on other phenomena such as pleasure and liking) could be conceptualized by appealing to the formal notion of precision in active inference. Mathematically, precision is a term used to express the inverse of the variance of a distribution, which in our context can be seen (loosely speaking) as the inverse of entropy (Holmes, 2022; Friston, 2010)—in the sense that the higher the entropy, the lower the precision. In predictive coding and active inference, precision acts as a multiplicative weight on prediction errors: Prediction errors that are considered more precise have a greater impact on neural computations (Parr et al., 2022).

With respect to need states, precision can be interpreted as a higher salience attributed to the most relevant stimulus given the need state. There are different precisions associated with different subsystems, such as those related to interoceptive streams, rewards, policies, and so forth (Parr et al., 2022). At the neurophysiological level, policy precision, or the confidence that a policy should be followed, is typically associated with the dopaminergic subsystem in active inference (Holmes, 2022; Parr et al., 2022; FitzGerald, Dolan, & Friston, 2015). Therefore, Pavlovian cues that enhance policy precision and confidence that the policy should be followed would trigger dopamine bursts, which will attribute incentive salience, that is, wanting, to such cues (Berridge, 1996, 2007). This is in line with the idea that dopamine is linked with incentive salience and wanting, but also with reward cues and behavioral activation as they typically co-occur (Hamid et al., 2016, but see Berridge, 2023). Rather, precisions regarding hedonic contact with the reward (to ask questions such as: Is it good?) or the state of satiety (to ask questions such as: Am I well?) might be mediated by the opioid subsystem (Berridge & Kringelbach, 2015) and the serotonin subsystem (Parr et al., 2022; Liu, Lin, & Luo, 2020; Luo, Li, & Zhong, 2016). The impact of needing on stimuli that are processed within these subsystems are “natural” increases in precision because of prior preferences. The need-induced increase in precision implies more certainty that the state to which the stimulus or policy leads to is the least surprising. It is this certainty that amplifies wanting, liking, and so forth, and it is an active inference process that may be separated from learned reward prediction (also see Berridge, 2023).

This discussion helps appreciate the deep interdependence between needing (which acts on the system as a whole) and wanting as well as other subsystems such as the hedonic/liking and interoceptive ones. When one is in a surprising (need) state, the presence of a cue (e.g., a traffic or restaurant sign) might reduce uncertainty about goal/reward achievement by improving policy precision via the dopamine subsystem (wanting). The presence of, or contact with, the reward itself might reduce entropy by enhancing precision through the opioid subsystem (pleasure/liking). Finally, being in a preferred state or moving toward the preferred state determines an increase of precision—which is because of the fulfillment of prior preferences—via the serotonin subsystem (well-being).

In all these cases, rewards and cues are useful information sources: They reduce entropy by signaling the availability of a path to preferred states (π(p)), or equivalently a path away from surprising states (S(hn)), given some prior preference (Cp). Indeed, from the point of view of the organism in a need state, both encountering either a relevant reward or a cue that leads to that reward would reduce uncertainty about how to move to the preferred state and alleviate the need. This is despite making contact with a reward and encountering a cue that predicts reward might be treated in different subsystems: the liking and the wanting subsystems, respectively.

Summary

Our discussion so far has highlighted two crucial aspects of needing and its interactions. First, need states exert a directional influence on choices separately from reward prediction. This is because of the pervasiveness of need states that make states surprising with regard to prior preferences and to the animals' tendency to reduce surprise, which naturally increases the value of rewards, cues, and actions that lead to the preferred state. This tendency exists irrespective of reward prediction (Berridge, 2023; Smith & Read, 2022; Zhang, Berridge, Tindell, Smith, & Aldridge, 2009). Second, when there is a path to the preferred state, such as a reward or a Pavlovian cue, needing would increase the value of reward or Pavlovian cues within the subsystems that process them. This translates into a lowering of entropy about which state to occupy (i.e., entropy of the probability distribution of the to-be-reached states) to transition to the preferred state, and thus an increase in the precision of relevant stimuli (e.g., liking) or goal-achieving policies (e.g., wanting). Such high precision could be viewed as need-induced salience, which, for instance, in the wanting subsystem translates into higher incentive salience. With these insights in mind, we now shift from a conceptual discourse to the practical implementation of needing through an active inference framework.

In the following section, we present the mathematical formulation by introducing an agent endowed with needing. This agent is utilized in simulations to illustrate the functionality of the needing system and its influence on reward subsystems, such as wanting and liking. In this section, we adopt a parallel structure to that of section Needing System: A Conceptual Perspective. Concerning the directional impact of needing (section The Directional Effect of Needing), we delve into mathematical specifics of the subsections (from sections To Remain Within Their Physiological Boundaries, Organisms are Endowed with Apriori Preferred States That They Tend to Occupy to Needing Is an (Active) Inference Process That Aims at Reducing Pervasive Surprise). In addition, with respect to the interaction between needing and other subsystems (section Interaction between Needing and Other Subsystems), we explore the mathematical intricacies associated with both the need-related entropy, and need-induced precision increase. We then map the mathematical formulation to the conceptual perspective in sections The General and Directional Motivation of Needing and Interaction between Needing and Other Subsystems Such as Wanting, Liking, and So Forth.

The Mathematical Details of the Directional Effect of Needing

The Probability of a State Given Prior Preferences

Animals possess inherent prior preferences that condition the probability of what specific states “should” occur, noted as:
(4)
which means the probability of observing a state Y (remember that in our setting, hidden states and observations are the same) given prior preferences C (i.e., biological costs or rewards). This equation represents the probability of a state occurring as dictated by the agent's nature or phenotype, which we categorize as prior preferences. For instance, the probability of a “living human” having a temperature significantly below or above the range between 36.1°C (97°F) and 37.2°C (99°F), or having salt/water levels beyond certain limits, is low given the model or phenotype that they are. Here, the states are fully embodied, encompassing the brain, body, and environment as one and the same context (see Cisek, 2022; Tschantz et al., 2022; Barrett & Simmons, 2015).

The Main Formulation of Need State as Surprise

A need state could be defined as a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is (biologically) surprising with regard to prior preferences. The general notation of such surprise is:
(5)
where ln denotes a natural logarithm and (P(Y|C) the probability of observing a state y given prior preference (C). A need state is less probable, that is, surprising, given prior preferences. Furthermore, because of pervasiveness, any other state that shares a joint probability with that need state becomes (proportionally to their joint probability) surprising, as described below.

Formulation for the Pervasiveness

The conditional probability of the states Y given prior preference will decrease proportionally to the amount of need (n) they embed, and this decrease in probability will increase the surprise of being in those states. Because of the pervasive effect of a need state, the value of any state yi with regard to (the amount of) need (noted yi,n) that one has while being in that state depends on its co-occurrence, that is, joint probability, with the state that generated the need, noted yn.

Thus, any state yi will have a need equivalent to:
(6)
The states that are affected by the need (n) will become elements of the set of surprising states, noted S(hn). Those that are not will be elements of set of states on the path to the preferred state, noted π(p). This (i.e., Equation 5) is to be considered over time as states on the path to the preferred state (such as relevant reward cues) can for a few moments co-occur with that need, but a few moments later, that need state would disappear (as they led to the preferred state). Hence, their joint probability with the need state, taking time into account, would still be lower.
We can then note the need-related surprise of each individual state as given the prior preference as:
(7)
where Cp,n represents the identity of the specific priori preferred state (denoted by the smaller p) that alleviates the specific need (denoted by n). The objective is to map each need state with its corresponding relevant preferred state for computational purposes.

Assuming that animals have prior preferences over levels of satiety/need (e.g., a preferred level of sugar in the body), the prior preference probability over satiety is equal to p(Cp,n) = 1.

Thus, the preference for each state i given the prior preference and the need becomes their joint probability:
(8)
Thus our Equation 6 can, under those conditions, be written:
(9)
This illustrates that the co-occurrence with a need state is inversely related to co-occurrence with the preferred state. Thus, for a given need, the need-related surprise of each state is the negative logarithm of its joint probability with the preferred state.

Given the pervasiveness of the need state, almost all of the yi become surprising because their joint probability, that is, co-occurrence, with the preferred state decreases. Hence, only the states “food in stomach” and the states that lead to it have high probability as they (eventually) co-occur with the priori preferred state. Crucially, as it will become clear in our first simulation, this probability increases as need increases, because the sum of probability of p(yi,n|Cp,n) has to sum to 1.

Formulation of Surprise Minimization

Because the creature expects to occupy (or to move toward) these a priori probable states, the prior over states also translates into priors over actions or action sequences (policies) that achieve such states. In this simplified setting, action (and policy) selection simply corresponds to inferring a distribution of states that it prefers to occupy and policies to reach (sequences of) these states. In other words, the active inference agent tends to select policies that lead it to achieve goal states—which in Bayesian terms correspond to maximizing model evidence (Parr et al., 2022).

More formally, under the simplifying assumptions discussed above, the creature strives to maximize a measure of (log) evidence, or alternatively minimize the associated surprise −ln[P(yi,n|Cp,n)]. The creature does so by minimizing the expected surprise of every state, given the prior preferences, as denoted below:
(10)
The EQ(y|π) part means that the probability of states/outcomes is averaged across all policies/paths (π). In our framework, E represents the expectation over all reachable states under the path or policy. Essentially, the equation minimizes surprise by favoring states more probable under prior preference and more reachable along the path. In this sense, Q is a distribution representing the probability of a state under the path or policy. The term “path” or “policy” can denote a single action, state, or a sequence of states or of behaviors like reflexive, sensory, evaluative, or planning responses, depending on the situation and the creature's needs and abilities.
In active inference, the quantity shown in Equation 10EQ(y|π) ln[P(y|C)] (without the minus sign) is typically called a “pragmatic value,” and in this setting (with a minus sign), it corresponds to the expected free energy G(π) (an upper bound on expected surprise):
(11)
For completeness, it is important to consider that the quantity shown in Equation 11. (without the minus sign—the “pragmatic value”—is only one of the two terms of the expected free energy G(π) of active inference; however, in our setting, there is no ambiguity and the second term (“epistemic value”) is zero.

In this context, yi,n represents the need state, whereas minimizing G(π) denotes needing, the process facilitating the fulfillment/satisfaction of the need state. This later process is contingent upon the path or policy leading to the preferred state (while both yi,n and G(π) depend on prior preference). For instance, hunger on its own represents a need state where one is far from the preferred state of satiety. When there is a path to that preferred state, the tendency within different subsystems to value states along that path (e.g., food cues, taste, interoception) is what we describe as “needing.”

The expected free energy G(π) is particularly important since it is used for policy selection. Specifically, active inference agents are equipped with a prior over policies, denoted as P(π). The greater the expected free energy that policies are expected to minimize in the future, the greater their prior, that is,
(12)
where σ represents the softmax function, bounded between 0 and 1, and enforces normalization (i.e., ensures that the probability over policies sums to one). In simple terms, the equation “values” the path/policy (π) based on their “ability” to generate the states (i.e., y∣π) that minimize surprise (i.e., yC), and the goal is to choose the path/policy (π) that has the highest “value,” that is, minimal free energy.

Mathematical Definition of Needing from an Active Inference Perspective

Needing is thus the minimization of G(π), or to expressing this in a “positive” way, needing is the maximization of EQ(y|π) ln[P(yi,n|Cp,n)]. In other words, needing induces a tendency to seek states in which the expected log-probability given the prior preference is maximized under that specific (pervasive) need state.

Mathematical Detail of the Interaction of Needing and Other Subsystems

Formulation of Precision Increase

Now, we consider the effect of a need state on (1) the entropy over the states that a creature plans to occupy in the future by following its inferred policy and (2) the inverse of the entropy, that is, the precision, which is a measure of certainty about which states to occupy next.

The “need-related entropy” (noted Hn or Hn,p, explained further below) is the average, that is, the expectation (E), of all negative log probability of all states Y given prior preference under a specific need, and is given by the formula:
(13)
As discussed, given pervasiveness (i.e., joint occurrence with the need state), this variable Y can indeed be a surprising state, that is, embedded with need states, or it can be a rewarding state, that is, a state on the path to the preferred state. Thus, the creature's “need related entropy” (or simply entropy) as follows:
(14)
when there is no path to the preferred state, that is, no reward (in a broad sense); and
(15)
when the cue/reward is present and the creature has a potential path toward the preferred state.

Here, H (i.e., Hn or Hn,p) denotes the entropy and it can be calculated on two sets of states. When a reward (or its cue) state is available, the entropy is over which states to occupy by the creature when some of the states are on a path π(p) to the preferred state (p) whereas the rest of states are not (S(hn)). Alternatively, when there is no reward, then the entropy is over which states S(hn) to occupy.

Thus, Y = S(hn) means Y is any S(hn), and Y = S(hn), π(p) simply means Y can be any S(hn) or any π(p). The π(p) and S(hn) represent states in different subsets of prior preferences, with the π(p) representing states that are on the path to the preferred state. These can be viewed as rewarding (or cues) states or events that lead (transition) to the preferred state if one follows a policy leading to the preferred state (discussed in section The General and Directional Motivation of Needing). The S(hn) represent the states that lead to surprise as they are (eventually) embedded with the need. This formulation highlights that the (need-related) entropy of an agent that faces costly/surprising states S(hn) is reduced when there is a path toward the preferred state π(p). Thus, the inequality below holds:
(16)
We can calculate the precision as the inverse of the entropy:
(17)
when there is no reward (cue) state, and
(18)
when there is a reward (or its related cue or action) state and hence a path to the preferred state.
Given the inequality in Equation 16, when many states become more surprising in terms of need, that is, biologically costly, the precision increases, providing that there is a path toward the preferred state, which implies that:
(19)
Given that we are discussing the motivational, that is, active part, here, entropy means (average) uncertainty over which state to occupy rather than uncertainty over what state is. Similarly, precision means certainty over what state to occupy. The principle is the same whether applied to what states to occupy or what policy to follow. The idea is to make it general so it can apply to incentive salience (wanting subsystem) or to hedonic sensation (liking subsystem), and also to simpler organisms that might not have a sophisticated brain.

In the specific context of the interaction between needing and “wanting,” we can draw a connection between our precision equation and the computational models pertaining to wanting and incentive salience proposed by Smith and Read (2022) and Zhang and colleagues (2009). In both models, a variable denoted as “k” represents the impact of physiological and other brain states on the dopaminergic state. This “k” factor alters the reward value “r,” subsequently leading to an amplification of the reward cues, expressed as the function r_rt,k. When such a variable “k” is influenced by need states, the function r_rt,k becomes associated with precision over policies that guide the attainment of the preferred state, denoted here as Pn,p.

Illustrating the connection between needing and wanting (Figure 2).

Figure 2.

Illustration of the wanting subsystem, which depends on the Pavlovian cues, paired with some stimulus, noted rt, under the influence of different inputs that elevate dopaminergic state k, such that the overall effect is r_rt,k as described by Zhang and colleagues (2009) and Smith and Read (2022). If the input is the need state (in green), then r_rt,k becomes Pn,p and needing amplifies wanting.

Figure 2.

Illustration of the wanting subsystem, which depends on the Pavlovian cues, paired with some stimulus, noted rt, under the influence of different inputs that elevate dopaminergic state k, such that the overall effect is r_rt,k as described by Zhang and colleagues (2009) and Smith and Read (2022). If the input is the need state (in green), then r_rt,k becomes Pn,p and needing amplifies wanting.

Close modal

The General and Directional Motivation of Needing

A need state could be defined as a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is (biologically) surprising with regard to prior preferences. Such surprise is noted: hn(y) = −lnP(yi,n|Cp,n). The pervasiveness of need states, given into prior preferences, propagates such surprise depending on the joint probability between a need state yn and another state yi given by p(yi, yn) [n]. Prior preferences then “judges” how any state yi should be preferred given the preferred state in which that need is alleviated, noted Cp,n, and in which the system expects to be in, and this is given by: P(yi,n|Cp,n). Based on that, the pervasiveness of need states make many states less probable, that is, surprising, noted S(hn). Hence the system, expressing states in terms of surprise, that is, −ln P(yi,n|Cp,n), can “infer” the trajectory, that is, the couple of states which are noted π(p), that lead to the preferred state, by choosing states (actions, cues or rewards) from which one expects minimization of such surprise: −EQ(y|π) ln[P(yi,n|Cp,n)]. Needing is this active inference process that aims at reducing such pervasive surprise by seeking states that maximize EQ(y|π) ln[P(yi,n|Cp,n)], that is, states whose expected log-probability given the prior preference is maximized under that specific (pervasive) need state.

Interaction between Needing and Other Subsystems Such as Wanting, Liking, and So Forth

Need states can amplify wanting or liking and thus assign a high precision Pn,p to Pavlovian cues or hedonic contacts with rewards that lead to the preferred state. Indeed, by following policies that minimize −EQ(y|π) ln[P(yi,n|Cp,n)] toward the preferred state and away from surprising states S(hn), when there is a possibility/path to reach such preferred state, the states on that trajectory π(p) become more and more probable given the prior preference. This is because, because of pervasiveness, the states on π(p) have a lower joint probability with the need state, and this decreases entropy over which state to occupy. In other words when one is in need (surprising) state that becomes pervasive, and there is a path to the preferred state, the entropy Hn,p (Y = S(hn), π(p)) = Hn,p decreases by following the policies that minimize −EQ(y|π) ln[P(yi,n|Cp,n)]. Thus, the inverse of that entropy Hn,p1, that is, the precision (or neuronal gain) Pn,p assigned to states (stimuli/events/reward cues) π(p) that leads to the preferred state is increased. This increase in precision happens within any subsystem (wanting, liking, interoception, etc.) if such π(p) states (stimuli/events/reward cues) happen to be processed by that subsystem. So, the interaction between needing and, for instance, wanting happens when needing enhances mesolimbic dopamine reactivity, which assigns higher precision to Pavlovian cues that are relevant under the need state. The enhancement of dopamine reactivity amplifies/generates “wanting” associated with relevant Pavlovian cues by acting as neuronal gain expressed as Pn,p.

In the next sections, we subsequently illustrate the functioning of the model in two simulations, which exemplify how being in need states influences the tendency to reach preferred states independently of reward prediction (Simulation 1), and how the simultaneous presence of a state of need and the presence of a path to the preferred (reward or goal) state implies low entropy and high precision over which state to occupy (Simulation 2).

Simulation Environment

We designed a 3 × 3 grid world with nine states, in which only one state (State 2) is rewarding/preferred, one is a need state (State 8), that is, it is (biologically) costly/surprising, and the other seven states are “neutral”; see Figures 1 and 3. We can draw a parallel between the State 2 and 8 and human physiological states, such as hunger or temperature: The preferred state (2) corresponds to the optimal interval of sugar level in the bloodstream, or the temperature range (between 36.1°C [97°F] and 37.2°C [99°F]), and the State 8 is a deviation from that.

Figure 3.

Grid world environment used in our simulations. Each box represents one state in which the agent can be. These include seven neutral states (States 0, 1, 3, 4, 5, 6, 7), a reward state (State 2), and a costly state (State 8). The value of these states is initially unknown. The boxes represent different states, and the images on the left represent some increase in hunger. Please note that we put hunger “alone” to distinguish between the presence and absence of the pervasive effect. In this case, the pervasive effect of hunger would attain all the other states, except the State 2: food.

Figure 3.

Grid world environment used in our simulations. Each box represents one state in which the agent can be. These include seven neutral states (States 0, 1, 3, 4, 5, 6, 7), a reward state (State 2), and a costly state (State 8). The value of these states is initially unknown. The boxes represent different states, and the images on the left represent some increase in hunger. Please note that we put hunger “alone” to distinguish between the presence and absence of the pervasive effect. In this case, the pervasive effect of hunger would attain all the other states, except the State 2: food.

Close modal

In the grid world, each state corresponds to a box in Figure 3. In our simulations below, we will assume that the agents are in a need state (e.g., hunger) that can be low, mild, or severe. Accordingly, a negative reward of −1 (slightly hungry), −2 (hungry), or −5 (starving) is assigned to the “costly” state (State 8). State 2 represents a reward/preferred state that gives a reward of 1 and resolves the need state. Note that the agents that dwell in the simulated environment sense this biological reward/cost and will have to compute the expected biological costs, or values, by themselves. Of note, the grid world is used here for visual presentation purposes and the proposed framework can indeed be generalized beyond a grid world.

Simulation 1: Directional Aspect of Needing Separately from Reward Prediction

Our simulations will consider two agents (see the Appendix and the Appendix table). Agent 1 incorporates the main ideas discussed in this article about the needing system. It is a simplified active inference model that bases action selection on prior preferences about states and policies (Parr et al., 2022). Agent 2 is used as a comparison, to show that reward prediction alone is not sufficient to explain the directional effect of needing and its interplay with other subsystems such as wanting. It is a simplified reinforcement learning system that based action selection on learned action values computed by reward prediction (Sutton & Barto, 2018) and perceives a “need” without the pervasiveness of need states as described here. The goal of Simulation 1, illustrated below, is to assess the effects of increasing need states on the action selection mechanisms of the two agents.

The results illustrated in Figure 4 show that increasing the cost of the need state, when pervasiveness occurs, significantly increases the probability assigned to policies that reach the rewarding state in Agent 1. This is evident when considering that the probability increases from (about) 0.5, 0.8, and 1 in the three left rows. However, increasing the costs of the need state without need-related pervasiveness does not affect reward prediction as shown in Agent 2. This is evident when considering that the softmax of state-action (Q) values assigned by Agent 2 to the rewarding state is always relatively the same in the three right rows. These results help illustrate the idea that costly or need states might exert directional effects and impact on the probability (or tendency) to reach preferred states through pervasiveness, irrespective of reward prediction.

Figure 4.

Effects of biological needs on policy selection under pervasiveness and prior preference (Agent 1, left columns) versus computed reward prediction (Agent 2, right columns). The left and right columns show the results for Agent 1 and Agent 2, respectively. For Agent 1, which uses prior preferences, the y axis plots priors over policies P(π) to reach each of the states of the grid world, whereas for Agent 2, which computes reward predictions, the y axis plots the softmax of the maximal state-action (Q) values (of the action that reaches that state). As evident when looking at the green bars, both Agents 1 and 2 assign the greater probability to the policy (or action) that reaches the rewarding State 2. The three rows show the effects of setting the costly state (State 8; see Figure 1) to −1, −2, and −5, respectively. The results show that when need states are pervasive, increasing biological needs (across the three rows) increases the probability that Agent 1 selects policies to reach the preferred State 2, but does not increase per se the (external) reward prediction probabilities assigned by Agent 2 to State 2. This can be apprehended by noticing that in the three columns, Agent 1 assigns different probabilities to the policies that reach State 2, whereas Agent 2 assigns (closed to) the same probability to State 2. This is consistent with the idea that need states are pervasive and this allows them to (directionally) influence tendencies (i.e., probabilities) toward the preferred state independently of reward prediction. See the main text for explanation.

Figure 4.

Effects of biological needs on policy selection under pervasiveness and prior preference (Agent 1, left columns) versus computed reward prediction (Agent 2, right columns). The left and right columns show the results for Agent 1 and Agent 2, respectively. For Agent 1, which uses prior preferences, the y axis plots priors over policies P(π) to reach each of the states of the grid world, whereas for Agent 2, which computes reward predictions, the y axis plots the softmax of the maximal state-action (Q) values (of the action that reaches that state). As evident when looking at the green bars, both Agents 1 and 2 assign the greater probability to the policy (or action) that reaches the rewarding State 2. The three rows show the effects of setting the costly state (State 8; see Figure 1) to −1, −2, and −5, respectively. The results show that when need states are pervasive, increasing biological needs (across the three rows) increases the probability that Agent 1 selects policies to reach the preferred State 2, but does not increase per se the (external) reward prediction probabilities assigned by Agent 2 to State 2. This can be apprehended by noticing that in the three columns, Agent 1 assigns different probabilities to the policies that reach State 2, whereas Agent 2 assigns (closed to) the same probability to State 2. This is consistent with the idea that need states are pervasive and this allows them to (directionally) influence tendencies (i.e., probabilities) toward the preferred state independently of reward prediction. See the main text for explanation.

Close modal

Simulation 2: How Needing Amplifies Wanting

In Simulation 2 (using the same grid-world environment), we ask if being in a state of greater need increases precision associated with rewards, that is, states that are on the path to the preferred state. We compare two conditions, namely, a condition when a reward is present (i.e., the reward State 2 is baited with a reward of 1) or absent (i.e., the reward State 2 has the same cost as all the other neutral states). The results of the simulations of entropy and precision can be appraised graphically in Figure 5. These results shown indicate that compared with the case with no reward, the condition where a reward is present implies a significant decrease of the entropy over which states the Agent 1 plans to occupy in the path to the reward (Figure 5, left) and a significant increase of its precision, which is a measure of certainty about which states to occupy in the future (Figure 5, right). This is because the availability of a reward makes the agent more confident about the states to occupy and the policy to select, whereas in the absence of a reward, all states are equally costly/surprising and the agent has no strong preference about which states to occupy (i.e., high entropy and low precision). The presence of a reward (which in this simulation is known by the agent) is a path that makes it possible to pursue a preferred course of action toward the preferred state, reducing the entropy about the states to occupy and increasing the certainty (precision) about the states to visit in the path toward the preferred state.

Figure 5.

The impact of different need states on the entropy (A) and on its inverse, the precision (B), over which state to occupy for an agent that embodies prior preferences, in conditions in which a reward is available (blue lines) or no reward is available (orange lines).

Figure 5.

The impact of different need states on the entropy (A) and on its inverse, the precision (B), over which state to occupy for an agent that embodies prior preferences, in conditions in which a reward is available (blue lines) or no reward is available (orange lines).

Close modal

Given the importance of more directly comparing our theoretical framework to empirical data and demonstrating how and why needing arises as pervasive surprise from prior preferences, we adjusted our parameters and devised a (third) simulation closely resembling Robinson and Berridge's (2013) experiment. They showed that needing can increase the wanting associated with a cue even if that cue predicts disgust sensation, demonstrating that needing can amplify wanting independently of prediction (Robinson & Berridge, 2013) also discussed in Berridge (2023).

The Original Experiment (Figure 5A)

Oral infusions of NaCl solution at Dead Sea concentration into the mouth of a rat typically elicit strong “disgust” reactions and Pavlovian cues that predict such stimuli quickly become repulsive and elicit retreat after a few pairings (Robinson & Berridge, 2013). In Robinson and Berridge (2013), each rat quickly learned to shrink away from the conditioned stimulus that was paired with Dead Sea salt—the CS + salt lever—whenever it appeared, retreating to a far wall as if trying to escape from the Pavlovian cue and its predicted salty infusion (Berridge, 2023). However, on a particular test day, the rats suddenly found themselves for the first time in their lives in a new state of physiological sodium deficiency through an injection of drugs that mimic the natural brain hormonal signals of salt appetite (Berridge, 2023). This was a never before encountered state because these (modern laboratory) rats, like most modern humans, have never experienced a salt appetite of that intensity: Their food, like ours, contains more than enough salt (Berridge, 2023). Crucially, on the test day, the rats re-encountered their Pavlovian CS + salt cue before ever experiencing (e.g., tasting) new positive hedonic value of Dead Sea saltiness as ‘liked’ (the liking part was tested later and was also shifted (see Robinson & Berridge, 2013)). They had only their past memories of Dead Sea disgust to guide their learned value of the CS + salt cue. However on the test day, before ever retasting the salty stimulus, the sodium-deficient rats immediately ran to their CS + salt lever as soon as it appeared, jumping onto and avidly nibbling the metal lever that had previously repulsed them (Berridge, 2023). The previously learned negative value of the salt cue was now discarded and the CS + salt cue instantly elicited positive wanting in their novel sodium-deficient state (Berridge, 2023; Robinson & Berridge, 2013), as shown in Figure 6A. This indicates that needing can, instantaneously, increase the wanting of a cue independently of learned prediction afforded by that cue, as we show in the simulation below.

Figure 6.

(A) The original Robinson and Berridge (2013) experiment, copied with permission from Berridge (2023), showing how novel need state (salt appetite/depletion) shifted the value and wanting associated with the Dead Sea salt, that is, the (previously) negatively valued stimulus/state. (B) The replicated (simulated) experiment. In this, the impact of a sudden unlearned need state on the valuation of a negative state, which becomes the sole means to reach the preferred state, was observed. Before the induction of the need state, both Agents 1 (“needing”) and 2 (“learned”) had a negative valuation for the state “Dead Sea salt” as can be seen on their last two decisions before “salt depletion.” However, immediately following the induction of the need state, only the needing agent successfully adjusted to the new value of the Dead Sea salt, whereas the learning agent continued to rely on its learned value. For an easier/intuitive visual comparison, we utilized the softmax function for both agents (employing negative free energy for Agent 1 and state-action value for Agent 2). Subsequently, we divided the results by the number of possible states (10) and converted them into logarithmic values. This transformation ensures that the values can be either negative or positive. By taking the logarithm of the probability divided by 10, we ensure that a state is assigned a positive value if it is more preferred or desired compared with some others, and a negative value if the opposite is true.

Figure 6.

(A) The original Robinson and Berridge (2013) experiment, copied with permission from Berridge (2023), showing how novel need state (salt appetite/depletion) shifted the value and wanting associated with the Dead Sea salt, that is, the (previously) negatively valued stimulus/state. (B) The replicated (simulated) experiment. In this, the impact of a sudden unlearned need state on the valuation of a negative state, which becomes the sole means to reach the preferred state, was observed. Before the induction of the need state, both Agents 1 (“needing”) and 2 (“learned”) had a negative valuation for the state “Dead Sea salt” as can be seen on their last two decisions before “salt depletion.” However, immediately following the induction of the need state, only the needing agent successfully adjusted to the new value of the Dead Sea salt, whereas the learning agent continued to rely on its learned value. For an easier/intuitive visual comparison, we utilized the softmax function for both agents (employing negative free energy for Agent 1 and state-action value for Agent 2). Subsequently, we divided the results by the number of possible states (10) and converted them into logarithmic values. This transformation ensures that the values can be either negative or positive. By taking the logarithm of the probability divided by 10, we ensure that a state is assigned a positive value if it is more preferred or desired compared with some others, and a negative value if the opposite is true.

Close modal

Replicated/Simulated Experiment (Figure 6B)

To replicate the Robinson and Berridge (2013) experiment, we conducted a “one-step decision” simulation. We created a simulation (similar to simulation one) where there was a negative state, the Dead Sea salt, which had a negative value, whereas all other states were positive. We compared an active inference agent (Agent 1) built with needing system, and an agent that learned through temporal difference (TD; Sutton & Barto, 2018) over hundreds of trials (Agent 2). Both agents (1 and 2) initially evaluated a state called “Dead Sea salt cue” negatively under satiety, similar to the conditioning step in the Robinson and Berridge (2013) experiment. Then, we tested them with a single-step decision, after inducing a need state assumed to be related to salt, where only the Dead Sea salt cue could satisfy the need. That is, only the Dead Sea salt cue would become positive. The “depletion test” was a single-step decision, directly after changing their state into a “need state.” As can be seen in Figure 6, on their last two decisions before “salt depletion,” both Agents 1 and 2 (i.e., under needing and under learned values) had a negative valuation for the state “Dead Sea salt” before induction of the need state. At the one-step immediately after need induction, Agent 1 shifted the value of the Dead Sea salt cue, similar to the findings of Robinson and Berridge (2013), whereas Agent 2 retained learned value in the one time-step decision, as shown in Figure 6B.

In this scenario, attributing positive value to the Dead Sea salt/cue does not depend on learning (not even the “incentive learning,” i.e., learning of the need-state/reward pair; see Balleine & Dickinson, 1998), because the agents or the rats have never experienced salt depletion, and came to “want” the Dead Sea salt/cue before ever (re)experiencing it in the salt-depleted state (Robinson & Berridge, 2013). Rather, the Dead Sea salt has a positive value because salt level is encoded as a prior preference. The depletion of salt (i.e., being far from prior preference) generates a pervasive need, and the Dead Sea salt is the only way to fulfill the prior preference—hence, it is sought after and valued. In other words, the sudden positive valuation of the Dead Sea salt reflects the needing process described in this article. In this sense, the sensory representation linking the Dead Sea salt cue to salt level becomes the path/policy under which the state (Dead Sea salt) that reduces pervasive surprise (deviation from salt level) is more probable. This translates to the increase in precision described in Simulation 2.

We do not imply that animals solely rely on needing and not learning; rather, they utilize both mechanisms. A needing-based agent lacking proper learning mechanisms may survive in a stable environment as a simple creature, but not as effectively as adaptive animals do in dynamic environments with numerous states and events. Needing and learning can indeed work together (for instance in the case of incentive learning; see Balleine & Dickinson, 1998). However, in scenarios like Robinson and Berridge's (2013) experiment, the needing mechanism likely accounts for the immediate increase in wanting associated with the dead salt cue in the unlearned salt depletion state. If salt depletion persists, learning will eventually assign high value to the dead salt cue over multiple trials. However, this gradual update contrasts with the instant change observed in Robinson and Berridge's (2013) experiment, suggesting an active inference process in determining which sensory representation to value, driven by the pervasiveness that renders all other sensory representations surprising except the Dead Sea salt cue. The fact that needing influenced both the Dead Sea salt lever cue (wanting) and (later) the hedonic contact with the Dead Sea salt (liking) indicates that needing increases different forms of precision within different subsystems, depending on whether the available stimulus is processed by that subsystem. Moreover, all this shows that, under need state, the valuation of states (stimuli or actions) has to do with both minimization of surprise and availability as a path to the preferred state, which are represented via yC and y∣π, respectively, in the −EQ(y|π) ln[P(yi,n|Cp,n)] equation. The tendency or process under wich π is selected in a way that minimizes the aforementioned equation is what we describe as “needing.”

The role of needing and the way it interacts with other reward subsystems are not completely understood. Here, we aimed to provide a computationally guided analysis of the mechanisms of needing from the normative perspective of active inference and the hypothesis of (embodied) pervasiveness of need states.

We firstly defined a need in terms of pervasiveness and surprise; that is, a state that is pervasive over time and over other dimensions of the individual's life, and whose negative impact is surprising with regard to prior preferences. We then defined needing as an active inference process that aims at reducing such pervasive surprise by inducing a tendency to transition from states to states, toward the preferred state. Here, the term “surprise” is used in the sense prescribed by theories like predictive coding and active inference. The key idea is that living creatures strive to remain within tight physiological boundaries (i.e., preferred states) and are surprised outside them—like a fish out of water. This perspective suggests that being in a need state is intrinsically costly (where the cost is related to surprise), and thus a state of need may exert a directional effect on action selection and motivation, because creatures would have an automatic tendency to select policies that avoid surprises and lead to preferred states. Importantly, this automatic tendency would be present independently of any reward or reward cue. Moreover, the pervasive effect of a need state acts on the system as a whole, making all states surprising with regard to prior preferences, except the relevant rewarding states, that is, those on the path to the preferred state. Hence, it is the embodiment of that pervasiveness into prior preferences that allows needing to activate prior policies that direct the system, in a gradient-like manner, toward the relevant rewarding states.

We further defined the interaction between needing and other subsystems, specifically the wanting (or liking) subsystem, where needing increases the precision of relevant stimuli or action processed within those subsystems. In the case of wanting, this precision is that of policies that achieve preferred (goal or reward) states in active inference, consistent with previous work that linked policy precision with dopaminergic reactivity (Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2017; FitzGerald et al., 2015) and thus incentive salience (Berridge, 2004). From this perspective, Pavlovian cues that signal that there is a path to preferred state acquire incentive salience (Berridge, 2004) and generate “wanting.” Needing amplifies such wanting, as a state of greater need can amplify policy precision by amplifying the value of reward cues that do not co-occur with the pervasive surprise (because they lead to the preferred state). Hence, the higher the initial state of need, the greater the wanting associated with Pavlovian cues related to relevant rewards.

Simulation 1: The Needing System and Its Directional Effect on Behavior

This simulation demonstrates that for an organism that uses prior preferences to embody the pervasiveness effect, need states can have directional effects independently of reward prediction. Needing governs directional motivation because of the inherent tendency of living beings to move toward preferred states. This propensity activates policies leading to those preferred states, which in turn elevates the preference (and value) of states within their trajectory. Homeostasis and allostasis, which assist animals in maintaining viable physiological boundaries (Holmes, 2022; Demekas, Parr, & Friston, 2020; Barrett, 2017; Sterling, 2004; Sterling & Eyer, 1988; Cannon, 1939), mediate this tendency. Indeed, from the active inference perspective, a living organism continuously strives to reach or remain in its preferred states (which could be sometimes evolutionarily defined) through homeostatic or allostatic regulation, at the somatic, autonomic, and neuroendocrine levels (Parr et al., 2022; Swanson, 2000). These preferred states act as drives or goals that, via homeostasis and allostasis, direct action (Barrett, 2017); hence, the directional effect of need states. This directional influence, contingent on the propensity to occupy preferred states, also underlies the amplifying effect of need states on wanting, liking/pleasure, interoceptive prediction, choice, and so forth, by increasing the precision of their related states (stimuli or actions) located on the path toward the preferred state. This leads to Simulation 2 below.

Simulation 2: The Effect of Needing on Reward Subsystems (Wanting, Liking, Etc.)

This simulation shows that the presence (vs. the absence) of (a path to) a reward decreases the entropy of the states that Agent 1 plans to occupy and increases the associated precision, that is, the certainty that it should occupy these states. Importantly, our Simulation 2 also shows that the decrease in entropy over which state to occupy, and the increase of associated precision, are magnified when Agent 1 is in a more severe state of need (i.e., when the costs of the nonrewarded states are increased) and there is a path to the preferred state. As an example, if this path to the preferred state is signaled by a Pavlovian reward cue processed within the dopaminergic subsystem, then the wanting associated with that cue will be magnified. In other words, the more costly (surprising) these states are, the more the agent is certain that it needs to go to the preferred state. This illustrates how need states amplify the wanting (and/or the liking) of stimuli: by reducing entropy and making the agent more confident about what stimulus, response or course of action to select. Need states have cascading effects also on the stimuli and actions in the path toward goal or reward states. When in a severe need state, relevant stimuli, reward cues, and actions have a greater role in reducing entropy and increasing the confidence in the selected course of actions (Holmes, 2022; Parr et al., 2022). These relevant stimuli, reward cues, and actions are therefore assigned a greater value and a greater “need-generated” salience, which, within the wanting subsystem, would neurophysiologically correspond to increased dopaminergic activity that attribute higher incentive salience, that is, higher wanting, to reward cues (Berridge, 2004).

Competing Needs and Competing Goals

The proposed interplay of surprise, precision, and pervasiveness may offer a normative framework for creatures to navigate situations of competition between needs or between needs and other goals. For example, when faced with hunger and thirst simultaneously, the decision may hinge on the degree of deviation (surprise) or the intensity of the need state. If hunger greatly outweighs thirst, seeking food is likely. In addition, the presence or absence of intermediary states (precision) fulfilling one need over the other plays a role. If food is more accessible than water, eating may be chosen; conversely, if water is more available, drinking may be pursued. Furthermore, pervasiveness factors into this choice: If pursuing one need inadvertently exacerbates the other more than vice versa, the latter may be prioritized because of pervasiveness, as it would “translate” its impact and make it more noticeable. For instance, if obtaining food increases thirst significantly because of exertion, whereas getting water only affects hunger minimally, opting for water first may be preferable, because thirst would become more “felt” when trying to resolve hunger. Similarly, when a non-need goal is in competition with a need state, the decision process considers need level, precision, and pervasiveness. Strong needs and high pervasiveness, as well as a path to the preferred state (precision), tend to amplify the difficulty of bypassing the need. For instance, mild hunger in winter may not deter meeting someone outside, but a cold-related need state will make the decision to meet outside more challenging as the need will increase and be pervasive, even more so if there is a warm place where one could meet. All in all, behavior balances goals, reflexes, and sensory inputs to control one's state in the environment (Cisek, 2022).

Strength and Weakness of the Model

One advantage of using active inference is its natural incorporation of the pervasiveness of need states through the concept of prior preferences. Conversely, the choice of a reinforcement learning model is motivated by the ability to capture the independent nature of reward prediction and needing. In our study, we used a relatively simple TD model, which may limit the interpretation of our results. Future studies could broaden the results, by considering more advanced reinforcement learning models, such as those that consider more directly homeostatic needs and the notion of progress to goal (Juechems & Summerfield, 2019; Keramati & Gutkin, 2014).

In summary, our simulations show that our model using active inference provides a natural way to model the needing as its own system. Traditional models that discuss need-related motivation often assume an automatic link between needing and reward prediction (Keramati & Gutkin, 2014; also see Berridge, 2023). The proposed model presents a more nuanced view, acknowledging the association and dissociation of needing with other subsystems, such as the wanting subsystem, that process external cues. This broader perspective has the potential to explain a wider range of experimental findings, both at the neural and behavioral levels. Indeed, the increase in precision, which, in active inference, is interpreted as neuronal gain (Friston, 2010), could possibly be viewed as the (need induced) salience conferred by need states to relevant stimuli (see Chen, Papies, & Barsalou, 2016). In this sense, if we interpret the increase in precision (in our results) as a need-induced neuronal gain toward stimuli/events that are treated within some subsystem, it becomes clear how needing can function independently while interacting with other subsystems like wanting, liking, and interoception. For instance, needing can increase precision of stimuli/events treated within other subsystems, such as the liking or interoceptive ones (Bosulu et al., 2022; Becker et al., 2019; Cabanac, 2017; Berridge & Kringelbach, 2015), even in absence of a Pavlovian cue that triggers the wanting subsystem (Salamone et al., 2018; Wassum et al., 2011), and this fits well with some recent meta-analytic results of human fMRI data distinguishing between needing and wanting (Bosulu et al., 2022). Conversely, a specific need can intensify wanting when a relevant Pavlovian reward cue is present, possibly through this increase in (policy) precision. Notably, this can occur through “active inference,” that is, without (re)learning (or liking) the consequence of that reward in that specific need state (Berridge, 2023; Zhang et al., 2009; Berridge, 2007). We acknowledge that although simulations illustrate the benefits and predictions of the proposed active inference framework of needing, empirical data testing is still required.

The current discussion focuses on how needs influence perception of the external environment and their impact on the wanting (and liking, etc.) associated with relevant cues. However, a pertinent question arises: Could external cues (e.g., wanting) potentially heighten the sense of needing? Indeed, research suggests that needing and wanting are distinct constructs (Bosulu et al., 2022; Berridge, 2004). Because of pervasiveness, need states persist with the creature regardless of its location or time. This differs from usual external rewards (and usual aversive states), which are tied to specific states, that is, places or times. Our definition of pervasiveness, however, means that from the point of view of the animal, certain forms of expectations or wanting may be perceived as needing, if they are sustained over time and impact other states. This would essentially happen for more complex creatures, for example, humans, that are capable of holding in their mind the cues that cause the desire/wanting. This article does not discuss cases in which wanting or expectations influence needing, and these could be covered in future research.

Regarding pervasiveness, it is worth noticing that the joint probability between a state and a need state does not depend on the level of need. Another perspective is that prior preferences, particularly those possibly genetically encoded, persist with the creature regardless of its location or time. Therefore, when a need state arises, the deviation from prior preference is felt regardless of the situation, proportionally to the pervasiveness. For the sake of simplicity, we did not fully consider how different need states might map to different levels of pervasiveness.

Summary and Future Developments

The proposed model of needing has potential to be extended to broader psychological phenomena in humans (Stussi & Pool, 2022; Pool, Sennwald, Delplanque, Brosch, & Sander, 2016; Maner, DeWall, Baumeister, & Schaller, 2007; Baumeister & Leary, 1995; Maslow, 1943). That being said, it is important to recognize that the term “need” can occasionally be interpreted as an expression of personal requirement, as in “I need more information.” There is a growing body of literature showing that animals show information-seeking and curiosity driven behavior (Gottlieb, Cohanpour, Li, Singletary, & Zabeh, 2020), and dopamine reflects information-seeking dynamics, not just reward prediction dynamics (Blanco-Ponzo, Akam, & Walton, 2024). Although this aspect lies beyond the scope of the current article, it presents an intriguing avenue for further investigation. This framework could also be applied to drug dependence/addiction, where a crucial question arises: Does drug consumption stem from needing, where the drug state becomes embedded as a prior preference over internal states (Turel & Bechara, 2016; O'Brien et al., 2006), or from wanting, driven by excessive dopamine sensitization to drug-related policies leading to the drug (Berridge & Robinson, 2016; O'Brien et al., 2006)? Depending on the case, different brain regions and behavioral therapy approaches may be targeted.

Overall, this study aimed to provide a conceptual model for needing and its interaction with reward subsystems, based on the active inference framework and the embodied pervasiveness hypothesis. However, further work is needed to fully clarify and empirically test the relationships between the abstract notions introduced here and their underlying biological substrates. A systematic mapping between the information-theoretic concepts used and neurobiological evidence remains an open objective for future research.

Agent 1

Agent 1 incorporates the main ideas and equations discussed in this article about needing and wanting systems. It is a simplified version of active inference, in which the perceptual part is kept as simple as possible, by assuming that all the states of the grid world are observable. Technically, this means that we are dealing with a Markov Decision Process and not a Partially Observable Markov Decision Process as more commonly done in active inference (see Friston et al., 2017; Friston, Daunizeau, & Kiebel, 2009). This simplifying assumption is motivated by the fact that our focus in this work is on action selection and not perceptual discrimination.

In Agent 1, the need state has a pervasive effect on all the states of the grid world (except the reward state) as described above. Following active inference, this pervasive effect is reflected in the prior preferences for the states to occupy (denoted as C, to follow the usual notation of active inference in discrete time), which is greater for the rewarding state than for any other state. The prior preferences followed a normal distribution centered on the value of the preferred state with a degree of sensitivity regarding deviations from that centered value. The center, that is, prior preference, is the mean; the sensitivity is the variance; and their values were 1 and 0.5, respectively. (Note that the agent may have behaved differently if the variance/sensitivity was different.) This followed the logic of the “homeostatic reinforcement learning” model of Keramati and Gutkin (2014), which is based on minimizing the sum of discounted deviations from a setpoint.

To model the pervasive effect of the need state, we assume a joint probability of 1 between the (increasing) need state and other states, except the preferred/satiety state, which had a joint probability of 0 with the need state. (Please note that the behavior of the agent may have been different if the joint probability with the other states was significantly less than one.)

Given the pervasiveness of the need state, almost all of the yi acquire the negative valence and their conditional probability based on prior preference (of satiety) decreases, because their joint probability, that is, co-occurrence, with satiety decreases. Hence, only the state (or group of states) on the path to the preferred state has high probability as it co-occurs with the prior preference.

Because the agent expects to occupy (or to move toward) these a priori probable states, the prior over states also translates into priors over actions or action sequences (policies) that achieve such states. In this simplified setting, action (and policy) selection simply corresponds to inferring a distribution of states that it prefers to occupy and policies to reach (sequences of) those states. In other words, Agent 1 tends to select policies that lead it to achieve goal states—which in Bayesian terms corresponds to maximizing model evidence (Parr et al., 2022).

Agent 2

Agent 2 had the “need state” (State 8) but did not have the pervasive effect of need states as Agent 1. To implement an agent that is guided by reward prediction, we use the reinforcement learning framework. The goal of Agent 2 is to maximize a reward function, through reward prediction. Thus, the agent makes decisions based on prediction of rewards assessed by state-action values, that is, each decision to pursue a course of actions will depend on the value of actions given the current states (see Sutton & Barto, 2018). Here, the policies depending on the action values are denoted Qπ(s, a) and given by:
(20)
The equation shows the value or “quality” (Q) of the action (a) in state (s) under a policy (π). The function denoted Qπ(s, a) expresses the expected (E) return (R), which is the (expected) discounted (βi) sum of rewards (rt+i), starting from state (s) and taking the action (a), and thereafter following policy (π). Here, the state s for Agent 2 is equivalent to the state/observation y of Agent 1.
Learning of action is updated by the TD between the previous action value and the current one as in the equation below:
(21)
where α represents a learning rate.
The agent's decision after learning is based on the optimal policy π*, that is, the one that maximizes the expected return, and therefore the optimal Qπ(s, a), noted Q*(s, a) is equal to:
(22)
where maxa is related to the action that maximizes Q(s, a).
For a better comparison with the Agent 1 (that embodied pervasiveness), the actions of the Agent 2 were also transformed into a softmax function:
(23)
where σ is the softmax function that turns action values into probabilities.

We should keep in mind that Agent 2 is used only to illustrate prediction in its simplest form and does not include advanced reinforcement learning techniques.

Side by Side Table Showing Functioning of Our Agent 1 (Needing) and Agent 2 (Learning)

DescriptionNeedingExternal Learning
Prior preference Cp,n N/A 
Reward in state N/A R 
Need yn = n = −ln[P(yn|Cp,n)] yn = n = −r 
Need under a state yi,n = p(yi, yn) × [ynyn = yn 
yi = yi 
yi,n = yn or yi,n = yi 
Learning no yes: TD learning 
Decision: find π that maximizes EQ(y|π) ln[P(yi,n|Cp,n)] Eπ{R|yi,n, a
DescriptionNeedingExternal Learning
Prior preference Cp,n N/A 
Reward in state N/A R 
Need yn = n = −ln[P(yn|Cp,n)] yn = n = −r 
Need under a state yi,n = p(yi, yn) × [ynyn = yn 
yi = yi 
yi,n = yn or yi,n = yi 
Learning no yes: TD learning 
Decision: find π that maximizes EQ(y|π) ln[P(yi,n|Cp,n)] Eπ{R|yi,n, a

We thank Kent C. Berridge for providing valuable insights and feedback during the development of this article.

Corresponding author: Juvenal Bosulu, University of Pennsylvania, Richards Medical Research, Laboratories, Philadelphia, PA, or via e-mail: [email protected].

All data are available upon request.

Juvenal Bosulu: Conceptualization; Data curation; Methodology; Visualization; Writing—Review & editing. Giovanni Pezzulo: Writing—Review & editing. Sébastien Hétu: Funding acquisition; Writing—Review & editing.

The research was supported in part by NSERC Discovery grant #RGPIN-2018-05698 to S. H. and UdeM institutional funds to J. B. and S. H. This research is supported by funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the specific grant agreement no. 945539 (Human Brain Project SGA3) to G. P. and no. 952215 (TAILOR) to G. P., and the European Research Council under the grant agreement no. 820213 (ThinkAhead) to G. P.

Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.

Balleine
,
B.
(
1992
).
Instrumental performance following a shift in primary motivation depends on incentive learning
.
Journal of Experimental Psychology: Animal Behavior Processes
,
18
,
236
250
. ,
[PubMed]
Balleine
,
B. W.
, &
Dickinson
,
A.
(
1998
).
Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates
.
Neuropharmacology
,
37
,
407
419
. ,
[PubMed]
Barrett
,
L. F.
(
2017
).
The theory of constructed emotion: An active inference account of interoception and categorization
.
Social Cognitive and Affective Neuroscience
,
12
,
1
23
. ,
[PubMed]
Barrett
,
L. F.
, &
Simmons
,
W. K.
(
2015
).
Interoceptive predictions in the brain
.
Nature Reviews Neuroscience
,
16
,
419
429
. ,
[PubMed]
Baumeister
,
R. F.
, &
Leary
,
M. R.
(
1995
).
The need to belong: Desire for interpersonal attachments as a fundamental human motivation
.
Psychological Bulletin
,
117
,
497
529
. ,
[PubMed]
Becker
,
S.
,
Bräscher
,
A.-K.
,
Bannister
,
S.
,
Bensafi
,
M.
,
Calma-Birling
,
D.
,
Chan
,
R. C. K.
, et al
(
2019
).
The role of hedonics in the Human Affectome
.
Neuroscience & Biobehavioral Reviews
,
102
,
221
241
. ,
[PubMed]
Berridge
,
K. C.
(
1996
).
Food reward: Brain substrates of wanting and liking
.
Neuroscience & Biobehavioral Reviews
,
20
,
1
25
. ,
[PubMed]
Berridge
,
K. C.
(
2004
).
Motivation concepts in behavioral neuroscience
.
Physiology & Behavior
,
81
,
179
209
. ,
[PubMed]
Berridge
,
K. C.
(
2007
).
The debate over dopamine's role in reward: The case for incentive salience
.
Psychopharmacology
,
191
,
391
431
. ,
[PubMed]
Berridge
,
K. C.
(
2012
).
From prediction error to incentive salience: Mesolimbic computation of reward motivation
.
European Journal of Neuroscience
,
35
,
1124
1143
. ,
[PubMed]
Berridge
,
K. C.
(
2018
).
Evolving concepts of emotion and motivation
.
Frontiers in Psychology
,
9
,
1647
. ,
[PubMed]
Berridge
,
K. C.
(
2023
).
Separating desire from prediction of outcome value
.
Trends in Cognitive Sciences
,
27
,
932
946
. ,
[PubMed]
Berridge
,
K. C.
, &
Kringelbach
,
M. L.
(
2015
).
Pleasure systems in the brain
.
Neuron
,
86
,
646
664
. ,
[PubMed]
Berridge
,
K. C.
, &
Robinson
,
T. E.
(
2016
).
Liking, wanting, and the incentive-sensitization theory of addiction
.
American Psychologist
,
71
,
670
679
. ,
[PubMed]
Blanco-Pozo
,
M.
,
Akam
,
T.
, &
Walton
,
M. E.
(
2024
).
Dopamine-independent effect of rewards on choices through hidden-state inference
.
Nature Neuroscience
,
27
,
286
297
. ,
[PubMed]
Bosulu
,
J.
,
Allaire
,
M.-A.
,
Tremblay-Grénier
,
L.
,
Luo
,
Y.
,
Eickhoff
,
S.
, &
Hétu
,
S.
(
2022
).
“Wanting” versus “needing” related value: An fMRI meta-analysis
.
Brain and Behavior
,
12
,
e32713
. ,
[PubMed]
Bouton
,
M. E.
(
2016
).
Learning and behavior: A contemporary synthesis
(2nd ed.).
Sinauer Associates
.
Cabanac
,
M.
(
2017
).
Pleasure and joy, and their role in human life
. In
Creating the productive workplace
(pp.
73
82
).
Routledge
.
Cannon
,
W. B.
(
1939
).
The wisdom of the body
.
New York
:
W. W. Norton & Company
.
Chen
,
J.
,
Papies
,
E. K.
, &
Barsalou
,
L. W.
(
2016
).
A core eating network and its modulations underlie diverse eating phenomena
.
Brain and Cognition
,
110
,
20
42
. ,
[PubMed]
Cisek
,
P.
(
2022
).
Evolution of behavioural control from chordates to primates
.
Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences
,
377
,
20200522
. ,
[PubMed]
Craig
,
A. D.
(
2003
).
Interoception: The sense of the physiological condition of the body
.
Current Opinion in Neurobiology
,
13
,
500
505
. ,
[PubMed]
Demekas
,
D.
,
Parr
,
T.
, &
Friston
,
K. J.
(
2020
).
An investigation of the free energy principle for emotion recognition
.
Frontiers in Computational Neuroscience
,
14
,
30
. ,
[PubMed]
Dickinson
,
A.
, &
Balleine
,
B.
(
1994
).
Motivational control of goal-directed action
.
Animal Learning & Behavior
,
22
,
1
18
.
FitzGerald
,
T. H. B.
,
Dolan
,
R. J.
, &
Friston
,
K.
(
2015
).
Dopamine, reward learning, and active inference
.
Frontiers in Computational Neuroscience
,
9
,
136
. ,
[PubMed]
Friston
,
K.
(
2010
).
The free-energy principle: A unified brain theory?
Nature Reviews Neuroscience
,
11
,
127
138
. ,
[PubMed]
Friston
,
K.
, &
Ao
,
P.
(
2012
).
Free energy, value, and attractors
.
Computational and Mathematical Methods in Medicine
,
2012
,
937860
. ,
[PubMed]
Friston
,
K. J.
,
Daunizeau
,
J.
, &
Kiebel
,
S. J.
(
2009
).
Reinforcement learning or active inference?
PLoS One
,
4
,
e6421
. ,
[PubMed]
Friston
,
K.
,
FitzGerald
,
T.
,
Rigoli
,
F.
,
Schwartenbeck
,
P.
, &
Pezzulo
,
G.
(
2017
).
Active inference: A process theory
.
Neural Computation
,
29
,
1
49
. ,
[PubMed]
Friston
,
K.
,
Kilner
,
J.
, &
Harrison
,
L.
(
2006
).
A free energy principle for the brain
.
Journal of Physiology-Paris
,
100
,
70
87
. ,
[PubMed]
Gottlieb
,
J.
,
Cohanpour
,
M.
,
Li
,
Y.
,
Singletary
,
N.
, &
Zabeh
,
E.
(
2020
).
Curiosity, information demand and attentional priority
.
Current Opinion in Behavioral Sciences
,
35
,
83
91
.
Hamid
,
A. A.
,
Pettibone
,
J. R.
,
Mabrouk
,
O. S.
,
Hetrick
,
V. L.
,
Schmidt
,
R.
,
Vander Weele
,
C. M.
, et al
(
2016
).
Mesolimbic dopamine signals the value of work
.
Nature Neuroscience
,
19
,
117
126
. ,
[PubMed]
Hogarth
,
L.
, &
Chase
,
H. W.
(
2011
).
Parallel goal-directed and habitual control of human drug-seeking: Implications for dependence vulnerability
.
Journal of Experimental Psychology: Animal Behavior Processes
,
37
,
261
276
. ,
[PubMed]
Holmes
,
J.
(
2022
).
Friston's free energy principle: New life for psychoanalysis?
BJPsych Bulletin
,
46
,
164
168
. ,
[PubMed]
Juechems
,
K.
, &
Summerfield
,
C.
(
2019
).
Where does value come from?
Trends in Cognitive Sciences
,
23
,
836
850
. ,
[PubMed]
Keramati
,
M.
, &
Gutkin
,
B.
(
2014
).
Homeostatic reinforcement learning for integrating reward collection and physiological stability
.
eLife
,
3
,
e04811
. ,
[PubMed]
Liu
,
Z.
,
Lin
,
R.
, &
Luo
,
M.
(
2020
).
Reward contributions to serotonergic functions
.
Annual Review of Neuroscience
,
43
,
141
162
. ,
[PubMed]
Livneh
,
Y.
,
Sugden
,
A. U.
,
Madara
,
J. C.
,
Essner
,
R. A.
,
Flores
,
V. I.
,
Sugden
,
L. A.
, et al
(
2020
).
Estimation of current and future physiological states in insular cortex
.
Neuron
,
105
,
1094
1111
. ,
[PubMed]
Luo
,
M.
,
Li
,
Y.
, &
Zhong
,
W.
(
2016
).
Do dorsal raphe 5-HT neurons encode “beneficialness”?
Neurobiology of Learning and Memory
,
135
,
40
49
. ,
[PubMed]
MacGregor
,
D.
(
1960
).
The human side of enterprise
(
Vol. 21
, pp.
108
110
).
New York
:
McGraw-Hill
.
Maner
,
J. K.
,
DeWall
,
C. N.
,
Baumeister
,
R. F.
, &
Schaller
,
M.
(
2007
).
Does social exclusion motivate interpersonal reconnection? Resolving the “porcupine problem”
.
Journal of Personality and Social Psychology
,
92
,
42
55
. ,
[PubMed]
Maslow
,
A. H.
(
1943
).
A theory of human motivation
.
Psychological Review
,
50
,
370
396
.
O'Brien
,
C. P.
,
Volkow
,
N.
, &
Li
,
T.-K.
(
2006
).
What's in a word? Addiction versus dependence in DSM-V
.
American Journal of Psychiatry
,
163
,
764
765
. ,
[PubMed]
Panksepp
,
J.
(
2004
).
Affective neuroscience: The foundations of human and animal emotions
.
Oxford University Press
.
Parr
,
T.
,
Pezzulo
,
G.
, &
Friston
,
K. J.
(
2022
).
Active inference: The free energy principle in mind, brain, and behavior
.
Cambridge, MA
:
MIT Press
.
Passingham
,
R. E.
, &
Wise
,
S. P.
(
2012
).
The neurobiology of the prefrontal cortex: Anatomy, evolution, and the origin of insight
.
Oxford University Press
.
Pezzulo
,
G.
,
Rigoli
,
F.
, &
Friston
,
K.
(
2015
).
Active inference, homeostatic regulation and adaptive behavioural control
.
Progress in Neurobiology
,
134
,
17
35
. ,
[PubMed]
Pool
,
E.
,
Sennwald
,
V.
,
Delplanque
,
S.
,
Brosch
,
T.
, &
Sander
,
D.
(
2016
).
Measuring wanting and liking from animals to humans: A systematic review
.
Neuroscience & Biobehavioral Reviews
,
63
,
124
142
. ,
[PubMed]
Robinson
,
M. J.
, &
Berridge
,
K. C.
(
2013
).
Instant transformation of learned repulsion into motivational “wanting”
.
Current Biology
,
23
,
282
289
. ,
[PubMed]
Sajid
,
N.
,
Ball
,
P. J.
,
Parr
,
T.
, &
Friston
,
K. J.
(
2021
).
Active inference: Demystified and compared
.
Neural Computation
,
33
,
674
712
. ,
[PubMed]
Salamone
,
J. D.
,
Correa
,
M.
,
Yang
,
J.-H.
,
Rotolo
,
R.
, &
Presby
,
R.
(
2018
).
Dopamine, effort-based choice, and behavioral economics: Basic and translational research
.
Frontiers in Behavioral Neuroscience
,
12
,
52
. ,
[PubMed]
Schultz
,
W.
,
Dayan
,
P.
, &
Montague
,
P. R.
(
1997
).
A neural substrate of prediction and reward
.
Science
,
275
,
1593
1599
. ,
[PubMed]
Smith
,
B. J.
, &
Read
,
S. J.
(
2022
).
Modeling incentive salience in Pavlovian learning more parsimoniously using a multiple attribute model
.
Cognitive, Affective, & Behavioral Neuroscience
,
22
,
244
257
. ,
[PubMed]
Sterling
,
P.
(
2004
).
Principles of allostasis: Optimal design, predictive regulation, pathophysiology, and rational therapeutics
. In
Allostasis, homeostasis, and the costs of physiological adaptation
(pp.
17
64
).
Cambridge
:
Cambridge University Press
.
Sterling
,
P.
, &
Eyer
,
J.
(
1988
).
Allostasis: A new paradigm to explain arousal pathology
. In
S.
Fisher
&
J.
Reason
(Eds.),
Handbook of life stress, cognition and health
(pp.
629
649
).
Hoboken, NJ
:
John Wiley & Sons
.
Sterling
,
P.
, &
Laughlin
,
S.
(
2015
).
Principles of neural design
.
Cambridge, MA
:
MIT Press
.
Stussi
,
Y.
, &
Pool
,
E. R.
(
2022
).
Multicomponential affective processes modulating food-seeking behaviors
.
Current Opinion in Behavioral Sciences
,
48
,
101226
.
Sutton
,
R. S.
, &
Barto
,
A. G.
(
2018
).
Reinforcement learning: An introduction
.
Cambridge, MA
:
MIT Press
.
Swanson
,
L. W.
(
2000
).
Cerebral hemisphere regulation of motivated behavior
.
Brain Research
,
886
,
113
164
. ,
[PubMed]
Toates
,
F.
(
1994
).
Comparing motivational systems—An incentive motivation perspective
. In
C. R.
Legg
&
D. A.
Booth
(Eds.),
Appetite: Neural and behavioural bases
(pp.
305
328
).
Oxford University Press
.
Tschantz
,
A.
,
Barca
,
L.
,
Maisto
,
D.
,
Buckley
,
C. L.
,
Seth
,
A. K.
, &
Pezzulo
,
G.
(
2022
).
Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference
.
Biological Psychology
,
169
,
108266
. ,
[PubMed]
Turel
,
O.
, &
Bechara
,
A.
(
2016
).
A triadic reflective-impulsive-interoceptive awareness model of general and impulsive information system use: Behavioral tests of neuro-cognitive theory
.
Frontiers in Psychology
,
7
,
601
. ,
[PubMed]
Wassum
,
K. M.
,
Ostlund
,
S. B.
,
Balleine
,
B. W.
, &
Maidment
,
N. T.
(
2011
).
Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling
.
Learning & Memory
,
18
,
475
483
. ,
[PubMed]
Watson
,
P.
,
Wiers
,
R. W.
,
Hommel
,
B.
, &
de Wit
,
S.
(
2014
).
Working for food you don't desire. Cues interfere with goal-directed food-seeking
.
Appetite
,
79
,
139
148
. ,
[PubMed]
Zhang
,
J.
,
Berridge
,
K. C.
,
Tindell
,
A. J.
,
Smith
,
K. S.
, &
Aldridge
,
J. W.
(
2009
).
A neural computational model of incentive salience
.
PLoS Computational Biology
,
5
,
e1000437
. ,
[PubMed]

Author notes

*

Now at Perelman School of Medicine, University of Pennsylvania.

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit https://creativecommons.org/licenses/by/4.0/legalcode.