We gracefully thank the different commentators for their thoughtful and inspiring reflections on the concept of incentive hope. We had proposed this concept in our target article to explain foraging motivation when access to food is unpredictable. Many of the commentaries consist of positive attempts to enlarge the applications of our concept beyond the limited context in which we used it, that is, mainly food seeking in nonhuman animals. Other commentaries are more critical, suggesting that the concept accounts for less than it should or is reducible to other concepts. These reflections have been helpful to us in broadening the scope of incentive hope to a certain extent and also reinforce our argumentation regarding some criticisms.
Our target article generated discussions about uncertainty and behavior from many distinct research fields that reach from protistology (Clark) to social anthropology (Aharonov-Majar & Suleiman; Misiak, Sorokowski, & Karwowski [Misiak et al.]). Despite the breadth of these fields, the comments and criticisms can be grouped in a small number of directions. Thus, the response is organized according to the degree of agreement with our view. Section R1 is a response to commentators who suggest that the concept of incentive hope is unnecessary (other explanations work) and/or insufficient (requires other parameters) to account for behavior under uncertainty. Section R2 specifically discusses the relevance of the incentive hope hypothesis in shedding light on maladaptive behaviors such as gambling and pessimism. Section R3 is about the scope and the limits of our computational model and its relation to the theory itself. Finally, section R4 considers possible extensions of the concept of incentive hope beyond food-seeking behavior and describes why it is potentially applicable to phylogenetically distant species, too.
R1. Is incentive hope a useless concept?
Several commentators suggest that incentive hope is an unnecessary and/or insufficient concept to explain stronger foraging motivation in an environment that is sending signals that the available amounts of food are low. In contrast, we explicitly stress the importance of incentive hope here, while recognizing that it is not the whole story about food seeking.
R1.1. “Wanting” and fear
According to Smulders, Boswell, & Henderson (Smulders et al.), incentive salience (or “wanting”) is not limited to approaching a present stimulus, and unpredictability is simply only one of its numerous modulatory factors. It is correct that “wanting” can be recruited whether a stimulus (conditioned or unconditioned [CS or UCS]) is present or absent, although a large majority of experiments refer to the direct perception of the stimulus. Our simplification aimed to suggest that organisms may differently process the predictable presence and the unpredictable absence of the stimulus, because unpredictability is only significant in its absence. We hypothesized that evolution shaped mechanisms that enhance food seeking under unpredictability because of potential negative consequences for survival. Unpredictability and its negative consequences cannot be determinants of “wanting.” Actually, these two factors lead to a paradox between motivation and preference: If unpredictability and its negative consequences increased incentive salience only (as the higher response rates suggest), they should be preferred to predictability and positive consequences for survival. As shown, this does not happen. The goal of the incentive hope hypothesis is precisely to solve this kind of paradox while remaining coherent relative to the well-established concept of incentive salience. Thus, our concept appears necessary and may be revealed to be useful each time an uncertainty-induced behavioral paradox is found. In addition, without incentive hope, an increase in foraging activity should be interpreted as the consequence of a strong deprivation-related “wanting.” However, intense food deprivation weakens the forager, causing slower movements and ineffective search. A process such as incentive hope, in which “wanting” is only a modulatory factor, can stimulate seeking behavior before the forager is close to starvation. We agree with Smulders et al. that a sophisticated neurophysiological machinery can have an impact on “wanting,” but this does not demonstrate that “wanting” is sufficient to explain animal foraging under all environmental conditions. Psychological and neuroscientific investigations (e.g., partial reinforcement and contrafreeloading) indicate that foraging motivation requires more than “wanting.” Distinct neurophysiological mechanisms may be correlated with distinct, complementary psychological processes. In particular, we believe that the interaction between dopamine and glucocorticoids (and perhaps corticotropin-releasing factor [CRF]) under reward unpredictability transforms “wanting” into “hope.”
Severely food-deprived organisms might experience fear of starvation (Zack), a situation that is likely to be associated with high glucocorticoid levels. But evidence based on such a measurement often indicates that food unpredictability causes only mild stress (Bauer et al. Reference Bauer, Glassman, Cyr and Romero2011; Marasco et al. Reference Marasco, Boner, Heidinger, Griffiths and Monaghan2015; Reneerkens et al. Reference Reneerkens, Piersma and Ramenofsky2002). In this case, glucocorticoid-induced dopamine release (motivation), rather than glucocorticoids themselves (stress), might control foraging performance (e.g., Sinha & Jastreboff Reference Sinha and Jastreboff2013). For example, Lemos et al. (Reference Lemos, Wanat, Smith, Reyes, Hollon, Van Bockstaele, Chavkin and Phillips2012) reported that CRF-induced dopamine in the nucleus accumbens has appetitive properties, causing conditioned place preference in mice that received intra-accumbens CRF (500 ng) but not in mice that received 6-hydroxydopamine (6-OHDA) in addition to CRF. The CRF antagonist α-helical CRF also reduces approach to and exploration of a novel object. In contrast, severe stress abolishes these appetitive effects of CRF on dopamine release for long periods (> 90 days) after stress exposure (see also Inoue et al. Reference Inoue, Tsuchya and Koyama1994). The abolition of depression-like responses to stress was prevented by the glucocorticoid receptor antagonist RU486. Recently, Mascia et al. (Reference Mascia, Neugebauer, Brown, Bubula, Nesbitt, Kennedy and Vezina2019) found that high dopamine outflows in the nucleus accumbens of rats occur in response to exposure to a variable-ratio (unpredictable) schedule for a saccharin reward, compared with rats exposed to a fixed-ratio (predictable) schedule. They did not observe any schedule-specific effects on response rates in these instrumental tasks. But the rats tested with the unpredictable schedule showed subsequent increase in locomotion after an amphetamine injection and responded at higher rates on a progressive-ratio schedule for amphetamine, obtaining more amphetamine than the rats tested with the predictable schedule. Such results are compatible with an interpretation of reward-seeking behavior in terms of motivation rather than stress or fear.
R1.2. Learning and exploration
As pointed out by some commentators, cognition plays a crucial role in reward seeking; information, learning, and the development of new action schemes are proposed to be sufficient bases for exploration. Effective reward seeking may indeed require cognitive abilities. But we have the impression that several authors see cognitive processing (in particular, expectations) as the motivation to forage itself. Pezzulo & Friston suggest that information gain is rewarding per se – an organism explores its environment (epistemic dimension) to be able to exploit the information collected (pragmatic dimension). Nobody would disagree with this exploration-exploitation principle, which approximately corresponds to the distinction between goal-directed behavior and habit (e.g., see Yin & Knowlton Reference Yin and Knowlton2006). What these authors call “active inference” requires an expectation-based motivational process controlling instrumental actions, while incentive hope is assumed to operate via Pavlovian processes, as does incentive salience. We think that foraging motivation is basically related to incentive processes rather than expectations leading to instrumental actions, even though those actions could indirectly be influenced through Pavlovian-to-instrumental transfer. How? Dopamine release in the nucleus accumbens core predicts both the cueing effect (Peciña & Berridge Reference Peciña and Berridge2013) and the strength of lever presses in rats during Pavlovian-to-instrumental transfer (Wassum et al. Reference Wassum, Ostlund, Loewinger and Maidment2013). With unreliable CSs, something similar could happen – except that infrequent, ambiguous CSs should enhance instrumental actions through incentive hope. Without Pavlovian-to-instrumental transfer under incentive hope, we could speculate that the higher strength of goal-directed foraging actions in winter would be difficult to explain because the action-outcome contingency is degraded (effort is less often rewarded) in comparison with the summer period. As long as the paradox between motivation and preference persists (propensity to respond vigorously to an uncertain situation versus preference for a more favorable situation), incentive hope is believed to maintain seeking activity at a high level – not only with respect to Pavlovian approach, but also indirectly to instrumental (goal-directed) actions. To return to the link between motivation and expectations, some colleagues might believe that expectations have a motivational or an emotional component, because we tend to expect events with some motivational or emotional salience (e.g., salary increase, next holidays, a big bill to pay). However, producing expectations is a cognitive process without intrinsic motivational power (modern robots can do that without having to be motivated by their tasks). Thus, expectation-based information gain cannot be attractive per se; it becomes attractive only if incentive properties are (potentially and subcognitively) associated with the information in question. Reducing uncertainty in an apparently useful context is a source of motivation to get those incentive properties.
Gozli & Gao note that uncertainty-induced behavioral invigoration can make the unpredictable more predictable – by shortening the delays for food – and can promote the learning of new action schemes. They see no need to hope for rewards to acquire a new, stable action scheme. We think that, like Pezzulo & Friston, they interpret exploration only as an expectation-based (cognitive) process. Given that an expectation is not motivation, what does motivate the production of new action schemes? There is probably no simple answer to this question. But incentive hope may indirectly contribute to acquiring action schemes through behavioral invigoration, forcing organisms to be more vigilant and to explore vaster spaces for longer durations. In a similar cognitive perspective, Wang & Hayden insist on the importance of mental models in driving behavior and mention that many organisms decide to “forego primary reward to seek information that provides strategic benefits or satisfies curiosity.” We agree, but it is important to know what is meant by “strategic” here. Very often, the strategy in question is the consequence of an evolutionary adaptation of which the organisms may have no awareness. For example, it is noticeable that the decisions made by mammals and birds exposed to free-choice procedures involving a fixed option and a variable option do not differ from the decisions made by bees and wasps in similar situations (Anselme Reference Anselme2018a). The decision to contrafreeload might also depend on incentive properties acting below any conscious control. Wang & Hayden also suggest that, in their attempts to build mental models, organisms sign-track when learning is incomplete and goal-track when learning is complete. However, learning mechanisms have been shown to be hardly plausible to account for the sign- versus goal-tracking distinction, whether uncertainty is present or absent (Anselme Reference Anselme2016; Berridge Reference Berridge2012). Current data indicate that sign- and goal-trackers have similar learning performance relative to their own CS (Meyer et al. Reference Meyer, Lovic, Saunders, Yager, Flagel, Morrow and Robinson2012). Also, if goal-tracking reflects complete learning, why are goal-trackers sometimes observed under reward uncertainty (Gottlieb Reference Gottlieb2005)? In fact, sign-trackers appear to have low top-down (cognitive) control compared with goal-trackers, and their behavior simply reflects the strength of their cue-triggered motivation in Pavlovian autoshaping (e.g., Kuhn et al. Reference Kuhn, Campus, Flagel, Tomie and Morrow2018; Meyer & Tripi Reference Meyer, Tripi, Tomie and Morrow2018; Paolone et al. Reference Paolone, Angelakos, Meyer, Robinson and Sarter2013).
R1.3. Social behavior
In the target article, we focused our attention on the environmental incentives of foraging to the detriment of the social dimension, which is significant in some species. Matsushima, Amita, & Ogura note that the presence of competitors increases unpredictability in the access to food and increases foraging effort, as the incentive hope hypothesis predicts. However, they report that dopaminergic manipulations in the substantia nigra – which projects to the medial striatum – in chicks have no effect on socially facilitated foraging effort (Ogura et al. Reference Ogura, Izumi, Yoshioka and Matsushima2015). We could also add that the increase in foraging effort seems to depend only on the mere perception of a congener, not on competition for food: When two chicks are separated by a Plexiglas panel, the social facilitation of foraging effort occurs irrespective of whether the feeders at the two ends of an I-maze are separated or shared (Ogura & Matsushima Reference Ogura and Matsushima2011). These findings are incompatible with what the incentive hope hypothesis would apparently predict. But we see two reasons why social facilitation might not require incentive hope. First, there is possibly no paradox between motivation and preference here, because of the presence of secondary benefits associated with social foraging. In other words, an animal could increase its foraging effort within a group of congeners and also prefer foraging in group because of dilution and confusion effects and other predator-related advantages (e.g., Fernández-Juricic et al. Reference Fernández-Juricic, Siller and Kacelnik2004; Roberts Reference Roberts1996; Vásquez & Kacelnik Reference Vásquez and Kacelnik2000). Second, the presence of a congener might motivate behavior differently from food. For example, farm animals such as pigs tested under a progressive-ratio schedule are not ready to deploy a lot of effort to obtain access to a congener, while effort is adjusted to demand with respect to food (Keeling & Jensen Reference Keeling, Jensen and Jensen2002). The observed decrease in activity in the nucleus accumbens in the presence of a competitor supports this view (Amita & Matsushima Reference Amita and Matsushima2014).
Social facilitation is a good example to illustrate that an increase in behavior under uncertainty does not necessarily call for an explanation in terms of incentive hope. We have provided other examples related to ratio and delay schedules in the target article; our concept is most likely to be useful when a motivation-preference (or behavior-preference) paradox exists. Sometimes, however, incentive hope could be recruited but remain behaviorally silent because of interfering factors. Misiak et al. say that the incentive hope hypothesis could explain food wasting in industrialized countries, in which food is abundant – presumably because abundance makes hoarding unnecessary. However, they indicate that hunter-gatherers such as the Hadza (Tanzania), the Maasai (Tanzania), and the Yali (West Papua) do not hoard or waste food – they share it in case of surplus. The authors argue that factors such as mobility, feeding of domesticated animals, and social visibility play an important role in explaining the absence of food hoarding and of food wasting among hunter-gatherers. We fully agree and would like to add that, even though incentive hope is expressed during hunting and gathering, its effects could be unobservable given a population's lifestyle and social norms. This might be different in sedentary populations of hunter-gatherers, known to store food (e.g., Testart et al. Reference Testart, Forbis, Hayden, Ingold, Perlman, Pokotylo, Rowley-Conwy and Stuart1982).
R1.4. Human obesity
Another context in which incentive hope might have limited effects is human obesity, which essentially depends on the abundance and cheapness of food rather than uncertainty in Western countries (Pool; Zack). In most cases, human obesity is a consequence of incentive salience alone, even though incentive hope could have an impact on poor women living in high-income countries (see Nettle et al. Reference Nettle, Andrews and Bateson2017). Furthermore, we recognize possible inconsistencies in the assessment of food insecurity in humans and nonhumans and also the existence of human-specific characteristics of feeding behavior (Pool). People could also lack information on low- and high-calorie foods in modern environments (Almiron-Roig, Pastor, Martínez, & Drewnowski [Almiron-Roig et al.]). However, it is important to point out that food attraction, seeking, and consumption are under the control of homologous brain structures in all mammalian species, including humans. Many people probably know that a hamburger is more caloric than a salad, but they prefer consuming hamburgers because of deep-rooted evolutionary biases leading us to find energy consumption delightful (Pankseep Reference Pankseep1998). Putting on an equal footing our modern feeding habits and adaptive strategies we have inherited for hundreds of thousands of years is somewhat unconvincing. The main difference between our Pleistocene ancestors and us is that we are exposed to needlessly high caloric food, causing overeating and maladaptive fat storage. With respect to food seeking, it is also interesting to note that anthropological studies have revealed common features in many animal species and human hunter-gatherers (Bettinger et al. Reference Bettinger, Garvey and Tushingham2015; Raichlen et al. Reference Raichlen, Wood, Gordon, Mabulla, Marlowe and Pontzer2014). Almiron-Roig et al. are right to point out that poverty is not the unique cause of human obesity; genetic and neurodevelopmental factors are crucial for the development of obesity in people with a high socioeconomic status (SES). But those factors are likely to act as modulators of cue-triggered “wanting” for food-related stimuli, just as abundance, cheapness, and uncertainty do (e.g., Robinson et al. Reference Robinson, Burghardt, Patterson, Nobile, Akil, Watson, Berridge and Ferrario2015b; Soussignan et al. Reference Soussignan, Schaal, Boulanger, Gaillet and Jiang2012).
R2. Phenotypic traits leading to maladaptive behavior
R2.1. Sign-tracking and pessimism
According to Zack, our hypothesis implies that sign-tracking, but not goal-tracking, is the adaptive phenotype. Sign-tracking is indeed supposed to be the adaptive phenotype, but only in an environment in which food is scarce, because sign-trackers should explore and inspect the surroundings more than goal-trackers. However, although there is a genetic basis for these two behaviors (e.g., Flagel et al. Reference Flagel, Robinson, Clark, Clinton, Watson, Seeman, Phillips and Akil2010), their expression is not immutable (Meyer et al. Reference Meyer, Cogan and Robinson2014), and uncertainty seems to favor the sign-tracking phenotype (Robinson et al. Reference Robinson, Anselme, Suchomel and Berridge2015a). Thus, many individuals could be sign-trackers under harsh environmental conditions.
Persistent harshness could therefore lead to a pathological propensity to track CSs. Pool & Sander claim that incentive hope through the study of sign-tracking in humans could help in understanding overeating, addiction, and pathological gambling. Many studies have already revealed the importance of cue-triggered “wanting” in drug and behavioral addictions. But it is a fact that not much attention has been paid to the role of unreliable cues in addictions, except with respect to gambling, of course. It would also be worth studying how incentive hope could affect anxiety and obsessive-compulsive disorders and could lead people to recover from depression.
In this respect, Linkovski, Weinbach, Edelman, Feldman, Lotem, & Kolodny (Linkovski et al.) suggest that adversity-predictive stimuli in the environment activate a pessimistic phenotype based on anticipated hardship, which could be at the origin of many pathologies in modern humans. Importantly, they argue that the trigger of incentive hope is anticipated hardship rather than uncertainty itself. This idea is interesting and should be pushed further. For that, however, some clarifications are needed. First, by uncertainty (or unpredictability) we mean that an organism cannot predict whether the next foraging or search trial will be rewarded or nonrewarded. The probabilistic definition of maximal uncertainty (50% chance of reward) is narrow and applicable only to Pavlovian autoshaping. In the wild, the probabilistic and temporal aspects of uncertainty are inseparable, the former having possibly more local (CS-related) effects than the latter (Hulme & Kvitsiani). Is our definition of uncertainty radically different from Linkovski et al.’s hardship? In our understanding, hardship simply means that finding rewards is difficult; that is, a significant number of attempts are unpredictably successful because of variability in the delays for food and in the reliability of its predictive CSs. Here, it is important to remember the definition of incentive hope: motivational excitement for rewards when non-rewards are likely, which suggests that uncertainty and hardship are relatively equivalent terms. Second, anticipated hardship is acceptable in a Pavlovian context only if “anticipated” refers to the predictive value of a stimulus independent of any cognitive assessment, for example, paramecia can learn Pavlovian associations while having not a single cognitive processing system (Hennessey et al. Reference Hennessey, Rucker and McDiarmid1979). In other words, a 50% chance of reward in autoshaping must be learned as an implicit regularity rather than an explicit rule. The implicit predictive value of a CS is likely to contribute to incentive hope (as “wanting” × uncertainty), given that exposing people to adversity-related words or simulated low SES is sufficient to alter nutritional desires (Cheon & Hong Reference Cheon and Hong2017; Laran & Salerno Reference Laran and Salerno2013). Thus, anticipated hardship should somehow be related to this implicit predictive value, but not to motivational salience or “wanting.” In consequence, the direct connection between incentive hope and pessimism suggested by the authors is unclear: Pessimistic individuals should be hopeless because of their high expectation of non-rewards (no uncertainty) and their apparent lack of motivation (no “wanting”), and hence not much inclined to seek and explore. In this view, we could see incentive hope as a sort of compensatory motivational response to anticipated hardship responsible for pessimism. But trying to replace the stimuli that cause anticipated hardship by others, which will activate a more optimistic phenotype – a strategy that should require no incentive hope – appears to be an excellent suggestion from these authors to cure a number of human psychopathologies.
R2.2. Pathological gambling
While recognizing the relevance of the incentive hope hypothesis in different contexts, two authors suggest that it might be inappropriate to account for pathological gambling (Robinson; Zack). One criticism is that gamblers seek money at casinos, but they do not hoard it – contrary to small birds storing fat or food items in winter (Robinson). In particular, slot machine players seem to play so they can obtain more coins to continue gambling. The pathological dimension of gambling could possibly alter the adaptive logic behind incentive hope. But we would like to note that hoarding (or fattening) does not necessarily follow seeking in nature; as shown in our computer model, organisms need the opportunity to accumulate reserves. In the case of gamblers, especially slot machine players who can place a lot of bets within short periods, the losses largely overcome the wins so that hoarding is virtually impossible. Of course, in comparison with recreational gamblers, it is likely that pathological gamblers would totally re-engage their considerable wins, which may occasionally occur. But even such big wins would probably not compensate for the money amounts previously placed. In this view, gambling represents an instance of foraging under extreme conditions, in which hard-earned coins must immediately be re-engaged to stay in the game. In foraging animals, that would mean that the food items found provide just enough energy to find new items and stay alive. Of course, we are not suggesting that gamblers chasing losses activate simple Pavlovian responses only (Zack). We simply mean that complex behaviors can, at least in part, result from basic processes. For example, “wanting” is not just a concept for rats in autoshaping boxes but is also likely to be at the origin of our human conscious desires (Anselme & Robinson Reference Anselme and Robinson2016). The same reasoning applies to incentive hope with respect to gamblers.
In our target article, we argue that uncertainty is not attractive in itself and, thus, is not preferred to certainty (Anselme Reference Anselme2018a; Kahneman & Tversky Reference Kahneman and Tversky1979; McDevitt et al. Reference McDevitt, Dunn, Spetch and Ludvig2016). Therefore, it may seem difficult to understand why gamblers take pleasure in recurrently going to a casino (Robinson; Zack). Is it because they can choose between going and not going to a casino? Why do they decide to spend their time and money there? First, it is important to realize that this choice is different from that made by individuals in an experimental context, where they have to decide between an uncertain reward and the same reward for sure. In contrast, the choice for gamblers is between uncertain money (casino) and no money for sure (e.g., staying at home) or losing money for sure (e.g., going to the cinema). Second, at least with respect to pathological gamblers, it is not certain that they have the opportunity to quit. Like many drug addicts, they may repeatedly try to avoid the events that cause their addiction without success. Third, given that uncertainty sensitizes dopamine neurons in the ventral striatum (e.g., Mascia et al. Reference Mascia, Neugebauer, Brown, Bubula, Nesbitt, Kennedy and Vezina2019), we agree that gambling activity is somehow desirable and, by extension, pleasurable. As said, uncertainty does not seem attractive per se. But casinos are confined environments full of attractive predictors of (uncertain) rewards that stimulate incentive hope as our ancestral motivation to forage (Anselme Reference Anselme, Tomie and Morrow2018b). Organisms are likely to find it desirable and pleasurable to express the behaviors for which they are adapted. A good example is that of caged animals that normally live and forage over a large range in the wild: Their confinement within small, impoverished captive environments causes the development of behavioral pathologies, such as stereotypies, despite receiving their food for free (see Eilam et al. Reference Eilam, Zor, Szechtman and Hermesh2006). Chasing is probably desirable and pleasurable for a lion, even if this often involves deploying a lot of effort for nothing. Similarly, gambling could generate positive feelings because this activity recruits seeking motivation and behaviors for which our Pleistocene ancestors developed adaptations (Anselme Reference Anselme, Tomie and Morrow2018b).
R3. Presuppositions in our computer model
Several contributors point out some limits of our definition of uncertainty. We will not repeat here our general definition, recalled earlier (see R2.1), but focus on more specific aspects. For example, Hulme & Kvitsiani describe a situation involving two patches that apparently contradicts our prediction: if patch #1 is unpredictable over short timescales (p = 0.5) but predictable over longer timescales, and patch #2 is predictable over short timescales (p = 0.9) but unpredictable over longer timescales, an animal should spend more time and energy seeking in patch #2 but should prefer patch #1. However, deciding which patch is more stimulating and which patch is preferred is rather difficult if we do not know how often the CSs are found. These authors define uncertainty only in probabilistic terms, while we consider two forms – one about CS reliability and one about delay shortness. For example, a CS-food association can be very frequent (p = 0.9) but the CSs scarce – any other combination also being possible. Our hypothesis predicts that delay uncertainty might have a stronger impact on the motivation to forage than probabilistic uncertainty.
Symes & Wheatley argue that the distribution of food in the environment is not random but patchy, so that one item found increases the probability of encountering other items. We agree that food patches may exist, although they are not as common as sometimes postulated (Arditi & Dacorogna Reference Arditi and Dacorogna1988). But, contrary to a widespread presupposition since Charnov (Reference Charnov1976b), food is not uniformly distributed within a patch; an animal is often forced to seek food items despite their relative abundance. Thus, we think that our argumentation remains unchanged. Our hypothesis and computer model describe trial-level randomness such as it could exist within a patch (e.g., a clearing), where some food items (e.g., bugs) are necessarily present, but their exact location unknown.
Just as we did not specify the size of our virtual environment, we did not allow uncertainty to fluctuate over time (Houston & Malhotra). The reason is that the model is assumed to measure phenomena occurring within relatively short intervals – no more than few hours. We can reasonably postulate that uncertainty remains stable under these conditions. Hulme & Kvitsiani rightly note that our model did not try to capture the inferential controllability of the environment (see also Gozli & Gao; Pezzulo & Friston; Wang & Hayden). Assuredly, expectations play an important role in animal foraging – at least in “higher” vertebrates. But here we had to demonstrate how motivation as a subcognitive process could affect foraging, without having to disentangle its effects from those of cognitive expectations. In winter, the high expectation of non-rewards on some days may lead a bird to reduce its activity, and our hypothesis predicts that this decision results from the absence of incentive hope for the opportunity to find edible items. Energy resources (fat reserves) are therefore conserved for later use (Alquist & Baumeister).
We agree with Houston & Malhotra, who suggest that our model is not adequate when applied to laboratory experiments, especially with respect to risk-sensitive foraging theory. Although we explicitly refer to temporal discounting to explain sensitivity to delays (a mechanistic rather than functional interpretation), this dimension is absent in the computer model and should somehow be included in subsequent research. It is also true that our model does not take choice and food amounts into account. Indeed, based on some findings, we postulated that wild animals encounter food items sequentially (they do not have to make choices; Shapiro et al. Reference Shapiro, Siller and Kacelnik2008) and are indifferent to variability in food amounts (variable, higher amounts do not stimulate sign-tracking more than fixed, lower amounts; Anselme et al. Reference Anselme, Robinson and Berridge2013). In fact, for reasons developed in the target article, choice between fixed and variable food amounts should not depend on incentive hope. Imagine that the fixed and the variable food amounts are equivalent on average. Because variability is different from real uncertainty (some food is always provided) and is not associated with any advantage (such as a shorter delay or more reliable CSs), the two options are likely to be preferred in a similar fashion, as often observed (Kacelnik & Bateson Reference Kacelnik and Bateson1996). In other words, incentive hope plays no role, and incentive salience is equivalent in both cases. Now, if both options are not equivalent on average, the most profitable should be more attractive and preferred, in accordance with the incentive salience hypothesis (e.g., Real et al. Reference Real, Ott and Silverfine1982).
It is true that our equations are not fully justified in functional terms and should contain more factors whose fluctuation is dynamical rather than fixed in advance (Hulme & Kvitsiani). In the future, models based on a refinement of our ideas would be useful to examine the psychological underpinnings of animals foraging in poor environments in the absence of cognitive control. For example, Chmait, Dowe, Green, & Li propose a different approach to food predictability (large food amount in a specific location), which totally avoids any risk of starvation for a while once the food is detected. Their model shows that finding unpredictable resources requires more exploration and lesser learning capacities than finding predictable resources, as our hypothesis would predict.
R4. New horizons for incentive hope
Several commentators suggest that the concept of incentive hope describes a phenomenon that is, in fact, more general than considered in our target article. We focused our reflections on food seeking in birds and mammals because it is in these zoological groups that a parallel was logical to establish with Pavlovian autoshaping, for which the incentive hope hypothesis was initially formulated (Anselme Reference Anselme2015a). But, given that incentive hope denotes a core (unconscious) psychological process rather than a mental (conscious) state, it can potentially be applied to other zoological groups and to non-food-related contexts, as shown below. Alquist & Baumeister provide convincing evidence for mental energy conservation and alertness in humans having to deal with uncertain events. Like the preservation of fat reserves through activity reduction in small birds, conservation of mental energy may prepare people to cope with adversity. Energy conservation (whether physical or mental) is not a consequence of incentive hope, but may be necessary for energy accumulation (whether measured as fat reserves, experience, self-confidence, etc.) that results from enhanced performance in response to unfavorable events through incentive hope. We agree with Alquist & Baumeister when they postulate the existence of a “broadly adaptive default response to uncertainty.” A non-food-related illustration of this idea could be religious faith, which is widespread and strong in countries with low per-capita incomes resulting from persistent existential insecurity and infrequent sources of gratification (Norris & Inglehart Reference Norris and Inglehart2011). People experiencing adversity may come to endure the situation rather than trying to overcome the encountered difficulties (Oettingen & Chromik Reference Oettingen, Chromik, Gallagher and Lopez2018) – in Alquist & Baumeister's terms, we could see the ability to endure as a way of conserving mental energy for other tasks. However, people rationally invest much of their time and energy in activities – such as prayer and attendance to religious services – that they perceive as more likely to be rewarding.
Another consequence of recurrent feelings of uncertainty and insecurity in low-SES people is the expression of greater impulsivity in their decision making (Pepper & Nettle Reference Pepper and Nettle2017). Accordingly, Walsh, Cheries, & Kwak show that experimentally reducing those feelings in low-SES people decreases their propensity to be impulsive. As they note, this phenomenon is compatible with the incentive hope hypothesis, which predicts that scarce resources will generate a hope for short delays and reliable CSs, causing invigoration in the activity of seeking rewards. Based on this idea, they envision an interesting therapeutic perspective in reducing impulsivity in low-SES people. We add that, as in the case of psychopathologies (Linkovski et al.), incentive hope is not in itself responsible for maladaptive behaviors. Hope is associated with positive effects for many people (e.g., Snyder Reference Snyder1994). But hoping for unrealistic achievements in long-lasting unfavorable contexts, such as experienced by low-SES people, could be psychologically damaging. More secure or predictable environmental conditions (in terms of health, employment, governmental effectiveness, etc.) avoid keeping hopes up for nothing and should contribute to avoiding a number of irrational behaviors with potential pathological consequences (Clark).
Aharonov-Majar & Suleiman suggest extending incentive hope to human social contexts, reporting that people tend to form groups when variability in the chance of reward is high, because the redistribution of gains is a more profitable strategy than solitary gambling. They also report that making multiple gambles (increasing foraging) is a way of reducing uncertainty correlated with solitary gambling. These ideas fit well our interpretation of incentive hope as a mechanism of uncertainty reduction working in the absence of cognitive strategies. The psychological underpinnings of group formation are certainly diverse and unrelated to incentive hope in many cases. But we could imagine that this mechanism of the brain is sometimes recruited to motivate collective behaviors against environmental uncertainty – a positive effect of sociality also noted in bees (Schmickl & Crailsheim Reference Schmickl and Crailsheim2004). These commentators mention that the neuropeptide oxytocin is important in eliciting trust and cooperation in humans. Could uncertainty reduction in a social context emerge from cooperation, in which case incentive hope appears unnecessary? Or does cooperation itself require incentive hope, because cooperation is based on a desire to reach a goal without a guarantee that the goal will actually be reached or even that the others will collaborate? After all, if oxytocin reinforces trust and cooperation in general, it does not explain why we form groups to struggle against adversity. The answers to these question require further research. Incentive hope could have been particularly useful in hunter-gatherer societies because, as Aharonov-Majar & Suleiman point out, collective foraging combines foraging effort (maximizing food sharing) with decreased or unchanged body fat (caused by food sharing). In a sense, collective foraging and consumption would have functional consequences similar to those of hoarding behavior: increasing the available resources without having to store them directly in the body.
Tops proposes an integrative neurobiological view in which incentive hope could share neuronal systems with apprehension, explaining why uncertainty can motivate individuals. This view is in line with our suggestion that incentive hope – a dopamine-dependent motivational process – requires the release of glucocorticoids, which are typically involved in stress responses. We left open the question of whether other neurotransmitters or neural circuits play a role in its expression as well. Discovering brain correlates of behavior is important, but our main interest has been to explain behavioral facts. We need psychological concepts describing precisely what animals and humans do. For example, we can describe the brain activity of a rat running away from an aversive CS, but we can only understand what the rat is doing through the concept of fear. Incentive hope is conceptually compatible with the idea of an overlap between its neurobiological mechanisms and those of apprehension. But identifying neuronal processing systems does not eliminate the need for concepts to possibly shed light on distinct psychological components correlated with these systems (Smulders et al.). A good example is the idea that uncertainty might generate an error prediction signal in the brain that interrupts routinized responding (Tops) and favors learning (Schultz Reference Schultz1998): Understanding this process is difficult without an appropriate motivational concept. Interestingly, Laurent et al. (Reference Laurent, Balleine and Westbrook2018) recently found that a rat's motivational state determines the positive prediction error signal for appetitive excitors; motivation is required to generate the error on which learning is subsequently computed.
Clark indicates that organisms without a nervous system may respond to unpredictability similarly to taxonomically more recent animals, and, therefore, that incentive hope is appropriate for describing their behavior. Although largely studied in mammals, Pavlovian conditioning has indeed been observed in protozoa (Hennessey et al. Reference Hennessey, Rucker and McDiarmid1979) and risk sensitivity in pea plants (Dener et al. Reference Dener, Kacelnik and Shemesh2016) – aneural organisms. Of course, noticeable differences necessarily exist in the physiological mechanisms involved among species so phylogenetically distant, but it is a fact that most organisms share the same functional purposes – notably approaching rewards and avoiding uncertainty. For example, in the absence of rewards, common woodlice, Porcellio scaber, exposed to random visuotactile patterns show enhanced rearing-up – exploratory – activity than when exposed to regular patterns (Anselme Reference Anselme2015b). Woodlice act as if they hoped for more opportunities to escape when the environment is less predictably unrewarding. In the same vein, there are good reasons to suspect the existence of a “wanting” system in insects, even though octopamine rather than dopamine is sometimes its main brain correlate (Perry & Barron Reference Perry and Barron2013; Søvik et al. Reference Søvik, Perry and Barron2015). Clark also suggests that the concept of incentive hope could explain the origin of irrational behaviors. Incentive hope may certainly lead organisms to take some risks. For example, if an animal's decision to forage harder is unable to compensate for the investment in the task (because food is too scarce), this should cause a rapid loss of energy reserves and a high risk of starvation. Incentive hope may lead people living under precarious conditions to work hard to keep their jobs, but working hard for the jackpot in casinos may cause a rapid loss of considerable sums. It is unclear to us whether incentive hope could be the origin of any kind of irrational behavior, such as suboptimal decision rules identified in the brainless, unicellular, slime mold, Physarum polycephalum (Latty & Beekman Reference Latty and Beekman2011a). But the evidence that unpredictability may distort perception and lead to error-prone decisions in neural and aneural organisms supports the idea of universal decision rules and behaviors in living beings.
In our target article, we suggest that incentive hope is a brain mechanism designed by natural selection because of its ability to reduce starvation risk in an unpredictable environment. Snyder & Creanza think that incentive hope could also have implications for the sexual selection of song in songbirds. Indeed, better-fed offspring will develop better singing, causing greater reproductive success and better transmission of their genes. So, the propensity of the parents to forage harder to feed their offspring increases the offspring's fitness, in addition to increasing the chance that the genes controlling the ability to forage harder will be transmitted as well, and so forth. We like this idea, even though this example might be an isolated case where sexual selection contributes to shape incentive hope. This process should be particularly crucial in poor environments, such as semidesert regions, but also in richer environments for young birds that hatch too early in the year, when insects are not yet abundant. Snyder & Creanza rightly note that our hypothesis does not account for the parental dilemma that consists of having to choose between favoring their own survival and that of their offspring. Answering this question is important, but it would involve a clear understanding of the evolutionary decision rules underpinning animal motivations; such decision rules are likely to be independent of incentive hope per se. According to the selfish gene hypothesis (Dawkins Reference Dawkins1976), genes rather than the individuals (their vehicle) attempt to maximize their survival, that is, their propagation over generations. If correct, we could speculate that genes can favor the parents or their offspring (which have half of their genes) depending on the most likely adaptive effectiveness of one or the other strategy. Although this explanation does not provide any genetic or neuronal mechanism, it suggests that genetic control could orient incentive hope toward distinct beneficiaries according to circumstances.
R5. Conclusion
Interdisciplinary articles may easily lead readers to various misunderstandings of what the authors attempt to say, but we were delightfully surprised to see that this was not the case here. Few commentators think that incentive hope is not useful to account for behavior under reward uncertainty, giving us the opportunity to clarify and refine our arguments. However, most commentators perceived the interest of our hypothesis and even tried to extend it to issues not discussed in our target article – such as social foraging, decision making in invertebrates, and sexual selection in songbirds. Although more facts have to be collected about predictions of the incentive hope hypothesis, many commentaries contributed to amplifying and diversifying the applications of our hypothesis beyond food seeking in “higher” vertebrates, including original therapeutic approaches for psychopathologies and maladaptive behaviors possibly associated with too much insecurity in life. They also contribute to suggesting the existence of a common, ancestral mechanism put in place by evolution, allowing very distinct organisms to deal with uncertain, significant events.
Target article
How foraging works: Uncertainty magnifies food-seeking motivation
Related commentaries (22)
A neural basis for food foraging in obesity
Beyond uncertainty: A broader scope for “incentive hope” mechanisms and its implications
Beyond “incentive hope”: Information sampling and learning under reward uncertainty
Complex social ecology needs complex machineries of foraging
Considerations for the study of “incentive hope” and sign-tracking behaviors in humans
Does the “incentive hope” hypothesis explain food-wasting behavior among humans? Yes and no
Extending models of “How Foraging Works”: Uncertainty, controllability, and survivability
Food security and obesity: Can passerine foraging behavior inform explanations for human weight gain?
Food seeking and food sharing under uncertainty
Food-seeking behavior has complex evolutionary pressures in songbirds: Linking parental foraging to offspring sexual selection
Foraging extends beyond food: Hoarding of mental energy and information seeking in response to uncertainty
Hoarding all of the chips: Slot machine gambling and the foraging for coins
Hope, exploration, and equilibrated action schemes
How uncertainty begets hope: A model of adaptive and maladaptive seeking behavior
Mechanistic models must link the field and the lab
Overlapping neural systems underlying “incentive hope” and apprehension
Random isn't real: How the patchy distribution of ecological rewards may generate “incentive hope”
Simulating exploration versus exploitation in agent foraging under different environment uncertainties
The value of uncertainty: An active inference perspective
Unpredictable homeodynamic and ambient constraints on irrational decision making of aneural and neural foragers
“How Foraging Works”: Let's not forget the physiological mechanisms of energy balance
“Incentive hope” and the nature of impulsivity in low-socioeconomic-status individuals
Author response
Incentive hope: A default psychological response to multiple forms of uncertainty