R2. Clarifying our theoretical positioning

Some commentaries (Moors; Sheldon & Ryan; Ozgan & Allen) rejected our theoretical position on the grounds that it is a reductionism and, therefore, prima facie untenable. We find it useful to respond to these comments first in order to clarify our theoretical positioning in the philosophy of mind. First of all, reductionism encompasses a wide range of perspectives and should not be dismissed solely on the basis of being reductionist. One extreme version of reductionism is reductionist atomism, which believes that the only scientific way to understand complex phenomenon is to analyze it into its component parts (Sawyer, Reference Sawyer2002). Put in the context of our article, this position aims to replace or eliminate the constructs of motivation (i.e., higher-level explanation) by introducing mental computational processes (i.e., lower-level explanation). Sheldon & Ryan and Ozgan & Allen seem to interpret our proposal as the commitment to this extreme version of reductionism. But this is not our position (although we admit that some sentences in our target article were misleading with this regard as noted by Elliot & Sommet). Our point is that theories have long taken motivation constructs for granted in the past and did not make sufficient effort to dig into the mental computational processes underlying motivated behavior. This is explicitly stated in the target article: “No level of understanding should be dismissed as ‘wrong’ (i.e., one level of explanation should not be replaced with a lower-level explanation … but the problem of motivation literature is that most researchers are satisfied with higher-level explanations (i.e., supposing high-level motivation constructs to explain behavior) and little effort has been made to pursue lower-level explanations” (sect. 5.2).

At the same time, our position is, in fact, partly compatible with some forms of reductionism, because we argued that the function of motivation constructs (i.e., to cause behavior) is realized through lower-level mental computational processes. But this is not a controversial argument by itself – no scientist today doubts that everything in the mind is realized somehow through the brain and neural activities. If we accept this, it is unwarranted to dismiss our standpoint as reductionist. Importantly, if one rejects this standpoint by calling it reductionism, this could lead to holism, which holds that higher-order explanations are completely independent from lower-level explanations. This perspective is akin to dualism and does not have clear logical coherence in the current philosophy of mind (Sawyer, Reference Sawyer2002).

To avoid holism, one must accept that higher-order processes emerge from lower-level units in some way, and we are arguing that a better understanding of the higher-level motivation constructs can be gained by investigating these lower-level processes. And there is more benefit than scientific knowledge and understanding: Better understanding the possible mechanisms of such a process can help to design better interventions. A helpful analogy may be drug trials. For many drugs, the process via which they have effects is a black box. Despite that, pharmaceutical medication is used to help people and save lives. But if we had a better understanding of the processes that lead to the drugs' actions, we could create even better medication that would have greater effects and help more people.

Another point of clarification is that, while we agree that both levels (i.e., motivation constructs and mental computational processes) of explanation are important, it is the mental computational processes which directly cause so-called motivated behavior. Higher-level motivation constructs have a lot of utility to predict behavior (which we acknowledged in the target article) but are not the direct cause. Jurjako positioned such a position of ours as mental fictionalism in philosophy of mind (Toon & Toon, Reference Toon and Toon2023) or mental modelism (see Crane & Farkas, Reference Crane, Farkas, Demeter, Parent and Toon2022), which we appreciate and agree with. According to this standpoint, motivation constructs can be considered as a useful fiction or a hypothetical model in that they do not have direct causal effects on behavior but are used in daily narratives to explain causal effects. They also have, however, a significant role in theorizations, practical interventions, and understanding of human behavior in daily life settings. Jurjako used the equator, the average person, and the ideal gas law as examples of such mental fictions or models. This is critically different from the assertion that motivation is a post-hoc rationalization or epiphenomenon, void of meanings (e.g., Nisbett & Wilson, Reference Nisbett and Wilson1977).

Scarantino made a distinction between the inferences (or psychological construction) made by ordinary people and scientists. He argued that motivational constructs are the inferences made by scientists or experts, not by ordinary people; therefore, they are useful as theoretical constructs. We actually agree (and we acknowledge that our example using ordinary people was misleading). He criticized that we did not provide any evidence that motivational constructs do not allow for interesting psychological generalizations, but we did not do so in the target article because we do believe that motivational constructs allow for generalizations and make useful theoretical predictions (see sect. 3.1 in the target article). But this is different from the assertion that motivation constructs have a direct causal effect. On this point we disagree. Scarantino stated “motivation concepts are plainly causal” by showing that intervening on higher-order motivation constructs changes behavior. Sheldon & Ryan also argued that the causal effects of needs are empirically demonstrated by interventions. However, even with interventions it is extremely challenging to conclude that the target constructs (e.g., need for autonomy) have a causal effect, when the target constructs are broad and not well-specified such as higher-order motivation constructs (Bailey et al., Reference Bailey, Jung, Beltz, Eronen, Gische, Hamaker and Murayama2024). We can certainly establish the causal effects of intervention itself (e.g., attempts to change teachers' autonomy support behavior) and the outcome, but it is a much harder job to demonstrate that the target motivation constructs intervene on the effect (see also Eronen, Reference Eronen2020).

Moors also indicated that our standpoint is reductionism in that we are trying to replace higher-order motivation constructs or goals with lower-order ones. Again, we do not. She takes the position that goals or motivation are hierarchically organized and questioned our proposal by indicating that higher-level goals play an important role in understanding lower-level goals or actions. Interestingly, Dubourg, Chambon, & Baumard (Dubourg et al.) brought in a similar idea of goal/motivation hierarchy from evolutionary psychology, but they saw it as consistent with our proposal. More specifically, they indicated that evolutionary psychology has seen it critical to specify lower-level variables in a way that is consistent with higher-level motivation. Our positioning is closer to Dubourg et al. Motivation constructs are hypothetical in that they are posited to conveniently explain patterns of behavior and subjective experiences, but they are still informative and play a critical instrumental role to think about mental computational processes. They are not completely detached in this regard – they inform each other.

However, one critical deviation from these “hierarchical models” (Heckhausen & Rheinberg; Dubourg et al.; Sheldon & Ryan; Elliot & Sommet; Del Giudice) is that we do not think these high-level “fundamental motivation” constructs (achievement motivation, affiliation motivation, etc.), which are allegedly created through evolution, have a top-down causal influence on mental computational processes and, thus, behavior. It may look like they do, but we do not need to think that way. It is sufficient to suppose that mental computational processes went through evolutionary processes. People show a broad range of social behavior not because evolution shaped a central motivation system ordering us to be social in general – it is these behaviors (and the processes that caused the behaviors) which were shaped by evolution. We are increasingly convinced by the idea based on recent accumulating neuroscientific literature suggesting that there is no single fixed brain area or system that is dedicated to a particular type of motivation or emotion (e.g., Meliss, Tsuchiyagaito, Byrne, van Reekum, & Murayama, Reference Meliss, Tsuchiyagaito, Byrne, van Reekum and Murayama2024; Pessoa, Reference Pessoa2017). If there is a top-down signal that orients various types of specific goal-directed behavior in a particular manner (e.g., in a manner that makes the organism competent), where does that come from? (see also commentaries by Wurm, van der Ham, & Schomaker [Wurm et al.] for the importance of considering the constraints from the neural data). At least for now, we do not see good evidence of such non-specific, top-down signal of high-level motivation constructs.

Related to the comments on reductionism, André & Baumeister expressed a concern that we are attempting to reduce motivation to cognition. Other commentaries also equated mental computational processes with cognition (e.g., Moors; Sheldon & Ryan; Heckhausen & Rheinberg; Eccles & Wigfield). In fact, the history of motivation research has often been portrayed as the tension between motivation and cognition (Bem, Reference Bem1967; Weiner, Reference Weiner1991), forming the sentiment that we should separate motivation from cognition. André & Baumeister especially indicated that motivation cannot be described by mental computational processes. Specifically, motivation is something inside us which makes rewards appealing; on the other hand, rewards cannot be appealing to computers because computers, as non-sentient objects, lack motivation. We are not sure if such a hard dichotomy helps us understand human functioning as a whole – the distinction between motivation and cognition is useful in many cases but is often blurred and creates unnecessary constraints in our explanation (Murayama, Reference Murayama, Bong, Kim and Reeve2022b). Unless we take the position of dualism (which is generally rejected by scientific consensus and philosophy of mind), mental computational processes must logically mediate appealing feelings (i.e., a rewarding feeling). We can already observe some integration of mental computational processes and motivational properties in the literature: For example, “incentive salience” describes the feeling that something is rewarding and desirable (Bindra, Reference Bindra1974; Bolles, Reference Bolles1972), and researchers have investigated its computational basis (Berridge, Reference Berridge2023). And again, it is not the motivation itself which is shaped by evolution – we feel certain stimulus rewarding because evolutionary processes programmed us to feel that way. In fact, in research of information-seeking, there are some attempts to computationally understand why we feel particular stimuli/situations as appealing via simulations (e.g., Giron et al., Reference Giron, Ciranka, Schulz, van den Bos, Ruggeri, Meder and Wu2023; Gruaz, Modirshanechi, & Brea, Reference Gruaz, Modirshanechi and Brea2024) or rational analysis perspectives (Dubey & Griffiths, Reference Dubey and Griffiths2020; see also Ten, Oudeyer, Sakaki, & Murayama, Reference Ten, Oudeyer, Sakaki and Murayama2024).

R3. Are there already many theories focusing on mental computational processes?

Several commentaries posited that motivation theories focusing on mental computational processes already exist, and are even the core agenda in motivation science (Custers, Eitam, & Higgins [Custers et al.]; Richter & Gendolla; Wright, Sciara, & Pantaleo [Wright et al.]; Heckhausen & Rheinberg; Eccles & Wigfield). Some authors are especially explicit in this regard: Heckhausen & Rheinberg stated “The good news is, there is no black-box problem and no need to reinvent the wheel,” and Richter & Gendolla claimed “Motivation science has always looked at filling the ‘black box’ by specifying constructs and explaining how behavioral tendencies are generated.”

We acknowledge that most of the examples suggested by these authors have tried to explain motivated behavior without resorting to the concepts of need or motivation. We appreciate the effort of these researchers to remove high-level concepts from explanation and the impact that their research has made in motivation science. However, we feel that many of these examples do not sufficiently address what we aspired for in the target article.

To respond to these commentaries, we first clarify what we meant by mental computational processes. As some commentaries correctly pointed out (Wurm et al.; Jurjako; Scarantino), we had Marr's (Reference Marr1982) three level of analysis in mind when discussing mental computational processes. Marr proposed that, when analyzing the capacity of a system, we have three levels of questions: (1) The computational level, which asks what is the function/nature of the system, (2) the algorithmic level, which asks what are the processes by which the function is computed, and (3) the implementation level, which asks what is the physical realization of the process (van Rooij & Baggio, Reference van Rooij and Baggio2021). By mental computational processes, we meant the algorithmic level.Footnote ¹ That is, beyond what it does (i.e., computational level), it should explain how it achieves what it does. Adopting the example by Cummins (Reference Cummins, Keil and Wilson2000; also cited by van Rooij & Baggio, Reference van Rooij and Baggio2021), let us consider a hypothetical psychological construct called multiplication system. The highest, computational level can explain that the function of this system is to multiply numbers. The algorithmic level can say that there are different ways to achieve the function – for example, the multiplication system may use a partial product algorithm or sequential addition. By clever empirical study, we can even tell which process is more likely to operate. For this example, by assessing reaction time for various numbers, we can test these two hypothesized processes realizing the multiplication system because sequential multiplication should take longer as a function of multiplier size.

With this distinction in mind, in our reading of the theories suggested by the commentators, while we acknowledge that these theories provide great insights into mental computational processes, they have two related issues.

First, some theories still rely on the concepts which are described at the computational level in our opinion. For example, several commentators (e.g., Del Giudice; Heckhausen & Rheinberg; Moors; Eccles & Wigfield; Wright et al.; Richter & Gendolla) mentioned various theories which include common terms in motivation research – expectancies, values, or goals (e.g., Atkinson, Reference Atkinson1957; Brehm & Self, Reference Brehm and Self1989; Eccles & Wigfield, Reference Eccles and Wigfield2020; Vroom, Reference Vroom1964). Here one common assumption is that people are more motivated to take action if the action has higher expectancy or value or fits with one's goal. But this simply describes what it does (e.g., if you find X valuable, you will do Y). It does not explain the process underpinning the purpose. As noted in our original manuscript (sect. 5.4), there are many algorithmic level questions we can ask. For example, how does one find something valuable? Perhaps value is not a quantity but takes a form of mental representation – what, then, are the mechanisms of representational change and how can one translate multidimensional representation into subjective feeling of values? For example, (situated) expectancy-value theory (Eccles & Wigfield, Reference Eccles and Wigfield2020) posits that perceived value is formed by socio-cognitive factors (e.g., cultural milieu, goals, other's beliefs and behaviors), but according to our perspective, specifying factors is important but not enough to unravel mental computational processes. The mechanism explaining how a factor influences these constructs is still a black box.

Vassena & Gottlieb provided a nice example in this regard. Effort and cost are critical components in many traditional motivation theories. Motivation intensity theory (Brehm & Self, Reference Brehm and Self1989) argued that effort investment is proportional to the importance of the outcome and the difficulty of the task (Richter, Gendolla, & Wright, Reference Richter, Gendolla and Wright2016). This is a simple but powerful theory explaining a variety of motivated behavior without using the concepts of motivation/needs, but it critically depends on the concepts of task difficulty and outcome importance. However, finding the optimal amount of effort by learning task difficulty and outcome importance is not a trivial job, and we need to specify how people can achieve this. In their computational model (Silvestrini, Musslick, Berry, & Vassena, Reference Silvestrini, Musslick, Berry and Vassena2023), the authors showed that it is critical to incorporate meta-learning mechanisms in the mental computational processes to explain the relevant motivated behavior, especially in complex environments (e.g., environments with volatility). This example illustrates how we can unpack the black box of motivation theories which are not reliant on the concepts of needs or motivation.

Second, even when these theories could be classed as mechanistic, they tend to be underspecified relative to typical algorithmic-level explanations in the literature. For example, several commentators mentioned theories which adopt the basic idea of a goal/motivation hierarchy (Del Giudice; Elliot & Sommet; Moors; Dubourg et al.). For example, Del Giudice's General Architecture of Motivation (GAM) model provides a comprehensive picture of human motivated behavior and takes a hierarchical position: Higher-level motivational systems (i.e., organisms' core biological goals such as physical safety, mating, and offspring care) send emotional signals to (and receive feedback from) a lower-level instrumental goal pursuit system which manages narrower, more specific goals in an open-ended manner. We already provided a criticism of the assumption that higher-level motivation constructs influence lower-level goals in such hierarchical models (see sect. R2). Indeed, these models do not explain where these higher-level goals come from. But let us assume they do.Footnote ² While we agree that such a hierarchical organization represents mechanisms to explain motivated behavior, there are many underspecified parts in the model. How are different levels of goals and actions organized or represented? What kind of information is carried from a high-level goal to the low-level ones? How do these goals constrain each other to produce a single action output? In the reinforcement-learning (reward-learning) literature, hierarchical reinforcement learning (Botvinick, Reference Botvinick2012) provides a nice algorithmic framework to concretely pin down how agents can manage such an action/goal hierarchy; however, we do not yet see an implementation of similar frameworks in the motivation literature.

A similar point can be made for the whole trait theory (Fleeson & Jayawickreme, Reference Fleeson and Jayawickreme2015) suggested by Ratchford & Jayawickreme. The theory assumes that environmental and cognitive factors as well as individual differences in how they are processed result in certain specific distributional patterns of affective, behavioral, and cognitive states. The theory suggests that these specific distributional patterns give rise to what we call traits (Fleeson & Jayawickreme, Reference Fleeson and Jayawickreme2015). It is indeed consistent with our perspective that the theory treats traits simply as patterns of various states and indicates that they do not directly cause behaviors (although they also seem to treat traits as real on some occasions, which is reflected in some phrases like “trait enactment”). At the same time, how these environmental and cognitive factors interact remain unspecified. It is this process that our proposal called for in our target article; and promisingly, the investigation of these factors is now an emerging area in personality psychology (e.g., Horstmann, Rauthmann, Sherman, & Ziegler, Reference Horstmann, Rauthmann, Sherman and Ziegler2020; Kuper et al., Reference Kuper, Breil, Horstmann, Roemer, Lischetzke, Sherman and Rauthmann2022; Roemer, Horstmann, & Ziegler, Reference Roemer, Horstmann and Ziegler2021).

We acknowledge that the issues are a matter of degree: We do not intend to say that the theories suggested by commentaries do not address mental computational processes at all. They may do so to some extent, but we can and should dig deeper. Some of these theories can be a great first step toward this aim. For example, some commentators (Custers et al.; Elliot & Sommet) mentioned classic incentive theories of motivation (Bindra, Reference Bindra1974; Bolles, Reference Bolles1972; Toates, Reference Toates1986). These theories are underspecified according to our perspective. At the same time, these theories can also be deemed as the foundation for contemporary reinforcement-learning theories (Dayan & Balleine, Reference Dayan and Balleine2002), which we believe address mental computational processes to a much greater extent.

The reward-learning framework of knowledge acquisition (Murayama, Reference Murayama2022a), which we presented as an example model describing mental computational processes, is also underspecified. As indicated by some commentators (Sheldon & Ryan; Schuetze & Rutten; which we also noted in the target article), the framework has an implicit assumption that awareness of a knowledge gap initiates information-seeking behavior. But how can one be aware of a knowledge gap? How should we describe the knowledge representation – via belief states (Golman, Gurney, & Loewenstein, Reference Golman, Gurney and Loewenstein2021) or a knowledge network (Murayama, Reference Murayama2022a; Sizemore, Karuza, Giusti, & Bassett, Reference Sizemore, Karuza, Giusti and Bassett2018)? How can we define the knowledge gap with that representation, and what kind of methods are used to compute it? As noted in the target article, this is a hot area in the field and we need to unpack the presented framework further in future studies.

It is also important to clarify that describing a theory in a mathematic form does not necessarily mean that it describes mental computational processes. Atkinson's expectancy and value theory of achievement motivation (Atkinson, Reference Atkinson1957) mentioned by Heckhausen & Rheinberg, Richter & Gendolla, and Wright et al., for example, is a clear mathematical theory but it does not explain the nature of expectancy as we discussed earlier (and the resultant value, which is an inverse of expectancy). The model of cognitive energetics mentioned by Richter & Gendolla (Kruglanski et al., Reference Kruglanski, Bélanger, Chen, Köpetz, Pierro and Mannetti2012), which adopted Lewin's (Reference Lewin1942) force-field approach, is a theory which explains social judgement and self-control with the concepts of driving force and restraining force. The model is described in mathematical forms, but like Lewin's formulation, the equations are described at a very general level. In addition, these mathematical theories critically lack time dynamics describing how the concepts causally influence each other over time. According to our view, any mental computational processes described in these theories are still underspecified.

R4. Should we push more?

Another set of commentaries challenge our article by indicating that the proposal we put forth (i.e., to specify mental computational processes), as it stands, is not enough and we need to step further. Gernigon, Altamore, Vallacher, van Geert, & Den Hartigh (Gernigon et al.) argued that the reward-learning framework of knowledge acquisition is driven by component-dominant dynamics which are fundamentally different from the interaction-dominant dynamics of the dynamic system approach (Den Hartigh, Cox, & van geert, Reference Den Hartigh, Cox, van geert, Magnani and Bertolotti2017; Van Orden, Holden, & Turvey, Reference Van Orden, Holden and Turvey2003). They argued that what we called “emergent property” is not, strictly speaking, emergent, because we specify the factors and causal relations underlying the phenomenon. Ozgan & Allen made a similar point by considering our stance as a substance-oriented framework.

Indeed, the reward-learning model of knowledge acquisition is not a dynamic systems model in a strict sense, which is characterized by the interaction of elements producing phenomena that cannot be predicted by examining the elements themselves. By looking at the specified reward-learning mechanisms, it is easy to see how an agent acts as if it had the need for competence. But it is important to emphasize that we presented the framework as one example of how mental computational processes can be specified. Regardless of whether the proposed mechanisms are component- or interaction-dominant, we welcome models and frameworks that seek to specify computational mechanisms underlying motivated behavior. Perhaps one important question is how much of motivated behavior can be explained by interaction-oriented dynamics (i.e., dynamic systems phenomena) and how much can be explained by component-dominant dynamics. There are certainly phenomena which can be better captured by dynamic systems perspective (Gernigon, Vallacher, Nowak, & Conroy, Reference Gernigon, Vallacher, Nowak and Conroy2015; Kaplan & Garner, Reference Kaplan and Garner2017; Laskar & van der Maas, Reference Laskar and van der Maas2024), but we do not believe that this perspective encompasses all motivation constructs. This can be tested if more work emerges focusing on mental computational processes in the future.

Alexander also provides a similar critical comment, although from a different perspective. Specifically, she suggested that the mental computational processes as illustrated by our reward-learning framework assume linearity and directionality, disregarding the dynamic and complicated nature of mental functioning. As noted above, we describe mental computational processes rather broadly, and they can accommodate complexity, dynamics, and non-linearity. The reward-learning model of knowledge acquisition is just one example, and we do not limit ourselves to it. However, unlike Gernigon et al., Alexander seems to have a more pessimistic view of whether we can truly specify such mental computational processes to explain motivated behavior (see also Heckhausen & Rheinberg). We do agree that mental computational processes underlying behavior are complicated and dynamic, especially when we leave the laboratory and attempt to study real-life behaviors. At the same time, recent years have seen a dramatic rise of modelling approaches in cognitive science and increased availability of real-life data via digital technologies (Allen et al., Reference Allen, Brändle, Botvinick, Fan, Gershman, Gopnik and Schulz2024). We feel that the time is ripe to take this bold step to further our understanding of motivated behavior in real life, rather than accumulating empirical evidence solely using broader motivational constructs. Our proposal aimed to encourage scholars to take such an endeavor.

Schuetze & Rutten brought another important perspective to our proposal – it is not the accuracy of mental computational processes per se that defines the best model but how useful the model is in practice. However accurate a model is, if the model is not interpretable, it is not useful for practitioners or policymakers. They provided interesting examples from the field of education, in which two prominent education researchers, Carroll (Reference Carroll1963) and Bloom (Reference Bloom1976), pushed forward computational process models of school learning but attracted minimum attention due to these models' complexity (Harnischfeger & Wiley, Reference Harnischfeger and Wiley1978).Footnote ³ We partly agree and indeed, investigating increasingly fine-grained levels of analysis may not be helpful to think about the best intervention for the phenomenon. At the same time, we believe that identifying the mental computational processes in many cases could provide the right bite-sized chunks for practice – not too broad, but not too intricate. When we say that the need for autonomy is important for well-being, self-determination theory provides several important practical suggestions, such as providing choices, encouraging self-initiation, and offering rationales for why autonomy is important (e.g., Pintrich & Schunk, Reference Pintrich and Schunk2002). But these practices themselves do not offer a universal solution (hence why it is difficult to promote people's well-being!). To find a more effective intervention, we need to understand how the provision of choices leads to increased well-being by identifying the underlying processes that give rise to the feeling of autonomy (see also Yan, Sana, & Carvalho, Reference Yan, Sana and Carvalho2024). Even the complex systems perspective, which has been criticized for its lack of usefulness for applications, could provide valuable insights into when and under what conditions an intervention works (Gernigon, Den Hartigh, Vallacher, & van Geert, Reference Gernigon, Den Hartigh, Vallacher and van Geert2024; van der Maas, Reference van der Maas2024; see also the concept of process causality suggested by Gernigon et al. and Ozgan & Allen).

Schuetze & Rutten's commentary provides an interesting contrast to the commentary by Wurm et al. They demanded that we should also consider the physiological implementation of the mental computational processes, that is, neural mechanisms (see also the commentary by Vassena & Gottlieb). They argued that neural level analyses can be an empirical tool to falsify or modify the proposed mental computational mechanisms, indicating the importance of taking multi-level perspective, considering different levels of analysis altogether. In a similar vein, Spurrett argued that, when we consider mental computational processes, we cannot avoid the role played by the bodies and actions (and their neural implementations). In fact, all the motivated decision making comes down to physical actions which compete with each other, which places substantial limitations on what we can do at a time. For us, such bodily constraints are also part of the level of physiological implementation by Marr (Reference Marr1982).

We agree with the point, and this can be also a great response to Schuetze & Rutten's commentary. Even when a certain theory turns out to be useful in practice, lower-level analysis still serves as a tool to empirically constrain theory (Marr, Reference Marr1982). At the same time, we also feel that seeking to understand brain mechanisms (and associated bodily mechanisms) adds further complexity to our already-challenging endeavor, especially given the limitations of currently available neuroscientific methods. We acknowledge that these methods have made substantial progress in recent years, but many challenges remain before we can specify the neural mechanisms underlying motivated behavior.

R5. Critical factors when considering mental computational mechanisms

Several commentaries raised additional critical factors concerning theories of mental computational mechanisms underlying high-level motivation constructs. We appreciate the suggestions. This kind of exchange would have been impossible if we stopped our thinking at the higher-level motivation construct, and illustrates the fruitfulness of our suggested direction.

van Lieshout, Zhang, Friston, & Bekkering (van Lieshout et al.) suggested that integrating the predictive processing framework would enrich our understanding of motivated behavior. These commentators state that the predictive processing framework encompasses all aspects of sensory experience, not a mere reward function, and agents choose actions according to the expected free energy – to maximize prediction of the world (Friston & Kiebel, Reference Friston and Kiebel2009). Reber, Haugen, & Martinussen (Reber et al.) also indicated the utility of this framework (but see the commentary by Moors for a critical remark). The reward-learning framework of knowledge acquisition is not inconsistent with the predicting processing framework (Fitzgibbon & Murayama, Reference Fitzgibbon and Murayama2022). Although not explained in the target article, Murayama (Reference Murayama2022a) indicated that “knowledge” or “information” is defined broadly, including perceptual or sensory information. Importantly, unlike other major models of information-seeking, the framework features a “knowledge base,” which represents all the past experiences of the agent (the “kind of thing that I am” according to their terminology). The knowledge base serves as the basis for prediction, and by taking actions that reduce expected uncertainty, the agent tries to construct the optimal world model, that is, expand and improve the knowledge base. At the same time, one critique we offer of the predictive processing framework is its computational tractability (Gottlieb & Oudeyer, Reference Gottlieb and Oudeyer2018; Ten et al., Reference Ten, Oudeyer, Sakaki and Murayama2024) – given the tremendous amount of experiences we accumulate over development, how can we efficiently calculate the expected information gain at every moment? To understand mental computational processes underlying the concept of need for competence, perhaps we also need to find concrete heuristics people take to master their environment (Ten et al., Reference Ten, Oudeyer, Sakaki and Murayama2024) or think seriously about how our knowledge is represented (Murayama, Reference Murayama2022a).

Bunzeck & Haesler argued that novelty is a key driver to explain exploration (and other motivated) behavior. Novelty and uncertainty are similar concepts but they can be distinguished via mental computational processes (Modirshanechi, Lin, Xu, Herzog, & Gerstner, Reference Modirshanechi, Lin, Xu, Herzog and Gerstner2023b; Poli, O'Reilly, Mars, & Hunnius, Reference Poli, O'Reilly, Mars and Hunnius2024). But novelty is not the only factor for exploration. In fact, one of the interesting aspects of information-seeking behavior is that people tend to become more and more interested as they acquire more knowledge (Alexander, Jetton, & Kulikowich, Reference Alexander, Jetton and Kulikowich1995; Fastrich & Murayama, Reference Fastrich and Murayama2020; Singh & Murayama, Reference Singh and Murayama2024; Witherby & Carpenter, Reference Witherby and Carpenter2022). This is because accumulated knowledge makes people aware the things they are not certain about (Murayama, Reference Murayama2022a; Murayama, FitzGibbon, & Sakaki, Reference Murayama, FitzGibbon and Sakaki2019). This means that the need for competence is likely to be governed by multiple processes such as uncertainty reduction, novelty seeking (Bunzeck & Duzel, Reference Bunzeck and Duzel2006), and savoring (Kobayashi, Ravaioli, Baranès, Woodford, & Gottlieb, Reference Kobayashi, Ravaioli, Baranès, Woodford and Gottlieb2019). How we weigh these different processes and integrate them depending on the contexts is an important area for future research (Modirshanechi, Kondrakiewicz, Gerstner, & Haesler, Reference Modirshanechi, Kondrakiewicz, Gerstner and Haesler2023a; Poli et al., Reference Poli, O'Reilly, Mars and Hunnius2024).

Reber et al. proposed that subjective metacognitive feelings play a critical role in motivated behavior and should be actively incorporated in specifying mental computational processes. A similar point was made by Del Giudice's GAM model, in which affect serves a critical interface between the higher-order and lower-order goals. While our proposal put subjective experiences outside of the mental computational processes (Fig. 1 in the target article), as implied by the arrow from subjective experiences to mental computational processes, we did not preclude the possibility that subjective experiences modulate mental computational processes. In fact, in our work of metamotivation, we showed that people often have the wrong metacognition about how motivation functions and take actions that are not adaptive for motivation (Hatano, Ogulmus, Shigemasu, & Murayama, Reference Hatano, Ogulmus, Shigemasu and Murayama2022; Kim, Sakaki, & Murayama, Reference Kim, Sakaki and Murayama2024; Kuratomi, Johnsen, Kitagami, Hatano, & Murayama, Reference Kuratomi, Johnsen, Kitagami, Hatano and Murayama2023; Murayama, Kitagami, Tanaka, & Raw, Reference Murayama, Kitagami, Tanaka and Raw2016). One critical question in this regard is, what are the mental computational processes that give rise to these metacognitive feelings? If these metacognitive feelings are calculated by relatively simple algorithms such as familiarity or fluency (Reber, Winkielman, & Schwarz, Reference Reber, Winkielman and Schwarz1998), then it is possible that they serve as important heuristics for us to efficiently reduce uncertainty in our knowledge (see also our response to van Lieshout et al.). In fact, Shenhav (Reference Shenhav2024) recently argued that the “goal” concept in decision-making literature may be an emergent property which is produced by individuals' affective associations.

Ainslie pushed the reward-learning framework further and discussed the nature of intrinsic or endogenous rewards (Ainslie, Reference Ainslie2013), and how we manage them. Indeed, when unpacking the black box of motivational constructs according to the reward-learning framework, it is imperative to unpack the computational mechanisms that give rise to endogenous rewards. Ainslie also argued that for information gain (uncertainty reduction) to be regarded as rewards, they should (a) perform like rewards that have been studied in other contexts, (b) have a variable effect over a time course, and (c) depend on some kind of appetite. Temporal change in the rewarding value of information gain has been studied in empirical studies (Hsiung, Poh, Huettel, & Adcock, Reference Hsiung, Poh, Huettel and Adcock2023; Noordewier & van Dijk, Reference Noordewier and van Dijk2015) but for other aspects, we still know little. We agree that this is an important area for research in the future.