The authors propose Resource Rational Analysis as a unifying modeling paradigm that combines rational analysis with cognitive constraints, arguing that it addresses the problem of under-determination of cognitive mechanism by data, and provides a way to theorize about cognitive constraints while retaining the rigor of optimality analyses. The authors offer a multi-step method that includes a step that derives an optimal algorithm to run on the mind's computational architecture, and ground the paradigm formally in the framework of bounded optimality from artificial intelligence (AI). We endorse this paradigm and method: we made these arguments and proposed a paradigm and method in Howes et al. (Reference Howes, Lewis and Vera2009) with just these features: Cognitively Bounded Rational Analysis (grounding it in bounded optimality in Lewis et al. [Reference Lewis, Howes and Singh2014]). Each step of the authors’ Resource Rational Analysis method (except for the “iterate” step) corresponds to a step in Cognitively Bounded Rational Analysis (Fig. 1), including the crucial steps that distinguish it from Anderson's seminal Rational Analysis: positing an explicit space of algorithms to run on a cognitive machine, the selection of the algorithm that maximizes some utility, and the evaluation of the optimal algorithm against data. The illustrative example used in Howes et al. (Reference Howes, Lewis and Vera2009) has several desirable properties of Resource Rational Analysis highlighted by the authors, including a method for the automatic derivation of complex cognitive strategies (beyond optimization of quantitative parameters), and the calibration of cognitive constraints with independent data.
Figure 1. How the five steps of cognitively bounded rational analysis focus the space of behaviors. Step 1 defines architecture and environment. Steps 2–4 narrow that space by first determining the plausible strategies and then determining the subset of best strategies. Only Step 5 involves comparison to data. From Howes et al. (Reference Howes, Lewis and Vera2009).
The target article provides a useful survey of recent relevant work across multiple domains in cognitive science. The breadth is important because it makes clear that bounded optimality analyses are useful beyond perceptual decision making and motor control, and when brought to bear on higher cognition provide new insights into the nature of human rationality. Rather than address specific applications, we focus here on an important issue that is not addressed sufficiently clearly in the authors’ framework, our own previous work, or related work. The issue concerns a theory of utility.
That there is an issue can be seen in the different treatment of utility in Equations 2 and 3 in the target article. Equation 2 is a form of bounded optimality with a direct correspondence to the definitions of Russell & Subramanian (Reference Russell and Subramanian1995) and Lewis et al. (Reference Lewis, Howes and Singh2014). The utility function is an unconstrained function of agent–environment interactions. In contrast, Equation 3 (which does not build on Equation 2) has two distinctive features: it ascribes a belief state to the agent, and it decomposes the objective function into utility and resource cost terms. We now consider some implications of this decomposition.
There was no need to include separate “cost-of-computation” or “resource-cost” terms in the original formulation of bounded optimality because these costs are captured by the implications of the machine constraints for the utility of machine–environment interactions. In particular, it is easy to specify a speed-accuracy tradeoff in a utility function that may implicitly put pressure on the machine+algorithm to make various internal trade-offs of speed, memory, accuracy, and so on. In this sense, the separate resource cost term in Equation 3 adds neither expressive power nor theoretical constraint. The incorporation of a belief state, which we take to be a probability distribution over which expectations may be computed, is a theoretical constraint. It is not clear how many of the examples reviewed in the target article actually assume an analysis that incorporates belief states; we assume it is useful in some analyses but not a commitment of the resource rationality framework.
But there is an important sense in which Equation 3 is more expressive than the bounded optimality Equation 2: It allows for the cost (and so overall utility) to be a function of the internal state of the cognitive machine. It must be if any kind of cognitive “cost” or “effort” other than time is to be calculated. The distinction is an important one; we have assumed in the past (e.g., Howes et al. Reference Howes, Lewis and Vera2009) that the utilities in bounded rational analyses are subjective utilities, which implies that they are functions of internal agent state. But the form of Equation 2 in the target article and our own formalization in Lewis et al. (Reference Lewis, Howes and Singh2014) inherited from bounded optimality the property that utility is a function of states of the world/environment with which the agent interacts. We suggest that making the utility explicitly a function of internal agent state would yield a conceptually simpler and clearer definition that has the expressive power of Equation 3, without committing to a belief state formulation of agent state, or a particular kind of cost term.
These considerations put into sharp focus the need for constrained theories of subjective utility; otherwise the methodological benefits of bounded rational analysis may be diminished by the additional degrees of freedom available in specifying “resource costs.” Such theories must go beyond economic models of subjective utility and include explicit accounts of cognitive effort; the author's own work on effort (Shenhav et al. Reference Shenhav, Musslick, Lieder, Kool, Griffiths, Cohen and Botvinick2017) begins to provide such a theory. But, more generally such theories must also encompass formal accounts of so-called “intrinsic” motivations thought to drive exploration and learning. It is in fact possible to bring a bounded-optimality analysis to bear on such theorizing: the optimal rewards framework (Singh et al. Reference Singh, Lewis, Barto and Sorg2010; Sorg et al. Reference Sorg, Singh and Lewis2010) sets up a meta-optimization problem that derives internal reward functions adapted to the bounds of learning agents so that they maximize some measure of objective fitness.
We agree that combining rational analysis with cognitive bounds is a promising and still under-used methodology for cognitive science and psychology, and the target review contributes substantially to this case. The hope is that a relatively small set of computational abstractions will emerge over time that are broadly useful as theories of cognitive mechanism. This hope looks beyond a broadly applicable method to broadly applicable theory, a hope expressed by Newell (Reference Newell1990) in his call for unification in psychology.
You have
Access
The authors propose Resource Rational Analysis as a unifying modeling paradigm that combines rational analysis with cognitive constraints, arguing that it addresses the problem of under-determination of cognitive mechanism by data, and provides a way to theorize about cognitive constraints while retaining the rigor of optimality analyses. The authors offer a multi-step method that includes a step that derives an optimal algorithm to run on the mind's computational architecture, and ground the paradigm formally in the framework of bounded optimality from artificial intelligence (AI). We endorse this paradigm and method: we made these arguments and proposed a paradigm and method in Howes et al. (Reference Howes, Lewis and Vera2009) with just these features: Cognitively Bounded Rational Analysis (grounding it in bounded optimality in Lewis et al. [Reference Lewis, Howes and Singh2014]). Each step of the authors’ Resource Rational Analysis method (except for the “iterate” step) corresponds to a step in Cognitively Bounded Rational Analysis (Fig. 1), including the crucial steps that distinguish it from Anderson's seminal Rational Analysis: positing an explicit space of algorithms to run on a cognitive machine, the selection of the algorithm that maximizes some utility, and the evaluation of the optimal algorithm against data. The illustrative example used in Howes et al. (Reference Howes, Lewis and Vera2009) has several desirable properties of Resource Rational Analysis highlighted by the authors, including a method for the automatic derivation of complex cognitive strategies (beyond optimization of quantitative parameters), and the calibration of cognitive constraints with independent data.
Figure 1. How the five steps of cognitively bounded rational analysis focus the space of behaviors. Step 1 defines architecture and environment. Steps 2–4 narrow that space by first determining the plausible strategies and then determining the subset of best strategies. Only Step 5 involves comparison to data. From Howes et al. (Reference Howes, Lewis and Vera2009).
The target article provides a useful survey of recent relevant work across multiple domains in cognitive science. The breadth is important because it makes clear that bounded optimality analyses are useful beyond perceptual decision making and motor control, and when brought to bear on higher cognition provide new insights into the nature of human rationality. Rather than address specific applications, we focus here on an important issue that is not addressed sufficiently clearly in the authors’ framework, our own previous work, or related work. The issue concerns a theory of utility.
That there is an issue can be seen in the different treatment of utility in Equations 2 and 3 in the target article. Equation 2 is a form of bounded optimality with a direct correspondence to the definitions of Russell & Subramanian (Reference Russell and Subramanian1995) and Lewis et al. (Reference Lewis, Howes and Singh2014). The utility function is an unconstrained function of agent–environment interactions. In contrast, Equation 3 (which does not build on Equation 2) has two distinctive features: it ascribes a belief state to the agent, and it decomposes the objective function into utility and resource cost terms. We now consider some implications of this decomposition.
There was no need to include separate “cost-of-computation” or “resource-cost” terms in the original formulation of bounded optimality because these costs are captured by the implications of the machine constraints for the utility of machine–environment interactions. In particular, it is easy to specify a speed-accuracy tradeoff in a utility function that may implicitly put pressure on the machine+algorithm to make various internal trade-offs of speed, memory, accuracy, and so on. In this sense, the separate resource cost term in Equation 3 adds neither expressive power nor theoretical constraint. The incorporation of a belief state, which we take to be a probability distribution over which expectations may be computed, is a theoretical constraint. It is not clear how many of the examples reviewed in the target article actually assume an analysis that incorporates belief states; we assume it is useful in some analyses but not a commitment of the resource rationality framework.
But there is an important sense in which Equation 3 is more expressive than the bounded optimality Equation 2: It allows for the cost (and so overall utility) to be a function of the internal state of the cognitive machine. It must be if any kind of cognitive “cost” or “effort” other than time is to be calculated. The distinction is an important one; we have assumed in the past (e.g., Howes et al. Reference Howes, Lewis and Vera2009) that the utilities in bounded rational analyses are subjective utilities, which implies that they are functions of internal agent state. But the form of Equation 2 in the target article and our own formalization in Lewis et al. (Reference Lewis, Howes and Singh2014) inherited from bounded optimality the property that utility is a function of states of the world/environment with which the agent interacts. We suggest that making the utility explicitly a function of internal agent state would yield a conceptually simpler and clearer definition that has the expressive power of Equation 3, without committing to a belief state formulation of agent state, or a particular kind of cost term.
These considerations put into sharp focus the need for constrained theories of subjective utility; otherwise the methodological benefits of bounded rational analysis may be diminished by the additional degrees of freedom available in specifying “resource costs.” Such theories must go beyond economic models of subjective utility and include explicit accounts of cognitive effort; the author's own work on effort (Shenhav et al. Reference Shenhav, Musslick, Lieder, Kool, Griffiths, Cohen and Botvinick2017) begins to provide such a theory. But, more generally such theories must also encompass formal accounts of so-called “intrinsic” motivations thought to drive exploration and learning. It is in fact possible to bring a bounded-optimality analysis to bear on such theorizing: the optimal rewards framework (Singh et al. Reference Singh, Lewis, Barto and Sorg2010; Sorg et al. Reference Sorg, Singh and Lewis2010) sets up a meta-optimization problem that derives internal reward functions adapted to the bounds of learning agents so that they maximize some measure of objective fitness.
We agree that combining rational analysis with cognitive bounds is a promising and still under-used methodology for cognitive science and psychology, and the target review contributes substantially to this case. The hope is that a relatively small set of computational abstractions will emerge over time that are broadly useful as theories of cognitive mechanism. This hope looks beyond a broadly applicable method to broadly applicable theory, a hope expressed by Newell (Reference Newell1990) in his call for unification in psychology.