A healthy adult asked to run 60 m will likely sprint; a healthy adult asked to run 1,000 m will likely jog. In fact, there is hardly anyone on Earth who would even attempt to sprint for 1,000 m. This simple observation demonstrates what we intuitively already know: that human behavior is typically adapted to our own limitations. Therefore, a deep understanding of behavior necessitates that various sources of limitations are rigorously identified and precisely quantified. I applaud Lieder and Griffiths (L&G) for advocating for this practice.
L&G propose “resource-rational analysis,” which is a methodological device that an experimenter uses to discover something about human behavior. However, the target article appears to sometimes conflate this methodological device with the substantive claim that humans are actually resource-rational. To be fair, L&G stop short of claiming that people are actually resource-rational. They even offer that “we should not expect people's heuristics to be perfectly resource-rational” (sect. 3, para. 6). Nevertheless, other parts of the target article give a sense that L&G really do think that people are (mostly) resource-rational. For example, they consider seriously the “assumption that the brain is approximately bounded-optimal” (sect. 5.3.2., para. 2), claim that resource-rational analysis “has already shed new light on the debate about human rationality” (abstract), and even state that “people's decision-mechanisms appear to be surprisingly resource-rational” (sect. 6, para. 3). These statements leave the realm of methodological devices and venture into the land of substantive claims about human rationality.
The problem is that, as currently constructed, resource-rational analysis does not and could not provide evidence for the rationality of human behavior. There are at least three reasons for this.
First, resource-rational analysis in overly flexible as a tool for establishing the nature of human behavior. As Box 2 demonstrates, a researcher who follows the methodology prescribed by L&G should test a number of different constraints and computational architectures until some combination of them provides a good fit to the data. To L&G's credit, they do advice that the experimenter stops trying out new combinations after “reasonable attempts have been made to model the constraints” (Box 2). Nevertheless, for most experimental tasks, it is not too difficult to find a set of assumptions that makes behavior to appear close to rational. This does not, however, imply that the underlying behavior is rational because the experimenter may have unwittingly postulated computational architectures or resource limitations that do not exist, or, more likely, exist but are mischaracterized. For example, a tendency to underuse explicitly stated probabilities (Rahnev and Denison Reference Rahnev and Denison2018a) can be cast as optimal decision making by an organism that misrepresents probabilities (Zhang and Maloney Reference Zhang and Maloney2012). However, this explanation could be given regardless of whether the organism actually adopts optimal decision making based on skewed representations of probability or adopts a suboptimal decision strategy on internal representations of probability that are less skewed. Therefore, substantive claims about human rationality require models that are prespecified and have no free parameters (e.g., the misrepresentation of probabilities should be predetermined for each subject). Very few papers, however, fit such zero-parameter models to the data.
Second, the types of tasks that we study in the laboratory tend to be the most constrained and simple tasks that an organism could ever face. Yet, even for such tasks, suboptimality is the norm (Rahnev and Denison Reference Rahnev and Denison2018a). Regardless of how close to rationality humans get in such tasks, it does not follow that behavior would be similarly rational in the infinitely more complex real world. As L&G admit themselves, it is “challenging to [apply resource-rational analysis] to decision-making in the real world where the sets of options and possible outcomes are much larger and often unknown” (sect. 6, para. 7).
Third, the computations required to establish the truly rational strategy are intractable and will always remain so. Indeed, as demonstrated by Equation 4 in the target article, specifying what is actually rational requires quantitatively describing all environments that one has ever experienced (including environments that have been experienced by one's ancestors and have influenced brain development over evolutionary scales), which is clearly infeasible in practice. Therefore, in the strictest sense of Equation 4, we will never be able to test whether any behavior is truly rational or not.
If there is little hope that we could ever establish whether human behavior is really rational, does that mean that resource-rational analysis is also futile? Not at all. As the example of running a shorter versus longer distance demonstrates, we are profoundly constrained by our limitations, and our behavior is often roughly adapted to these limitations. Therefore, resource-rational analysis offers at least two large benefits (in addition to what was highlighted by L&G). First, resource-rational analysis can be used to approximate human behavior under the assumption that evolution has adapted our behavior to the particular task used by the experimenter. Clearly, for a non-resource-rational human, the approximation may be crude and sometimes very imprecise, but at the very least could be used as a starting point. Second, behavior that is systematically deviating from resource-rationality may indicate the existence of a new, previously undiscovered limitation or cognitive architecture. As highlighted above, postulating limitations just for the sake of fitting data is a dangerous undertaking, and thus any proposal for a new limitation should be tested with independent data and, ideally, under new conditions.
Regardless of one's preferred view of human nature and the best methods to reveal that human nature, it is critical that substantive claims about behavior and methodological approaches about studying said behavior are kept separate from each other (Rahnev and Denison Reference Rahnev and Denison2018b). The person who jogs for 1,000 m is unlikely to do so at the optimal pace. That is, she is unlikely to be fully resource-rational. However, we will certainly understand her behavior better if we put in the effort to quantify the exact rate at which her muscles tire. Resource-rational analysis can be useful even if we are trying to characterize non-resource-rational humans.
You have
Access
A healthy adult asked to run 60 m will likely sprint; a healthy adult asked to run 1,000 m will likely jog. In fact, there is hardly anyone on Earth who would even attempt to sprint for 1,000 m. This simple observation demonstrates what we intuitively already know: that human behavior is typically adapted to our own limitations. Therefore, a deep understanding of behavior necessitates that various sources of limitations are rigorously identified and precisely quantified. I applaud Lieder and Griffiths (L&G) for advocating for this practice.
L&G propose “resource-rational analysis,” which is a methodological device that an experimenter uses to discover something about human behavior. However, the target article appears to sometimes conflate this methodological device with the substantive claim that humans are actually resource-rational. To be fair, L&G stop short of claiming that people are actually resource-rational. They even offer that “we should not expect people's heuristics to be perfectly resource-rational” (sect. 3, para. 6). Nevertheless, other parts of the target article give a sense that L&G really do think that people are (mostly) resource-rational. For example, they consider seriously the “assumption that the brain is approximately bounded-optimal” (sect. 5.3.2., para. 2), claim that resource-rational analysis “has already shed new light on the debate about human rationality” (abstract), and even state that “people's decision-mechanisms appear to be surprisingly resource-rational” (sect. 6, para. 3). These statements leave the realm of methodological devices and venture into the land of substantive claims about human rationality.
The problem is that, as currently constructed, resource-rational analysis does not and could not provide evidence for the rationality of human behavior. There are at least three reasons for this.
First, resource-rational analysis in overly flexible as a tool for establishing the nature of human behavior. As Box 2 demonstrates, a researcher who follows the methodology prescribed by L&G should test a number of different constraints and computational architectures until some combination of them provides a good fit to the data. To L&G's credit, they do advice that the experimenter stops trying out new combinations after “reasonable attempts have been made to model the constraints” (Box 2). Nevertheless, for most experimental tasks, it is not too difficult to find a set of assumptions that makes behavior to appear close to rational. This does not, however, imply that the underlying behavior is rational because the experimenter may have unwittingly postulated computational architectures or resource limitations that do not exist, or, more likely, exist but are mischaracterized. For example, a tendency to underuse explicitly stated probabilities (Rahnev and Denison Reference Rahnev and Denison2018a) can be cast as optimal decision making by an organism that misrepresents probabilities (Zhang and Maloney Reference Zhang and Maloney2012). However, this explanation could be given regardless of whether the organism actually adopts optimal decision making based on skewed representations of probability or adopts a suboptimal decision strategy on internal representations of probability that are less skewed. Therefore, substantive claims about human rationality require models that are prespecified and have no free parameters (e.g., the misrepresentation of probabilities should be predetermined for each subject). Very few papers, however, fit such zero-parameter models to the data.
Second, the types of tasks that we study in the laboratory tend to be the most constrained and simple tasks that an organism could ever face. Yet, even for such tasks, suboptimality is the norm (Rahnev and Denison Reference Rahnev and Denison2018a). Regardless of how close to rationality humans get in such tasks, it does not follow that behavior would be similarly rational in the infinitely more complex real world. As L&G admit themselves, it is “challenging to [apply resource-rational analysis] to decision-making in the real world where the sets of options and possible outcomes are much larger and often unknown” (sect. 6, para. 7).
Third, the computations required to establish the truly rational strategy are intractable and will always remain so. Indeed, as demonstrated by Equation 4 in the target article, specifying what is actually rational requires quantitatively describing all environments that one has ever experienced (including environments that have been experienced by one's ancestors and have influenced brain development over evolutionary scales), which is clearly infeasible in practice. Therefore, in the strictest sense of Equation 4, we will never be able to test whether any behavior is truly rational or not.
If there is little hope that we could ever establish whether human behavior is really rational, does that mean that resource-rational analysis is also futile? Not at all. As the example of running a shorter versus longer distance demonstrates, we are profoundly constrained by our limitations, and our behavior is often roughly adapted to these limitations. Therefore, resource-rational analysis offers at least two large benefits (in addition to what was highlighted by L&G). First, resource-rational analysis can be used to approximate human behavior under the assumption that evolution has adapted our behavior to the particular task used by the experimenter. Clearly, for a non-resource-rational human, the approximation may be crude and sometimes very imprecise, but at the very least could be used as a starting point. Second, behavior that is systematically deviating from resource-rationality may indicate the existence of a new, previously undiscovered limitation or cognitive architecture. As highlighted above, postulating limitations just for the sake of fitting data is a dangerous undertaking, and thus any proposal for a new limitation should be tested with independent data and, ideally, under new conditions.
Regardless of one's preferred view of human nature and the best methods to reveal that human nature, it is critical that substantive claims about behavior and methodological approaches about studying said behavior are kept separate from each other (Rahnev and Denison Reference Rahnev and Denison2018b). The person who jogs for 1,000 m is unlikely to do so at the optimal pace. That is, she is unlikely to be fully resource-rational. However, we will certainly understand her behavior better if we put in the effort to quantify the exact rate at which her muscles tire. Resource-rational analysis can be useful even if we are trying to characterize non-resource-rational humans.