Generalization is inherent to scientific inference – physicists learn about the properties of distant stars based on lab-based experiments, vision scientists use illusions to infer mechanisms of basic human perception, and meteorologists generalize from the simulations of mathematical models to form expectations about tomorrow's weather. How do scientists draw conclusions about the general features of the environment using such incomplete and indirect evidence?

The position taken in the target article is that generalization is licensed by what we have observed. In this view, we learn about the robustness of a phenomenon by observing it in a range of contexts, and on that basis we can generalize it to a wider range of environments (i.e., expect the phenomenon to occur in those other environments similarly). However, as Yarkoni demonstrates in his Section 4, the choice of any specific generalization that goes beyond what has been directly observed is arbitrary and irrational (see also Hume, Reference Hume1739). Unfortunately, it seems that Yarkoni does not take his own critique seriously, and so misses the opportunity to explain why generalization works.

What is absent from the above description is the role theory plays in generalization. Scientific theories make claims about how our physical environment behaves and why, and so they imply what features of these environments generalize (Deutsch, Reference Deutsch2011). That is, rather than on the basis of repeated observations, we expect to see a phenomenon generalize to a new context because a theory implies that we should.

Take the Stroop effect, widely considered to be one of the most general findings in psychological science. The robustness of the Stroop effect comes not from having observed it repeatedly and in many contexts, but because we have a compelling explanation for why it happens. Namely, the processes of reading and naming both rely on the same semantic representation. Also, quickly reading the words we see is a well-established habit, while naming the color in which a word is printed is unusual and thus slower. Therefore, if reading a word creates semantic activation that could be mistaken as the output of naming, the word is spoken preemptively. Such an explanation dictates the kinds of experiments in which we should observe an effect – for example, if we encourage reading by making most words either helpful (the same color as the ink) or benign (words unlikely to activate concepts related to colors), and it helps us avoid experiments likely to fail – such as printing the color word in a language foreign to the reader. Thus it is the general implications of this explanation that allow us to produce the Stroop effect in so many contexts.

The generality implied by theories, however, should not be taken for granted. We should not, for example, take seriously any specific prediction about generality coming from flexible theories, since they could be easily changed to predict any possible result (Szollosi & Donkin, Reference Szollosi and Donkin2021). Similarly, most phenomena in psychology are explained by a number of possible theories, with no clear reasons to decide between them. Where such theories diverge in their claims about generality, it would be entirely arbitrary to expect the predictions of any one theory. So, while Yarkoni is right to be concerned about generalizations in psychology research, the real problem is that generalization is expected to occur despite having no good reason to expect it over the countless other (and often non-general) explanations that are at least equally (and often more) viable.

Generalization in psychology is also complicated by the fact that what is often considered its explicanda – people's thoughts, motives, and behaviors – all tend to change in response to new knowledge. For example, the effectiveness of a demonstration of the Stroop effect will likely diminish as the task is repeated. Once aware of the effect, a participant may try to explain what is happening, and come up with strategies to defeat it, such as only looking at a small part of the text or unfocusing their eyes.

More generally, when placed into an unfamiliar setting or environment, people are liable to adapt, combine, and repurpose (aspects of) their existing knowledge until they find an adequate representation or explanation (e.g., Bramley, Dayan, Griffiths, & Lagnado, Reference Bramley, Dayan, Griffiths and Lagnado2017; Lake, Ullman, Tenenbaum, & Gershman, Reference Lake, Ullman, Tenenbaum and Gershman2017; Szollosi & Newell, Reference Szollosi and Newell2020). This new or restructured knowledge may permit behaviors that are entirely novel, and could never have been predicted (else that knowledge already existed). Thus, since the most characteristically human aspects of behavior are driven by representations and these are always subject to change, it is unreasonable to expect any single observed behavior to be completely general. Instead, psychology should seek the causes of this flexibility and adaptivity in observed behaviors (Chomsky, Reference Chomsky1959).

Therefore, to understand what can be generally true of cognition, we should explain how people can create and change their representations of their surroundings. This suggests that primary explicanda of psychology are people's capacities, not any particular behavior (e.g., van Rooij & Baggio, Reference van Rooij and Baggio2021). Psychological explanations should not only account for what people did in some experiment, but also for what they could have done. In explaining the Stroop effect, a capacity-focused explanation must explain not only why people exhibit the Stroop effect when first exposed, but also reasons why they might not, or why the effect might change in the future (e.g., if the Stroop effect were to be taught in high schools, to the point that people learn strategies to avoid it).

Understanding the general aspects of human behavior by collecting a list of psychological effects and the contexts in which they occur – as suggested in the target article – is incompatible with the fact that we should expect behavior not to be the same across contexts. It is not clear how the effect-focused psychological science for which Yarkoni advocates could ever be reconciled with the flexibility of human cognition. Instead, we suggest that a focus on explaining people's capacity – in terms of a general ability to create and adapt representations of environments – could result in stronger, broader psychological generalizations.