The past decade has clearly demonstrated that psychology is replete with contradictory, redundant, and irreplicable findings. Ego depletion exists (Baumeister & Vohs, Reference Baumeister and Vohs2007), except when it doesn't (Hagger et al., Reference Hagger, Chatzisarantis, Alberts, Anggono, Batailler, Birt and Zwienenberg2016), or when it's too small to have a meaningful effect (Dang et al., Reference Dang, Barker, Baumert, Bentvelzen, Berkman, Buchholz and Zinkernagel2021). Depending on which psychologist is writing the paper, people may alternatively have grit (Duckworth & Quinn, Reference Duckworth and Quinn2009), be resilient (Friborg, Barlaug, Martinussen, Rosenvinge, & Hjemdal, Reference Friborg, Barlaug, Martinussen, Rosenvinge and Hjemdal2005), or just demonstrate specific aspects of conscientiousness (Schmidt, Nagy, Fleckenstein, Möller, & Retelsdorf, Reference Schmidt, Nagy, Fleckenstein, Möller and Retelsdorf2018). To an objective observer, it may seem that large-swaths of psychology have long moved past Feynman's (Reference Feynman1974) warnings of a cargo cult science and all-but-abandoned the science part altogether. As more psychologists have publicly acknowledged that huge portions of our science are merely either the selling of old wine in new bottles or attempts to sell bottled water by calling it wine, the past decade in psychological science has been a crisis of crises. The replication crisis has given way to a measurement crisis (Flake & Fried, Reference Flake and Fried2020), a validity crisis (Clark & Watson, Reference Clark and Watson2019; Flake, Pek, & Hehman, Reference Flake, Pek and Hehman2017; Lundh, Reference Lundh and Valsiner2019), a practicality crisis (Berkman & Wilson, Reference Berkman and Wilson2021), and a litany of others (Hughes, Reference Hughes2018). To this burgeoning but rich legacy of crises, Yarkoni has added another: The generalizability crisis.

Yarkoni has argued that psychology has fallen into what Meehl (Reference Meehl1990) warned of, having created a field of scientists whose entire careers are devoted to describing, demonstrating, and marketing nothing. This, in turn, wastes resources such as time, effort, and funds, as all are being spent on experimental studies that have no chance of actually answering the question they purport to answer. Perhaps more concerningly, this crisis points to a loss of credibility for the entire field. Indeed, these costs are high. Yet, for some domains within psychology, a crisis of generalizability may come at a greater cost than wasted resources or statistical charades.

For seminal work in verbal overshadowing to fall short of basic generalizability is disappointing, particularly given the efforts invested in multisite replication efforts. Even so, would anyone argue that social costs of these efforts extend much further beyond the loss of those resources? Similarly, the three-decade-long-and-still-continuing obsession with ego depletion has undoubtedly done more to deplete financial and human capital resources than demonstrate any generalizable claim about willpower or self-control (Friese, Loschelder, Gieseler, Frankenbach, & Inzlicht, Reference Friese, Loschelder, Gieseler, Frankenbach and Inzlicht2019). Yet the societal costs of those Sisyphean attempts are likely relatively low. Failed experiments certainly may have derailed careers, and such losses should not be ignored. Even so, the average layperson will be more influenced by the ice-cream they choose to eat for dessert today than they will by failure of ego depletion to deliver on its years of bold claims. Could the same be said for interventions in clinical psychology? If evidence-based treatments and “gold-standard” therapies for depression, post-traumatic stress, or anorexia lack generalizability, are the costs accurately measured in wasted scientific resources?

Even though all applied scientists in psychology should be deeply concerned about the cost of a generalizability crisis, clinical science should be so in even greater measure. It is no accident that Meehl's perennially salient concerns about the quality and rigor of psychological science were born of a career that integrated clinical practice (Meehl, Reference Meehl1987). The costs of ungeneralizable and sloppy science are higher when that science is guiding practitioners' decisions. There is abundant evidence that many of clinical psychology's gold-standard treatments are built on statistical errors, questionable research practices, and bad inferences (Sakaluk, Williams, Kilshaw, & Rhyner, Reference Sakaluk, Williams, Kilshaw and Rhyner2019). If even a few of the remaining treatments that are not built on gross statistical errors or questionable research practices are found to lack generalizability – a critique that practicing clinicians have voiced for many years (Lilienfeld, Ritschel, Lynn, Cautin, & Latzman, Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013; Stewart, Stirman, & Chambless, Reference Stewart, Stirman and Chambless2012) – what then remains of the field?

Undoubtedly, the same criticisms and concerns that I have framed around clinical psychology might also apply to other applied domains. Educational psychology has influenced real-world decisions for years, and the work done in industrial and organizational psychology has affected careers of countless individuals. Similarly, there are social and cognitive psychologists that are heavily engaged in applied research that bears real-world implications for both individual people and broader policy. Such fields and scientists should be similarly shaken by the notion that their work fundamentally does not and cannot actually statistically evaluate the questions they claim to be evaluating. If the generalizability crisis does indeed come to the forefront – if enough people do as Yarkoni has suggested and stop honoring the charade of statistical inference – none of the major subfields of psychology will be safe. Even so, the costs for clinical psychology should and will be steep.

All of psychology claims to do impactful science. Yet clinical psychology's perennial claims of alleviating suffering and literally saving lives are unique among the subfields of our science. The logical upshot of such claims is that the costs of a generalizability crisis are measured in human lives, not wasted resources. Yarkoni closes by offering readers a choice. The first option is the choice to improve, to make more reasonable inferences, temper one's claims, and return to more basic forms of research such as descriptive and qualitative analyses. The second is to simply stick one's head in the sand and recommit to the cargo cult rituals that have brought success to so many for so long. Although the latter of these two options is certainly the easier path, for psychological scientists in applied domains, particularly those in clinical science, the human costs of ignoring such a crisis must leave them with only one choice.