Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-06T14:21:22.625Z Has data issue: false hasContentIssue false

The seductive allure of cargo cult computationalism

Published online by Cambridge University Press:  29 September 2022

Micah Allen*
Affiliation:
Aarhus Institute of Advanced Studies, Aarhus University, 8000 Aarhus, Denmark Center of Functionally Integrative Neuroscience, Aarhus University, 8000 Aarhus, Denmark micah@cfin.au.dk https://www.the-ecg.org/ Cambridge Psychiatry, University of Cambridge, Cambridge CB2 8AH, UK

Abstract

Bruineberg and colleagues report a striking confusion, in which the formal Bayesian notion of a “Markov blanket” has been frequently misunderstood and misapplied to phenomena of mind and life. I argue that misappropriation of formal concepts is pervasive in the “predictive processing” literature, and echo Richard Feynman in suggesting how we might resist the allure of cargo cult computationalism.

Type
Open Peer Commentary
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

The first principle is that you must not fool yourself – and you are the easiest person to fool.

— Richard Feynman (Reference Feynman1974)

In their compelling arguments, Bruineberg and colleagues reveal how the mathematical, Bayesian construct of a “Markov blanket” has become a go-to explanans in topics as far-reaching as neuroscience, sociology, the philosophy of mind, and epistemology. Through careful analyses of the prerequisite formalisms, they reveal how many of these works confuse a realist understanding of Markov blankets with their actual properties as defined by formal mathematics. A consequence of this confusion is that Markov blankets are ascribed properties they do not possess and are frequently leveraged to explain phenomena for which they have little direct relevance. Indeed, it has been argued Markov blankets demarcate the definition of life (Kirchhoff, Parr, Palacios, Friston, & Kiverstein, Reference Kirchhoff, Parr, Palacios, Friston and Kiverstein2018), can guide an Asimov-like attempt at psychohistory (Allen, Reference Allen2018; Ramstead, Badcock, & Friston, Reference Ramstead, Badcock and Friston2018; Veissière, Constant, Ramstead, Friston, & Kirmayer, Reference Veissière, Constant, Ramstead, Friston and Kirmayer2019), and even extend cognition to plants (Calvo & Friston, Reference Calvo and Friston2017). Bruineberg and colleagues argue that these and many similar arguments fail to capture what is and is not offered by Markov blankets and proffer a helpful framework for understanding and applying Markovian concepts, based on an informed analysis of their actual formal properties.

As I read the target article, I could not help but think of the late Richard Feynman's now infamous remarks, delivered to the Caltech class of 1974, in which he warned of the dangers of “cargo cult science” (Feynman, Reference Feynman1974). Prior to the commencement address, Feynman visited the Esalen Institute, a well-known nexus for “alternative” science. He recounted how, to his surprise, many of the advocates of esoteric mysticism and parapsychology he met there inevitably presented their ideas as scientific, when they were clearly anything but. But perhaps more worrying, he also remarked how many of the styles of argumentation he encountered could also be found in mainstream fields of psychology, neuroscience, and even physics. Feynman dubbed these trends as “cargo cult science” and outlined how to identify and avoid becoming one.

What then, is cargo cult science? Cargo cults were first described during the Second World War, when Melanesian and other Pacific Islanders sought to capture the technological and economic powers of the Allied and Japanese forces who would frequently land there to trade cargo for goods. Convinced by their spiritual leaders that such awesome wealth and technological power would be shared with them, the islands formed ritualistic cults fetishizing outward characteristics of the foreign powers. By wearing their uniforms, making totem rifles, and marching around the beach, these cults hoped that the gods would also bestow upon them the same powers of those they imitated.

Much like these namesake cults, Feynman described cargo cult science as generally being that which sought the appeal and authority of the scientific method, but which failed to live up to its standard in several key regards. First and foremost, a key quality lacking in cargo cult science was a radical commitment to scientific integrity – a commitment to acting as one's own harshest critic. Other key signs of possible cultism included: (1) a failure to engage critically with both the strengths and faults of any theoretical or empirical postulate, (2) an insistence on doing pseudo-experiments that could not have come out otherwise, (3) a kind of ahistorical perspective in which key data points or advances are overlooked or ignored entirely, and crucially, (4) a surface appeal to explanatory devices or scientific concepts without a deeper engagement.

In recent years, I have observed a steady growth of these sorts of errors in the predictive processing literature. Chief among these is a casual, devil-may-care appropriation of computational concepts and the outward appearances of computationalism without a deeper engagement. This takes several forms: For example, the description of pseudo-equations as Bayesian “models,”Footnote 1 or the frequent introduction of new psychological “theories” rehashing concepts such as “priors,” “prediction errors,” “precision,” or “Markov Blankets” as explanatory in and of themselves.Footnote 2 A more recent trend is found in the burgeoning volume of so-called “in silico” demonstrations, wherein an off the shelf Markovian model is merely re-parameterized and then described as a new “model” of some complex phenomenon – emotion (Hesp et al., Reference Hesp, Smith, Parr, Allen, Friston and Ramstead2021), ecological niche construction (Bruineberg, Rietveld, Parr, van Maanen, & Friston, Reference Bruineberg, Rietveld, Parr, van Maanen and Friston2018), or interoception (Allen, Levy, Parr, & Friston, Reference Allen, Levy, Parr and Friston2019) are all salient examples. Typically, such demonstrations involve minimal reshaping of the underlying models themselves, which is remarkable considering the breadth of topics to which they are applied.

A similar error can be found frequently in empirical studies of various psychological phenomena – a kind of computational prestidigitation, in which a construct such as “precision” is appealed to, an experiment is conducted, and then a paper produced that proudly claims to have provided evidence for the underlying computational theory. For example, experiment in which attention to the body is manipulated, and some consequent alteration in an ambiguous data feature is observed, which is then interpreted unambiguously as “evidence for precision weighting” (Petzschner et al., Reference Petzschner, Weber, Wellstein, Paolini, Do and Stephan2019). While predictive processing will certainly claim the credit here, it seems obvious that in the absence of an actual model fitting procedure, just about any psychological or computational theory could explain the obtained results. This trick is pervasive in a new flood of psychological and neuroscientific experiments in which attention, expectation, confidence, or other concepts with a similar sounding cousin in formal theory of can be found and manipulated, a high impact paper produced, and no attempt at true falsification made.

The issue is of course that across all these examples there is a failure to engage critically and directly with the underlying formal constructs, and a commensurate failure to apply appropriate computational methods to enable falsification and ultimately, safeguard against pseudoscience. Critical steps for establishing Feynman's radical honesty – such as model cross-validation, model falsification, or even fitting to empirical data at all, are few and far between (Palminteri, Wyart, & Koechlin, Reference Palminteri, Wyart and Koechlin2017; Wilson & Collins, Reference Wilson and Collins2019). It is salient then that in the same lecture, Feynman famously warned of his “first principle.” Too many of us increasingly risk violating these ideals, perhaps in hopes of riding the Bayesian wave to the promised land of high impact computational neuroscience papers. I applaud Bruineberg and colleagues for showing us how to leave the cargo cult behind.

Financial support

MA is supported by a Lundbeckfonden Fellowship (under Grant R272-2017-4345), and the AIAS-COFUND II fellowship programme that is supported by the Marie Skłodowska-Curie actions under the European Union's Horizon 2020 (under Grant 754513), and the Aarhus University Research Foundation.

Conflict of interest

None.

Footnotes

1. See, e.g., “This model is formalized by the following equation: P(Mommy|Interoception, Exteroception) ∝ P(Mommy) × P(Interoception|Mommy) × P(Exteroception|Mommy),” from Atzil, Gao, Fradkin, & Barrett (Reference Atzil, Gao, Fradkin and Barrett2018). See another similar example in Allen and Tsakiris (Reference Allen, Tsakiris, Tsakiris and De Preester2018).

2. Aptly dubbed the “Bayes Glaze” by anonymous twitter commentator, @Neuroskeptic.

References

Allen, M. (2018). The foundation: Mechanism, prediction, and falsification in Bayesian enactivism: Comment on “Answering Schrödinger's question: A free-energy formulation” by Maxwell James Désormeau Ramstead et al. Physics of Life Reviews, 24, 1720. https://doi.org/10.1016/j.plrev.2018.01.007CrossRefGoogle Scholar
Allen, M., Levy, A., Parr, T., & Friston, K. J. (2019). In the body's eye: The computational anatomy of interoceptive inference. BioRxiv, 603928. https://doi.org/10.1101/603928Google Scholar
Allen, M., & Tsakiris, M. (2018). The body as first prior: Interoceptive predictive processing and the primacy. In Tsakiris, M., & De Preester, H. (Eds.), The interoceptive mind: From homeostasis to Awareness (p. 27). Oxford Univeristy Press.Google Scholar
Atzil, S., Gao, W., Fradkin, I., & Barrett, L. F. (2018). Growing a social brain. Nature Human Behaviour, 2(9), 624636. https://doi.org/10.1038/s41562-018-0384-6CrossRefGoogle ScholarPubMed
Bruineberg, J., Rietveld, E., Parr, T., van Maanen, L., & Friston, K. J. (2018). Free-energy minimization in joint agent-environment systems: A niche construction perspective. Journal of Theoretical Biology, 455, 161178. https://doi.org/10.1016/j.jtbi.2018.07.002CrossRefGoogle ScholarPubMed
Calvo, P., & Friston, K. (2017). Predicting green: Really radical (plant) predictive processing. Journal of the Royal Society Interface, 14(131), 20170096. https://doi.org/10.1098/rsif.2017.0096CrossRefGoogle ScholarPubMed
Feynman, R. P. (1974). Cargo cult science. Engineering and Science, 37(7), 1013.Google Scholar
Hesp, C., Smith, R., Parr, T., Allen, M., Friston, K. J., & Ramstead, M. J. D. (2021). Deeply felt affect: The emergence of valence in deep active inference. Neural Computation, 33(2), 398446. https://doi.org/10.1162/neco_a_01341CrossRefGoogle ScholarPubMed
Kirchhoff, M., Parr, T., Palacios, E., Friston, K., & Kiverstein, J. (2018). The Markov blankets of life: Autonomy, active inference and the free energy principle. Journal of the Royal Society Interface, 15(138), 20170792. https://doi.org/10.1098/rsif.2017.0792CrossRefGoogle ScholarPubMed
Palminteri, S., Wyart, V., & Koechlin, E. (2017). The importance of falsification in computational cognitive modeling. Trends in Cognitive Sciences, 21(6), 425433. https://doi.org/10.1016/j.tics.2017.03.011CrossRefGoogle ScholarPubMed
Petzschner, F. H., Weber, L. A., Wellstein, K. V., Paolini, G., Do, C. T., & Stephan, K. E. (2019). Focus of attention modulates the heartbeat evoked potential. NeuroImage, 186, 595606. https://doi.org/10.1016/j.neuroimage.2018.11.037CrossRefGoogle ScholarPubMed
Ramstead, M. J. D., Badcock, P. B., & Friston, K. J. (2018). Answering Schrödinger's question: A free-energy formulation. Physics of Life Reviews, 24, 116. https://doi.org/10.1016/j.plrev.2017.09.001CrossRefGoogle ScholarPubMed
Veissière, S. P. L., Constant, A., Ramstead, M. J. D., Friston, K. J., & Kirmayer, L. J. (2019). Thinking through other minds: A variational approach to cognition and culture. Behavioral and Brain Sciences, 43, e90. https://doi.org/10.1017/S0140525X19001213CrossRefGoogle ScholarPubMed
Wilson, R. C., & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. ELife, 8, e49547. https://doi.org/10.7554/eLife.49547CrossRefGoogle ScholarPubMed