The first principle is that you must not fool yourself – and you are the easiest person to fool.
— Richard Feynman (Reference Feynman1974)
In their compelling arguments, Bruineberg and colleagues reveal how the mathematical, Bayesian construct of a “Markov blanket” has become a go-to explanans in topics as far-reaching as neuroscience, sociology, the philosophy of mind, and epistemology. Through careful analyses of the prerequisite formalisms, they reveal how many of these works confuse a realist understanding of Markov blankets with their actual properties as defined by formal mathematics. A consequence of this confusion is that Markov blankets are ascribed properties they do not possess and are frequently leveraged to explain phenomena for which they have little direct relevance. Indeed, it has been argued Markov blankets demarcate the definition of life (Kirchhoff, Parr, Palacios, Friston, & Kiverstein, Reference Kirchhoff, Parr, Palacios, Friston and Kiverstein2018), can guide an Asimov-like attempt at psychohistory (Allen, Reference Allen2018; Ramstead, Badcock, & Friston, Reference Ramstead, Badcock and Friston2018; Veissière, Constant, Ramstead, Friston, & Kirmayer, Reference Veissière, Constant, Ramstead, Friston and Kirmayer2019), and even extend cognition to plants (Calvo & Friston, Reference Calvo and Friston2017). Bruineberg and colleagues argue that these and many similar arguments fail to capture what is and is not offered by Markov blankets and proffer a helpful framework for understanding and applying Markovian concepts, based on an informed analysis of their actual formal properties.
As I read the target article, I could not help but think of the late Richard Feynman's now infamous remarks, delivered to the Caltech class of 1974, in which he warned of the dangers of “cargo cult science” (Feynman, Reference Feynman1974). Prior to the commencement address, Feynman visited the Esalen Institute, a well-known nexus for “alternative” science. He recounted how, to his surprise, many of the advocates of esoteric mysticism and parapsychology he met there inevitably presented their ideas as scientific, when they were clearly anything but. But perhaps more worrying, he also remarked how many of the styles of argumentation he encountered could also be found in mainstream fields of psychology, neuroscience, and even physics. Feynman dubbed these trends as “cargo cult science” and outlined how to identify and avoid becoming one.
What then, is cargo cult science? Cargo cults were first described during the Second World War, when Melanesian and other Pacific Islanders sought to capture the technological and economic powers of the Allied and Japanese forces who would frequently land there to trade cargo for goods. Convinced by their spiritual leaders that such awesome wealth and technological power would be shared with them, the islands formed ritualistic cults fetishizing outward characteristics of the foreign powers. By wearing their uniforms, making totem rifles, and marching around the beach, these cults hoped that the gods would also bestow upon them the same powers of those they imitated.
Much like these namesake cults, Feynman described cargo cult science as generally being that which sought the appeal and authority of the scientific method, but which failed to live up to its standard in several key regards. First and foremost, a key quality lacking in cargo cult science was a radical commitment to scientific integrity – a commitment to acting as one's own harshest critic. Other key signs of possible cultism included: (1) a failure to engage critically with both the strengths and faults of any theoretical or empirical postulate, (2) an insistence on doing pseudo-experiments that could not have come out otherwise, (3) a kind of ahistorical perspective in which key data points or advances are overlooked or ignored entirely, and crucially, (4) a surface appeal to explanatory devices or scientific concepts without a deeper engagement.
In recent years, I have observed a steady growth of these sorts of errors in the predictive processing literature. Chief among these is a casual, devil-may-care appropriation of computational concepts and the outward appearances of computationalism without a deeper engagement. This takes several forms: For example, the description of pseudo-equations as Bayesian “models,”Footnote 1 or the frequent introduction of new psychological “theories” rehashing concepts such as “priors,” “prediction errors,” “precision,” or “Markov Blankets” as explanatory in and of themselves.Footnote 2 A more recent trend is found in the burgeoning volume of so-called “in silico” demonstrations, wherein an off the shelf Markovian model is merely re-parameterized and then described as a new “model” of some complex phenomenon – emotion (Hesp et al., Reference Hesp, Smith, Parr, Allen, Friston and Ramstead2021), ecological niche construction (Bruineberg, Rietveld, Parr, van Maanen, & Friston, Reference Bruineberg, Rietveld, Parr, van Maanen and Friston2018), or interoception (Allen, Levy, Parr, & Friston, Reference Allen, Levy, Parr and Friston2019) are all salient examples. Typically, such demonstrations involve minimal reshaping of the underlying models themselves, which is remarkable considering the breadth of topics to which they are applied.
A similar error can be found frequently in empirical studies of various psychological phenomena – a kind of computational prestidigitation, in which a construct such as “precision” is appealed to, an experiment is conducted, and then a paper produced that proudly claims to have provided evidence for the underlying computational theory. For example, experiment in which attention to the body is manipulated, and some consequent alteration in an ambiguous data feature is observed, which is then interpreted unambiguously as “evidence for precision weighting” (Petzschner et al., Reference Petzschner, Weber, Wellstein, Paolini, Do and Stephan2019). While predictive processing will certainly claim the credit here, it seems obvious that in the absence of an actual model fitting procedure, just about any psychological or computational theory could explain the obtained results. This trick is pervasive in a new flood of psychological and neuroscientific experiments in which attention, expectation, confidence, or other concepts with a similar sounding cousin in formal theory of can be found and manipulated, a high impact paper produced, and no attempt at true falsification made.
The issue is of course that across all these examples there is a failure to engage critically and directly with the underlying formal constructs, and a commensurate failure to apply appropriate computational methods to enable falsification and ultimately, safeguard against pseudoscience. Critical steps for establishing Feynman's radical honesty – such as model cross-validation, model falsification, or even fitting to empirical data at all, are few and far between (Palminteri, Wyart, & Koechlin, Reference Palminteri, Wyart and Koechlin2017; Wilson & Collins, Reference Wilson and Collins2019). It is salient then that in the same lecture, Feynman famously warned of his “first principle.” Too many of us increasingly risk violating these ideals, perhaps in hopes of riding the Bayesian wave to the promised land of high impact computational neuroscience papers. I applaud Bruineberg and colleagues for showing us how to leave the cargo cult behind.
The first principle is that you must not fool yourself – and you are the easiest person to fool.
— Richard Feynman (Reference Feynman1974)In their compelling arguments, Bruineberg and colleagues reveal how the mathematical, Bayesian construct of a “Markov blanket” has become a go-to explanans in topics as far-reaching as neuroscience, sociology, the philosophy of mind, and epistemology. Through careful analyses of the prerequisite formalisms, they reveal how many of these works confuse a realist understanding of Markov blankets with their actual properties as defined by formal mathematics. A consequence of this confusion is that Markov blankets are ascribed properties they do not possess and are frequently leveraged to explain phenomena for which they have little direct relevance. Indeed, it has been argued Markov blankets demarcate the definition of life (Kirchhoff, Parr, Palacios, Friston, & Kiverstein, Reference Kirchhoff, Parr, Palacios, Friston and Kiverstein2018), can guide an Asimov-like attempt at psychohistory (Allen, Reference Allen2018; Ramstead, Badcock, & Friston, Reference Ramstead, Badcock and Friston2018; Veissière, Constant, Ramstead, Friston, & Kirmayer, Reference Veissière, Constant, Ramstead, Friston and Kirmayer2019), and even extend cognition to plants (Calvo & Friston, Reference Calvo and Friston2017). Bruineberg and colleagues argue that these and many similar arguments fail to capture what is and is not offered by Markov blankets and proffer a helpful framework for understanding and applying Markovian concepts, based on an informed analysis of their actual formal properties.
As I read the target article, I could not help but think of the late Richard Feynman's now infamous remarks, delivered to the Caltech class of 1974, in which he warned of the dangers of “cargo cult science” (Feynman, Reference Feynman1974). Prior to the commencement address, Feynman visited the Esalen Institute, a well-known nexus for “alternative” science. He recounted how, to his surprise, many of the advocates of esoteric mysticism and parapsychology he met there inevitably presented their ideas as scientific, when they were clearly anything but. But perhaps more worrying, he also remarked how many of the styles of argumentation he encountered could also be found in mainstream fields of psychology, neuroscience, and even physics. Feynman dubbed these trends as “cargo cult science” and outlined how to identify and avoid becoming one.
What then, is cargo cult science? Cargo cults were first described during the Second World War, when Melanesian and other Pacific Islanders sought to capture the technological and economic powers of the Allied and Japanese forces who would frequently land there to trade cargo for goods. Convinced by their spiritual leaders that such awesome wealth and technological power would be shared with them, the islands formed ritualistic cults fetishizing outward characteristics of the foreign powers. By wearing their uniforms, making totem rifles, and marching around the beach, these cults hoped that the gods would also bestow upon them the same powers of those they imitated.
Much like these namesake cults, Feynman described cargo cult science as generally being that which sought the appeal and authority of the scientific method, but which failed to live up to its standard in several key regards. First and foremost, a key quality lacking in cargo cult science was a radical commitment to scientific integrity – a commitment to acting as one's own harshest critic. Other key signs of possible cultism included: (1) a failure to engage critically with both the strengths and faults of any theoretical or empirical postulate, (2) an insistence on doing pseudo-experiments that could not have come out otherwise, (3) a kind of ahistorical perspective in which key data points or advances are overlooked or ignored entirely, and crucially, (4) a surface appeal to explanatory devices or scientific concepts without a deeper engagement.
In recent years, I have observed a steady growth of these sorts of errors in the predictive processing literature. Chief among these is a casual, devil-may-care appropriation of computational concepts and the outward appearances of computationalism without a deeper engagement. This takes several forms: For example, the description of pseudo-equations as Bayesian “models,”Footnote 1 or the frequent introduction of new psychological “theories” rehashing concepts such as “priors,” “prediction errors,” “precision,” or “Markov Blankets” as explanatory in and of themselves.Footnote 2 A more recent trend is found in the burgeoning volume of so-called “in silico” demonstrations, wherein an off the shelf Markovian model is merely re-parameterized and then described as a new “model” of some complex phenomenon – emotion (Hesp et al., Reference Hesp, Smith, Parr, Allen, Friston and Ramstead2021), ecological niche construction (Bruineberg, Rietveld, Parr, van Maanen, & Friston, Reference Bruineberg, Rietveld, Parr, van Maanen and Friston2018), or interoception (Allen, Levy, Parr, & Friston, Reference Allen, Levy, Parr and Friston2019) are all salient examples. Typically, such demonstrations involve minimal reshaping of the underlying models themselves, which is remarkable considering the breadth of topics to which they are applied.
A similar error can be found frequently in empirical studies of various psychological phenomena – a kind of computational prestidigitation, in which a construct such as “precision” is appealed to, an experiment is conducted, and then a paper produced that proudly claims to have provided evidence for the underlying computational theory. For example, experiment in which attention to the body is manipulated, and some consequent alteration in an ambiguous data feature is observed, which is then interpreted unambiguously as “evidence for precision weighting” (Petzschner et al., Reference Petzschner, Weber, Wellstein, Paolini, Do and Stephan2019). While predictive processing will certainly claim the credit here, it seems obvious that in the absence of an actual model fitting procedure, just about any psychological or computational theory could explain the obtained results. This trick is pervasive in a new flood of psychological and neuroscientific experiments in which attention, expectation, confidence, or other concepts with a similar sounding cousin in formal theory of can be found and manipulated, a high impact paper produced, and no attempt at true falsification made.
The issue is of course that across all these examples there is a failure to engage critically and directly with the underlying formal constructs, and a commensurate failure to apply appropriate computational methods to enable falsification and ultimately, safeguard against pseudoscience. Critical steps for establishing Feynman's radical honesty – such as model cross-validation, model falsification, or even fitting to empirical data at all, are few and far between (Palminteri, Wyart, & Koechlin, Reference Palminteri, Wyart and Koechlin2017; Wilson & Collins, Reference Wilson and Collins2019). It is salient then that in the same lecture, Feynman famously warned of his “first principle.” Too many of us increasingly risk violating these ideals, perhaps in hopes of riding the Bayesian wave to the promised land of high impact computational neuroscience papers. I applaud Bruineberg and colleagues for showing us how to leave the cargo cult behind.
Financial support
MA is supported by a Lundbeckfonden Fellowship (under Grant R272-2017-4345), and the AIAS-COFUND II fellowship programme that is supported by the Marie Skłodowska-Curie actions under the European Union's Horizon 2020 (under Grant 754513), and the Aarhus University Research Foundation.
Conflict of interest
None.