Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-06T14:22:25.992Z Has data issue: false hasContentIssue false

Computation, perception, and mind

Published online by Cambridge University Press:  23 March 2022

Jerome A. Feldman*
Affiliation:
ICSI and University of California, Berkeley, Berkeley, CA94704, USA. Feldman@icsi.berkeley.edu

Abstract

Advances in behavioral and brain sciences have engendered wide ranging efforts to help understand consciousness. The target article suggests that abstract computational models (such as integrated information theory [IIT]) are ill-advised. This commentary broadens the discussion to include mysteries of subjective experience that are inconsistent with current neuroscience. It also discusses progress being made through demystifying specific cases and pursuing evolutionary considerations.

Type
Open Peer Commentary
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

The target article proposes that integrated information theory (IIT), a computational formalism, is “A case of mistaken identity.” I concur and further suggest that the failure of the IIT identity assumption is an instance of a deeper problem, the widespread focus on computational models of the mind that are independent of both neural structure and subjective experience (Melloni et al., Reference Melloni2021). The target article's main suggestion, better formalisms, does not help much, but their proposed re-orientation to constitutive questions would be a major advance. An excellent new review of historical and ongoing attempts to explain the experience of the mind is (Cobb, Reference Cobb2020).

The ancient mind-body problem manifests in everyday experience. As one example consider the text below:

{\hbox{U Q C G O}} + {\hbox{C}}\bf{ O U Q G}

Close or cover one eye, focus on the +, and try to read the letters to the left. You will find it much easier to read the letters on the right. Neural encoding of fine detail is restricted to the fovea, covering about the size of your thumbnail at arms-length. Our experience of a stable high-resolution scene is inconsistent with conventional science of the visual system, as well as any proposed alternative (Feldman, Reference Feldman2012). A related phenomenon, the experience of a visual ego-center, is discussed in the target article. There are myriad of other inconsistency examples. If you touch your nose with your thumb, the contact feels simultaneous despite the large difference in neural conduction times.

The target article and IIT are part of the scientific approach to the mind-body problem and consciousness. Philosophers march to a different drummer, but both would be well served by recognizing the fundamental mysteries of mind.

The most beautiful and profound experience is the feeling of mystery. It underlies religion as well as all deeper aspirations in art and science. –Einstein

No one has proposed a physical explanation for the inconsistency of neuroscience and much subjective experience and this is seldom acknowledged.

Some investigators, foregoing grand theories of consciousness, are pursuing two productive scientific approaches to these mysteries: demystification and evolution. Much scientific progress is demystification, often involving inconsistency of theories and findings. One current mental demystification involves blindsight (Fox, Goodale, & Bourne, Reference Fox, Goodale and Bourne2020), until recently considered a major mystery. People with certain deficits are unable to report what they see, but can carry out appropriate actions such as grasping a tool. In retrospect, the mystery of blindsight arose from the simplistic assumption that visual perception was a single integrated function. Actually, there are several interconnected vision networks, including some, like reflexes, that do not require conscious perception. The exploration of multiple visual systems is also having clinical applications (Fox et al., Reference Fox, Goodale and Bourne2020). Extending this hybrid approach, an ambitious going project (Hesse & Tsao, Reference Hesse and Tsao2020), incorporates a wide range of experimental and computational methods in revealing the aspects of face perception in people and other primates.

Any physicalist explanation of the mind will need to incorporate an evolutionary story (Feldman, Reference Feldman2020). There are already useful studies of possible precursors of human consciousness (Merker, Reference Merker2007). These employ the modern eco-evo-devo formulation of evolution. All creatures need to incorporate a notion of valence, sensing what is good and bad for the creature. One informative example is the color constancy of the hawkmoth (Balkenius & Kelber, Reference Balkenius and Kelber2004). These moths recognize the same flower color under changing illumination conditions. This ability is especially important for foragers that are active under a range of lighting conditions. Receptor adaptation contributes a large part to color constancy; the sensitivity of specific receptors decreases as a result of adaptation of the photoreceptor cells being stimulated by the background illumination. Importantly, color constancy evolved by natural selection; moths with better vision tend to prosper. No learning by the moth is needed.

The moth story suggests a general explanation for the adaptive utility of subjective experience. The role of perception is to guide action and, in more complex creatures, memory, and planning (simulation). Successful action requires percepts that capture relevant aspects of the environment (umwelt), such as the color of flowers for the moth. In the human case, there need to be actionable internal representations of the external physical and social world. In many cases, such as my examples above, we also know that the actionable information is not captured by our sensors and is also not encodable as conventional neural systems (Feldman, Reference Feldman2012). Informal experiments as well as intuition show that the information captured by our sensors and expressed in the known brain circuits does not suffice for human functioning. One mystery is how the ecologically required information is represented in the brain. A further mystery is how we come to be aware of (some of) this information.

In a few remarkable cases (Iriki & Taoka, Reference Iriki and Taoka2012), we also know the neural correlates of the embodied experience of external objects such as the blind man's cane. For many years, tool embodiment was treated as a deep mystery (Pazzaglia, Reference Pazzaglia2020). In a 1996 experiment, Iriki et al. trained macaque monkeys to retrieve distant food using a rake, and recorded neuronal activity in the caudal postcentral gyrus where the somatosensory and visual signals converge (Iriki et al., Reference Iriki1996). There they found a large number of bimodal neurons, which appeared to code the body schema of the hand. After extensive tool use, the visual receptive fields of these cells adapted to include the entire length of the rake or cover the expanded accessible space.

Subsequent developments have included attempts to improve human prostheses by generating this kind of subjective embodiment (D'Anna et al., Reference D'Anna, Valle, Mazzoni, Strauss, Iberite, Patton and Micera2019). With training, people can improve performance by developing direct embodiment of a prosthetic arm with indirect feedback or, better, with direct connection to appropriate nerves. Of course, the mechanism of subjective experience is still unknown.

The many remaining mysteries of consciousness strongly suggest that any reductionist solution of the mind-body will require scientific reconceptualization on the scale of general relativity or quantum theory. Both of these groundbreaking theories are famously unintuitive and any physical explanation of consciousness, if one is discovered, may well be equally challenging.

Financial support

This work was partially supported by a grant from Google.

Conflict of interest

None.

References

Balkenius, A., & Kelber, A. (2004). Colour constancy in diurnal and nocturnal hawkmoths. Journal of Experimental Biology, 207, 33073316.CrossRefGoogle ScholarPubMed
Cobb, M. (2020). The idea of the brain: A history. Profile.Google Scholar
D'Anna, E., Valle, G., Mazzoni, A., Strauss, I., Iberite, F., Patton, J., … Micera, S. (2019). A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Science Robotics, 4, eaau8892.Google ScholarPubMed
Feldman, J. (2012). Mysteries of visual experience. ArXiv. http://arxiv.org/abs/1604.08612Google Scholar
Feldman, J. (2020) On the Evolution of Subjective Experience. http//arXiv.org/abs/2008.08073Google Scholar
Fox, D., Goodale, M. A., & Bourne, J. A. (2020). The age-dependent neural substrates of blindsight. Trends in Neurosciences, 43(4), 242252.CrossRefGoogle ScholarPubMed
Hesse, J., & Tsao, D. Y. (2020). The macaque face patch system: A turtle's underbelly for the brain. Nature Reviews Neuroscience, 21(12), 695716. doi: 10.1038/s41583-020-00393-wCrossRefGoogle ScholarPubMed
Iriki, A., … (1996). Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport, 7(14), 23252330.Google ScholarPubMed
Iriki, A., & Taoka, M. (2012). Triadic (ecological, neural, cognitive) niche construction: A scenario of human brain evolution extrapolating tool use and language from the control of reaching actions. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 367(1585), 1023.CrossRefGoogle ScholarPubMed
Melloni, L., … (2021). Making the hard problem of consciousness easier. Science (New York, N.Y.), 372(6545), 911912.CrossRefGoogle ScholarPubMed
Merker, B. (2007). Consciousness without cerebral cortex: A challenge for neuroscience and medicine. Behavioral and Brain Sciences, 30, 63134.CrossRefGoogle ScholarPubMed
Pazzaglia, M. (Ed.) (2020). Embodying tool use: From cognition to neurorehabilitation. Frontiers Research Topic. Published on 11 February 2020. Frontiers in Human Neuroscience.CrossRefGoogle Scholar