Although we believe that Firestone & Scholl (F&S) offer sound advice for culling theoretical excesses, here we argue that (1) contemporary advances within the neurosciences constitute legitimate challenges for foundational concepts invoked within the classical cognitive architecture of perception, and (2) the wider literature in fact does provide compelling support for the effects of motivational factors on perceptual processes.
The commentary authors briefly consider how systems neuroscience might inform their discussion, but they largely dismiss what knowledge of “descending neural pathways” (sect. 2.2) might be able to contribute to this debate on the basis that such knowledge is not novel. We respectfully disagree with that position. It must eventually be possible to relate any useful cognitive architecture to neurophysiology, and hence it is desirable to respect such constraints when they are known and interpretable. Franconeri et al. (Reference Franconeri, Alvarez and Cavanagh2013), for example, used well-established principles of cortical organization to provide substance to the previously fuzzy concept of cognitive “resources” in a way that is intuitive and coherent.
Let us take visual cortex (V1) as the prototypical sensory system. Classical feedforward feature-detector models account for only about 40% of the variance in V1 function (Carandini et al. Reference Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant and Rust2005). In a slightly more pessimistic estimate, Olshausen and Field (Reference Olshausen and Field2005) quantify our epistemic uncertainty concerning V1 functional properties to be closer to 85%. This point is not meant to disparage the remarkable progress of visual neuroscience research, but simply to demonstrate that it seems premature to claim an understanding of perception's architecture that is comprehensive enough to preclude original neurophysiological insights. Our current state of uncertainty about the earliest stages of vision appears less surprising if we consider a few salient neuroanatomical facts. Only about 5% of the excitatory synaptic input to layer IV of V1 derives from geniculate drive (Douglas & Martin Reference Douglas and Martin2007), and approximately 60%–80% of V1 responses are attributable to other V1 neurons or nongeniculate inputs (Muckli & Petro Reference Muckli and Petro2013). Such structural features ensure that primary visual cortex sustains multiple interactions with high-level sources of information. Recent developments in high-resolution tract tracing (Markov & Kennedy Reference Markov and Kennedy2013) and considerations of electrophysiological timing (Briggs & Usrey Reference Briggs and Usrey2005) suggest that perception is more properly conceptualized as a dynamically reverberating loop rather than encapsulated bottom-up and top-down streams. This revised conceptualization makes the prospect of carving the underlying machinery at the joints much more daunting. Indeed, within contemporary systems neuroscience, the organism's internal context is recognized to be as important for unraveling the nature of sensory processing as are the physical parameters of stimuli (Fontanini & Katz Reference Fontanini and Katz2008). From a neurobiological perspective, it is not so much a question of if perception is penetrable, but to what degree and under what circumstances the dynamics might change (Muckli Reference Muckli2010).
Research conducted in alert animals has long recognized that motivational factors rapidly and profoundly influence neuronal responses at the earliest modality-specific perceptual stages, resulting in increased gain and/or altered tuning curves (McGann Reference McGann2015). Findings collected across a range of mammalian species have demonstrated tonotopic map remodeling within primary auditory cortex that optimizes the processing of tones paired with rewards or punishers (Weinberger Reference Weinberger2004). The amount of representational area expansion for conditioned stimuli in primary sensory cortices may encode the magnitude of affective relevance (Rutkowski & Weinberger Reference Rutkowski and Weinberger2005) and even predict subsequent extinction learning (Bieszczad & Weinberger Reference Bieszczad and Weinberger2010). Research in humans has also documented pronounced time-varying changes for conditioned cues across several sensory cortices (Miskovic & Keil Reference Miskovic and Keil2012).
The preceding cases are well-established but modest demonstrations of perceptual penetrability. We would like to advance as a hypothesis a strong version of penetrability, according to which primitive affective qualities such as hedonic valence might be understood as perceptual attributes that are represented alongside other, more objective, physical properties. This strong version is therefore closer in nature to Wündt's (Reference Wündt1897) insights about the central role of affect in perception. This proposition suggests that the affective dimensions of perceptual experience enjoy a neural currency that is not altogether dissimilar from the dimensions that reflect the physics of stimuli (e.g., light wavelength).
Elementary valence attributes might be embedded within modality-specific sensory cortices in population codes – distributed activity, within or across brain regions, that represents the relationships between stimulus or experiential properties and their distances in a high-dimensional space (Kriegeskorte & Kievit Reference Kriegeskorte and Kievit2013). We recently employed a representational similarity analysis of blood-oxygen-level dependent (BOLD) signals to examine how external events are represented as pleasant or unpleasant, alongside other physical (e.g., low or high luminance) and semantic (e.g., representing either animate or inanimate objects) properties (Chikazoe et al., Reference Chikazoe, Lee, Kriegeskorte and Anderson2014). We found that activity patterns in the ventral temporal cortex and the anterior insular cortex contained the representational geometry of modally bound valence representations belonging to the visual and gustatory systems, respectively. In addition to such modality-specific representations, we also found evidence for a population code in orbitofrontal cortex that is shared across events originating from distinct modalities, which presumably allows subjective affect to be objectively quantified and compared on a common valence axis. That an aversive image and an acrid taste are both experienced as hedonically unpleasant may therefore be by virtue of their objective similarity in distributed neural population codes – a transfer function is interposed between purely physical sensations and their elementary valence. Whereas 680 nm of light might carry information related to the perceptual experience of red, how the affective tone of experience affects the observer may emerge from higher-level processes that become partially embedded within perceptual representations.
In short, we believe that analogies to dynamically reverberating loops and principles of reciprocal causation provide a much closer approximation to the ways that brains function, and that these ideas necessitate a more thoroughgoing reevaluation of many cognitivist axioms. It is quite possible – indeed it seems likely – that static distinctions between perception, cognition, and emotion reflect much more about historical intellectual biases in the field of cognitive science than about the true operations of the brain/mind.
Although we believe that Firestone & Scholl (F&S) offer sound advice for culling theoretical excesses, here we argue that (1) contemporary advances within the neurosciences constitute legitimate challenges for foundational concepts invoked within the classical cognitive architecture of perception, and (2) the wider literature in fact does provide compelling support for the effects of motivational factors on perceptual processes.
The commentary authors briefly consider how systems neuroscience might inform their discussion, but they largely dismiss what knowledge of “descending neural pathways” (sect. 2.2) might be able to contribute to this debate on the basis that such knowledge is not novel. We respectfully disagree with that position. It must eventually be possible to relate any useful cognitive architecture to neurophysiology, and hence it is desirable to respect such constraints when they are known and interpretable. Franconeri et al. (Reference Franconeri, Alvarez and Cavanagh2013), for example, used well-established principles of cortical organization to provide substance to the previously fuzzy concept of cognitive “resources” in a way that is intuitive and coherent.
Let us take visual cortex (V1) as the prototypical sensory system. Classical feedforward feature-detector models account for only about 40% of the variance in V1 function (Carandini et al. Reference Carandini, Demb, Mante, Tolhurst, Dan, Olshausen, Gallant and Rust2005). In a slightly more pessimistic estimate, Olshausen and Field (Reference Olshausen and Field2005) quantify our epistemic uncertainty concerning V1 functional properties to be closer to 85%. This point is not meant to disparage the remarkable progress of visual neuroscience research, but simply to demonstrate that it seems premature to claim an understanding of perception's architecture that is comprehensive enough to preclude original neurophysiological insights. Our current state of uncertainty about the earliest stages of vision appears less surprising if we consider a few salient neuroanatomical facts. Only about 5% of the excitatory synaptic input to layer IV of V1 derives from geniculate drive (Douglas & Martin Reference Douglas and Martin2007), and approximately 60%–80% of V1 responses are attributable to other V1 neurons or nongeniculate inputs (Muckli & Petro Reference Muckli and Petro2013). Such structural features ensure that primary visual cortex sustains multiple interactions with high-level sources of information. Recent developments in high-resolution tract tracing (Markov & Kennedy Reference Markov and Kennedy2013) and considerations of electrophysiological timing (Briggs & Usrey Reference Briggs and Usrey2005) suggest that perception is more properly conceptualized as a dynamically reverberating loop rather than encapsulated bottom-up and top-down streams. This revised conceptualization makes the prospect of carving the underlying machinery at the joints much more daunting. Indeed, within contemporary systems neuroscience, the organism's internal context is recognized to be as important for unraveling the nature of sensory processing as are the physical parameters of stimuli (Fontanini & Katz Reference Fontanini and Katz2008). From a neurobiological perspective, it is not so much a question of if perception is penetrable, but to what degree and under what circumstances the dynamics might change (Muckli Reference Muckli2010).
Research conducted in alert animals has long recognized that motivational factors rapidly and profoundly influence neuronal responses at the earliest modality-specific perceptual stages, resulting in increased gain and/or altered tuning curves (McGann Reference McGann2015). Findings collected across a range of mammalian species have demonstrated tonotopic map remodeling within primary auditory cortex that optimizes the processing of tones paired with rewards or punishers (Weinberger Reference Weinberger2004). The amount of representational area expansion for conditioned stimuli in primary sensory cortices may encode the magnitude of affective relevance (Rutkowski & Weinberger Reference Rutkowski and Weinberger2005) and even predict subsequent extinction learning (Bieszczad & Weinberger Reference Bieszczad and Weinberger2010). Research in humans has also documented pronounced time-varying changes for conditioned cues across several sensory cortices (Miskovic & Keil Reference Miskovic and Keil2012).
The preceding cases are well-established but modest demonstrations of perceptual penetrability. We would like to advance as a hypothesis a strong version of penetrability, according to which primitive affective qualities such as hedonic valence might be understood as perceptual attributes that are represented alongside other, more objective, physical properties. This strong version is therefore closer in nature to Wündt's (Reference Wündt1897) insights about the central role of affect in perception. This proposition suggests that the affective dimensions of perceptual experience enjoy a neural currency that is not altogether dissimilar from the dimensions that reflect the physics of stimuli (e.g., light wavelength).
Elementary valence attributes might be embedded within modality-specific sensory cortices in population codes – distributed activity, within or across brain regions, that represents the relationships between stimulus or experiential properties and their distances in a high-dimensional space (Kriegeskorte & Kievit Reference Kriegeskorte and Kievit2013). We recently employed a representational similarity analysis of blood-oxygen-level dependent (BOLD) signals to examine how external events are represented as pleasant or unpleasant, alongside other physical (e.g., low or high luminance) and semantic (e.g., representing either animate or inanimate objects) properties (Chikazoe et al., Reference Chikazoe, Lee, Kriegeskorte and Anderson2014). We found that activity patterns in the ventral temporal cortex and the anterior insular cortex contained the representational geometry of modally bound valence representations belonging to the visual and gustatory systems, respectively. In addition to such modality-specific representations, we also found evidence for a population code in orbitofrontal cortex that is shared across events originating from distinct modalities, which presumably allows subjective affect to be objectively quantified and compared on a common valence axis. That an aversive image and an acrid taste are both experienced as hedonically unpleasant may therefore be by virtue of their objective similarity in distributed neural population codes – a transfer function is interposed between purely physical sensations and their elementary valence. Whereas 680 nm of light might carry information related to the perceptual experience of red, how the affective tone of experience affects the observer may emerge from higher-level processes that become partially embedded within perceptual representations.
In short, we believe that analogies to dynamically reverberating loops and principles of reciprocal causation provide a much closer approximation to the ways that brains function, and that these ideas necessitate a more thoroughgoing reevaluation of many cognitivist axioms. It is quite possible – indeed it seems likely – that static distinctions between perception, cognition, and emotion reflect much more about historical intellectual biases in the field of cognitive science than about the true operations of the brain/mind.