Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-02-07T01:37:25.875Z Has data issue: false hasContentIssue false

Attention and multisensory modulation argue against total encapsulation

Published online by Cambridge University Press:  05 January 2017

Benjamin de Haas
Affiliation:
Institute of Cognitive Neuroscience, University College London, WC1N 3AR London, United Kingdom. benjamin.haas.09@ucl.ac.uks.schwarzkopf@ucl.ac.ukhttps://www.ucl.ac.uk/pals/research/experimental-psychology/person/benjamin-de-haas/https://sampendu.wordpress.com/sam-schwarzkopf/ Experimental Psychology, University College London, WC1H 0AP London, United Kingdom
Dietrich Samuel Schwarzkopf
Affiliation:
Institute of Cognitive Neuroscience, University College London, WC1N 3AR London, United Kingdom. benjamin.haas.09@ucl.ac.uks.schwarzkopf@ucl.ac.ukhttps://www.ucl.ac.uk/pals/research/experimental-psychology/person/benjamin-de-haas/https://sampendu.wordpress.com/sam-schwarzkopf/ Experimental Psychology, University College London, WC1H 0AP London, United Kingdom
Geraint Rees
Affiliation:
Institute of Cognitive Neuroscience, University College London, WC1N 3AR London, United Kingdom. benjamin.haas.09@ucl.ac.uks.schwarzkopf@ucl.ac.ukhttps://www.ucl.ac.uk/pals/research/experimental-psychology/person/benjamin-de-haas/https://sampendu.wordpress.com/sam-schwarzkopf/ Wellcome Trust Centre for Neuroimaging, University College London, WC1N 3BG London, United Kingdom. g.rees@ucl.ac.ukhttp://www.fil.ion.ucl.ac.uk/~grees/

Abstract

Firestone & Scholl (F&S) postulate that vision proceeds without any direct interference from cognition. We argue that this view is extreme and not in line with the available evidence. Specifically, we discuss two well-established counterexamples: Attention directly affects core aspects of visual processing, and multisensory modulations of vision originate on multiple levels, some of which are unlikely to fall “within perception.”

Type
Open Peer Commentary
Copyright
Copyright © Cambridge University Press 2016 

Firestone & Scholl (F&S) argue there is no good evidence for cognitive penetration of perception, specifically vision. Instead, they propose, visual processing is informationally encapsulated. Importantly, their version of encapsulation goes beyond Fodor's original proposal “that at least some of the background information at the subject's disposal is inaccessible to at least some of his perceptual mechanisms” (Fodor Reference Fodor1983, p. 66). Their hypothesis is much more ambitious: “perception proceeds without any direct, unmediated influence from cognition” (sect. 5, para. 1). We will refer to this view as total encapsulation.

One possible counterexample to total encapsulation is multisensory modulation. For example, sounds in rapid succession can induce the illusory reappearance of visual flashes (Shams et al. Reference Shams, Kamitani and Shimojo2000). Such reappearances increase objective sensitivity for visual features of the flash (Berger et al. Reference Berger, Martelli and Pelli2003) and are linked to individual structure and function of primary visual cortex (de Haas et al. Reference de Haas, Kanai, Jalkanen and Rees2012; Watkins et al. Reference Watkins, Shams, Tanaka, Haynes and Rees2006). Waving one's hand in front of the eyes can induce visual sensations and enable smooth pursuit eye movements, even in complete darkness (Dieter et al. Reference Dieter, Hu, Knill, Blake and Tadin2014). The duration of sounds can bias the perceived duration of concurrent visual stimuli (Romei et al. Reference Romei, de Haas, Mok and Driver2011), and sensitivity for a brief flash increases parametrically with the duration of a co-occurring sound (de Haas et al. Reference de Haas, Cecere, Cullen, Driver and Romei2013a). The noise level of visual stimulus representations in retinotopic cortex is affected by the (in)congruency of co-occurring sounds (de Haas et al. Reference de Haas, Schwarzkopf, Urner and Rees2013b). Category-specific sounds and visual imagery can be decoded from early visual cortex, even with eyes closed (Vetter et al. Reference Vetter, Smith and Muckli2014), and the same is true for imagined hand actions (Pilgramm et al. Reference Pilgramm, de Haas, Helm, Zentgraf, Stark, Munzert and Krüger2016). At the same time, the location of visual stimuli can bias the perceived origin of sounds (Thomas Reference Thomas1941), and a visible face articulating a syllable can bias the perception of a concurrently presented (different) syllable (McGurk & MacDonald Reference McGurk and MacDonald1976). F&S argue that multisensory effects can be reconciled with total encapsulation. The inflexible nature and short latency of such effects would provide evidence they happen “within perception itself,” rather than reflecting the effect of “more central cognitive processes on perception” (sect. 2.4, para. 1). However, multisensory effects have different temporal latencies and occur at multiple levels of processing, from direct cross-talk between primary sensory areas to top-down feedback from association cortex (de Haas & Rees Reference de Haas, Rees, Shams and Kim2010; Driver & Noesselt Reference Driver and Noesselt2008). They may further be subject to attentional (Navarra et al. Reference Navarra, Alsius, Soto-Faraco and Spence2010), motivational (Bruns et al. Reference Bruns, Maiworm and Röder2014), and expectation-based (Gau & Noppeney Reference Gau and Noppeney2015) modulations. Therefore, evidence regarding a strictly horizontal nature of multisensory effects seems ambiguous at best. If total encapsulation hinges on the hypothesis of strictly horizontal effects, this hypothesis needs to be clearer. Specifically, what type of neural or behavioural evidence could refute it?

A second, perhaps more definitive, counterexample is attentional modulation of vision. F&S acknowledge that attention can change what we see (cf. Anton-Erxleben et al. Reference Anton-Erxleben, Abrams and Carrasco2011; Carrasco et al. Reference Carrasco, Fuller and Ling2008) and that these effects can be under intentional control. For example, voluntary attention can induce changes in the perceived spatial frequency (Abrams et al. Reference Abrams, Barbot and Carrasco2010), contrast (Liu et al. Reference Liu, Abrams and Carrasco2009), and position (Suzuki & Cavanagh Reference Suzuki and Cavanagh1997) of visual stimuli. Withdrawal of attention can induce perceptual blur (Montagna et al. Reference Montagna, Pestilli and Carrasco2009) and reduce visual sensitivity (Carmel et al. Reference Carmel, Thorne, Rees and Lavie2011) and sensory adaptation (Rees et al. Reference Rees, Frith and Lavie1997). Nevertheless, F&S argue for total encapsulation. On such an account, attention would not interfere with visual processing per se but with the input to this process, “similar to changing what we see by moving our eyes” or “turning the lights off” (sect. 4.5, para. 4).

Attention-related spatial distortions and changes in acuity have been linked to effects on the spatial tuning of visual neurons (Anton-Erxleben & Carrasco Reference Anton-Erxleben and Carrasco2013; Baruch & Yeshurun Reference Baruch and Yeshurun2014). Receptive fields can shift and grow towards, or shrink around, attended targets (e.g., Womelsdorf et al. Reference Womelsdorf, Anton-Erxleben and Treue2008). Such effects go beyond mere amplitude modulation and can provide important evidence regarding their locus. In a recent study (de Haas et al. Reference de Haas, Schwarzkopf, Anderson and Rees2014), we investigated the effects of attentional load at fixation on neuronal spatial tuning in early visual cortices. Participants performed either a hard or an easy fixation task while retinotopic mapping stimuli traversed the surrounding visual field. Importantly, stimuli were identical in both conditions – only the task instructions differed. Performing the harder task, and consequently having to pay less attention to the task-irrelevant mapping stimuli (Lavie Reference Lavie2005), yielded a blurrier neural representation of the surround, as well as a centrifugal repulsion of population receptive fields in V1-3 (pRFs; Dumoulin & Wandell Reference Dumoulin and Wandell2008). Importantly, this repulsion in V1-3 was accompanied by a centripetal attraction of pRFs in the intraparietal sulcus (IPS), perhaps because the larger receptive fields in IPS specifically encode the attended location (Klein et al. Reference Klein, Harvey and Dumoulin2014). Critically, retinotopic shifts merely inherited from input modulations cannot trivially explain such opposing shifts, because any such effect should be the same (or very similar) across the visual processing hierarchy.

How can one reconcile these findings with total encapsulation? We can think of only one way: redefining visual processing in a way that excludes processing associated with retinotopic tuning of visual cortex but includes feedback processes from multisensory areas (as outlined in our second paragraph above). Such a re-defintion seems hard to reconcile with the widespread evidence that visually tuned neuronal populations in occipital cortex are involved with visual processing. We instead argue that attentional and multisensory modulations are inconsistent with total encapsulation, and at least here, the line between cognition and perception is blurred. F&S concede that accepting these exceptions to total encapsulation would be far less revolutionary than many of the claims they attack. We second their demand to back up extraordinary claims with rigorous evidence and applaud the standards they propose. Many effects they discuss may indeed fail to meet these standards. But precisely because attentional and multisensory effects are well established, total encapsulation itself strikes us as an extraordinary claim that is not supported by the available evidence.

ACKNOWLEDGMENTS

This work was supported by a research fellowship from the Deutsche Forschungsgemeinschaft (BdH; HA 7574/1-1), European Research Council Starting Grant 310829 (DSS, BdH), and the Wellcome Trust (GR).

References

Abrams, J., Barbot, A. & Carrasco, M. (2010) Voluntary attention increases perceived spatial frequency. Attention, Perception and Psychophysics 72(6):1510–21. doi:10.3758/APP.72.6.1510.Google Scholar
Anton-Erxleben, K., Abrams, J. & Carrasco, M. (2011) Equality judgments cannot distinguish between attention effects on appearance and criterion: A reply to Schneider (2011) Journal of Vision 11(13):8. doi:10.1167/11.13.8.Google Scholar
Anton-Erxleben, K. & Carrasco, M. (2013) Attentional enhancement of spatial resolution: Linking behavioural and neurophysiological evidence. Nature Reviews Neuroscience 14(3):188200. doi:10.1038/nrn3443.Google Scholar
Baruch, O. & Yeshurun, Y. (2014) Attentional attraction of receptive fields can explain spatial and temporal effects of attention. Visual Cognition 22(5):704–36. doi:10.1080/13506285.2014.911235.Google Scholar
Berger, T. D., Martelli, M. & Pelli, D. G. (2003) Flicker flutter: Is an illusory event as good as the real thing? Journal of Vision 3(6):406–12. doi:10:1167/3.6.1.Google Scholar
Bruns, P., Maiworm, M. & Röder, B. (2014) Reward expectation influences audiovisual spatial integration. Attention, Perception and Psychophysics 76(6):1815–27. doi:10.3758/s13414-014-0699-y.CrossRefGoogle ScholarPubMed
Carmel, D., Thorne, J. D., Rees, G. & Lavie, N. (2011) Perceptual load alters visual excitability. Journal of Experimental Psychology: Human Perception and Performance 37(5):1350–60. doi:10.1037/a0024320.Google Scholar
Carrasco, M., Fuller, S. & Ling, S. (2008) Transient attention does increase perceived contrast of suprathreshold stimuli: A reply to Prinzmetal, Long, and Leonhardt (2008) Perception and Psychophysics 70(7):1151–64. doi:10.3758/PP.70.7.1151.Google Scholar
de Haas, B., Cecere, R., Cullen, H., Driver, J. & Romei, V. (2013a) The duration of a co-occurring sound modulates visual detection performance in humans. PLoS ONE 8(1):e54789. doi:10.1371/journal.pone.0054789.Google Scholar
de Haas, B., Kanai, R., Jalkanen, L. & Rees, G. (2012) Grey matter volume in early human visual cortex predicts proneness to the sound-induced flash illusion. Proceedings of the Royal Society B: Biological Sciences 279(1749):4955–61. doi:10.1098/rspb.2012.2132.CrossRefGoogle Scholar
de Haas, B. & Rees, G. (2010) Multiple stages of cross-modal integration in visual processing: Comment on “Crossmodal influences on visual perception” by Shams, L. & Kim, R.. Physics of Life Reviews 7:287–88; discussion 29. PMID 20598657 DOI: 10.1016/j.plrev.2010.06.007.CrossRefGoogle ScholarPubMed
de Haas, B., Schwarzkopf, D. S., Anderson, E. J. & Rees, G. (2014) Perceptual load affects spatial tuning of neuronal populations in human early visual cortex. Current Biology: 24(2):R66–67. doi:10.1016/j.cub.2013.11.061.CrossRefGoogle ScholarPubMed
de Haas, B., Schwarzkopf, D. S., Urner, M. & Rees, G. (2013b) Auditory modulation of visual stimulus encoding in human retinotopic cortex. NeuroImage 70:258–67. doi:10.1016/j.neuroimage.2012.12.061.CrossRefGoogle ScholarPubMed
Dieter, K. C., Hu, B., Knill, D. C., Blake, R. & Tadin, D. (2014) Kinesthesis can make an invisible hand visible. Psychological Science 25(1):6675. doi:10.1177/0956797613497968.Google Scholar
Driver, J. & Noesselt, T. (2008) Multisensory interplay reveals crossmodal influences on “sensory-specific” brain regions, neural responses, and judgments. Neuron 57(1):1123. doi:10.1016/j.neuron.2007.12.013.Google Scholar
Dumoulin, S. O. & Wandell, B. A. (2008) Population receptive field estimates in human visual cortex. NeuroImage 39(2):647–60. doi:10.1016/j.neuroimage.2007.09.034.Google Scholar
Fodor, J. A. (1983) Modularity of mind: An essay on faculty psychology. MIT Press.Google Scholar
Gau, R. & Noppeney, U. (2015) How prior expectations shape multisensory perception. NeuroImage 124(Pt A):876–86. doi:10.1016/j.neuroimage.2015.09.045.Google Scholar
Klein, B. P., Harvey, B. M. & Dumoulin, S. O. (2014) Attraction of position preference by spatial attention throughout human visual cortex. Neuron 84(1):227–37. doi:10.1016/j.neuron.2014.08.047.Google Scholar
Lavie, N. (2005) Distracted and confused?: Selective attention under load. Trends in Cognitive Sciences 9(2):7582. doi:10.1016/j.tics.2004.12.004.CrossRefGoogle ScholarPubMed
Liu, T., Abrams, J. & Carrasco, M. (2009) Voluntary attention enhances contrast appearance. Psychological Science 20(3):354–62.CrossRefGoogle ScholarPubMed
McGurk, H. & MacDonald, J. (1976) Hearing lips and seeing voices. Nature 264:746–48.CrossRefGoogle ScholarPubMed
Montagna, B., Pestilli, F. & Carrasco, M. (2009) Attention trades off spatial acuity. Vision Research 49(7):735–45.Google Scholar
Navarra, J., Alsius, A., Soto-Faraco, S. & Spence, C. (2010) Assessing the role of attention in the audiovisual integration of speech. Information Fusion 11(1):411. doi:10.1016/j.inffus.2009.04.001.Google Scholar
Pilgramm, S., de Haas, B., Helm, F., Zentgraf, K., Stark, R., Munzert, J. & Krüger, B. (2016) Motor imagery of hand actions: Decoding the content of motor imagery from brain activity in frontal and parietal motor areas. Human Brain Mapping 37(1):8193. doi:10.1002/hbm.23015.CrossRefGoogle ScholarPubMed
Rees, G., Frith, C. D. & Lavie, N. (1997) Modulating irrelevant motion perception by varying attentional load in an unrelated task. Science 278(5343):1616–19.Google Scholar
Romei, V., de Haas, B., Mok, R. M. & Driver, J. (2011) Auditory stimulus timing influences perceived duration of co-occurring visual stimuli. Frontiers in Psychology 2:215. doi:10.3389/fpsyg.2011.00215.Google Scholar
Shams, L., Kamitani, Y. & Shimojo, S. (2000) Illusions: What you see is what you hear. Nature 408:788.Google Scholar
Suzuki, S. & Cavanagh, P. (1997) Focused attention distorts visual space: An attentional repulsion effect. Journal of Experimental Psychology: Human Perception and Performance 23(2):443–63.Google ScholarPubMed
Thomas, G. (1941) Experimental study of the influence of vision on sound localization. Journal of Experimental Psychology 28(2):163–77.CrossRefGoogle Scholar
Vetter, P., Smith, F. W. & Muckli, L. (2014) Decoding sound and imagery content in early visual cortex. Current Biology 24(11):1256–62. doi:10.1016/j.cub.2014.04.020.Google Scholar
Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D. & Rees, G. (2006) Sound alters activity in human V1 in association with illusory visual perception. NeuroImage 31(3):1247–56. doi:10.1016/j.neuroimage.2006.01.016.Google Scholar
Womelsdorf, T., Anton-Erxleben, K. & Treue, S. (2008) Receptive field shift and shrinkage in macaque middle temporal area through attentional gain modulation. The Journal of Neuroscience 28(36):8934–44. doi:10.1523/JNEUROSCI.4030-07.2008.CrossRefGoogle ScholarPubMed