We live in a multisensory world that appeals to all of our sensory modalities (Stein et al. Reference Stein, Huneycutt and Meredith1988). Crossmodal perception is the norm, rather than the exception; the study of the senses as independent modules rather than integrated systems might give rise to misunderstandings of how the mind allows us to sense, perceive, and cognitively process the world around us (Ghazanfar & Schroeder Reference Ghazanfar and Schroeder2006). Although some connections between the senses appear to be direct and low-level (Meredith & Stein Reference Meredith and Stein1986), others certainly are the product of learning and experience (Proulx et al. Reference Proulx, Brown, Pasqualotto and Meijer2014). The authors of the target article provide an excellent framework and review for examining the (lack of) influence of cognition on perception, yet we think it would have been even stronger had they not dismissed crossmodal perception and cognition.
One area of crossmodal research that certainly connects perception and cognition is the study of sensory substitution and augmentation (an area of research that is cited in the fantastic textbook Sensation and Perception [Yantis Reference Yantis2013], unlike the top-down studies Firestone & Scholl [F&S] criticize). Sensory substitution devices allow someone with a damaged sensory modality (such as the visual system for the visually impaired) to receive the missing information by transforming it into a format that another intact sensory modality (such as the auditory [Meijer Reference Meijer1992] or somatosensory system [Bach-y-Rita & Kercel Reference Bach-y-Rita and Kercel2003]) can process. Learning plays a clear role in using these devices to access the otherwise inaccessible information. But is the ability to “see” with such a system better classified as perception or cognition? Although some have argued that it must be classified as seeing in the perceptual sense for psychology to be a science (Morgan Reference Morgan1977), we accept that this might be open to some debate.
Considering the distinction between perception and judgment in the target article, amongst other key issues, it seems that the authors might indeed debate whether sensory substitution allows for perception. Furthermore, there is evidence that long-term users of such devices who once had vision (who had acquired blindness) have visual imagery that is evoked immediately by the sounds they hear with a device, and therefore express that they have the perception of sight (Ward & Meijer Reference Ward and Meijer2010). Such cases might not be classified as an immediate crossmodal effect, such as those noted in the target article, so what might account for this ability instead? Any kind of information the sensory organs receive is meaningless because there is no inherent meaning in sensory information without experience and knowledge (Proulx Reference Proulx2011), and F&S acknowledge this point to some extent by allowing for unconscious inference to play a role in perception (von Helmholtz Reference von Helmholtz2005). But if unconscious inference plays a role in perception, as it certainly does even in the perception of a red object, then there must be some role of recognition and judgment even for low-level perception. The sharing of neural resources for the processing of perceived and imagined information (Klein et al. Reference Klein, Dubois, Mangin, Kherif, Flandin, Poline and Le Bihan2004; Kosslyn et al. Reference Kosslyn, Pascual-Leone, Felician, Camposano, Keenan, Ganis and Alpert1999) is also suggestive of an interplay between perception and cognition, if not perhaps the idea that there is continuity between such steps of information processing, rather than categorical differences between them.
F&S provide an excellent checklist to carefully assess perception versus cognition that is useful regardless of the continual or categorical nature of these phenomena. For example, their discussion of a possible distinction between perception and judgment in the context of the El Greco fallacy provides a useful approach to assess how users are able to learn sensory substitution. The relative reliance on perception, judgment, response bias, memory, and recognition could be assayed at different points in training to provide a full profile of how the information is being transferred from one sense to another, and thus reveal the mechanisms of seeing with sound or touch. Novel methods such as sensory substitution, and related areas of crossmodal cognition including synesthesia, might provide crucial ways of examining perception and cognition in a new light (or sound).
We live in a multisensory world that appeals to all of our sensory modalities (Stein et al. Reference Stein, Huneycutt and Meredith1988). Crossmodal perception is the norm, rather than the exception; the study of the senses as independent modules rather than integrated systems might give rise to misunderstandings of how the mind allows us to sense, perceive, and cognitively process the world around us (Ghazanfar & Schroeder Reference Ghazanfar and Schroeder2006). Although some connections between the senses appear to be direct and low-level (Meredith & Stein Reference Meredith and Stein1986), others certainly are the product of learning and experience (Proulx et al. Reference Proulx, Brown, Pasqualotto and Meijer2014). The authors of the target article provide an excellent framework and review for examining the (lack of) influence of cognition on perception, yet we think it would have been even stronger had they not dismissed crossmodal perception and cognition.
One area of crossmodal research that certainly connects perception and cognition is the study of sensory substitution and augmentation (an area of research that is cited in the fantastic textbook Sensation and Perception [Yantis Reference Yantis2013], unlike the top-down studies Firestone & Scholl [F&S] criticize). Sensory substitution devices allow someone with a damaged sensory modality (such as the visual system for the visually impaired) to receive the missing information by transforming it into a format that another intact sensory modality (such as the auditory [Meijer Reference Meijer1992] or somatosensory system [Bach-y-Rita & Kercel Reference Bach-y-Rita and Kercel2003]) can process. Learning plays a clear role in using these devices to access the otherwise inaccessible information. But is the ability to “see” with such a system better classified as perception or cognition? Although some have argued that it must be classified as seeing in the perceptual sense for psychology to be a science (Morgan Reference Morgan1977), we accept that this might be open to some debate.
Considering the distinction between perception and judgment in the target article, amongst other key issues, it seems that the authors might indeed debate whether sensory substitution allows for perception. Furthermore, there is evidence that long-term users of such devices who once had vision (who had acquired blindness) have visual imagery that is evoked immediately by the sounds they hear with a device, and therefore express that they have the perception of sight (Ward & Meijer Reference Ward and Meijer2010). Such cases might not be classified as an immediate crossmodal effect, such as those noted in the target article, so what might account for this ability instead? Any kind of information the sensory organs receive is meaningless because there is no inherent meaning in sensory information without experience and knowledge (Proulx Reference Proulx2011), and F&S acknowledge this point to some extent by allowing for unconscious inference to play a role in perception (von Helmholtz Reference von Helmholtz2005). But if unconscious inference plays a role in perception, as it certainly does even in the perception of a red object, then there must be some role of recognition and judgment even for low-level perception. The sharing of neural resources for the processing of perceived and imagined information (Klein et al. Reference Klein, Dubois, Mangin, Kherif, Flandin, Poline and Le Bihan2004; Kosslyn et al. Reference Kosslyn, Pascual-Leone, Felician, Camposano, Keenan, Ganis and Alpert1999) is also suggestive of an interplay between perception and cognition, if not perhaps the idea that there is continuity between such steps of information processing, rather than categorical differences between them.
F&S provide an excellent checklist to carefully assess perception versus cognition that is useful regardless of the continual or categorical nature of these phenomena. For example, their discussion of a possible distinction between perception and judgment in the context of the El Greco fallacy provides a useful approach to assess how users are able to learn sensory substitution. The relative reliance on perception, judgment, response bias, memory, and recognition could be assayed at different points in training to provide a full profile of how the information is being transferred from one sense to another, and thus reveal the mechanisms of seeing with sound or touch. Novel methods such as sensory substitution, and related areas of crossmodal cognition including synesthesia, might provide crucial ways of examining perception and cognition in a new light (or sound).