By questioning the existence of the neural code, Romain Brette opens again a strong debate between representational views of the brain (cognitivism and computationalism) and sensorimotor/enaction theories (O'Regan and Noë Reference O'Regan and Noë2001; Varela et al. Reference Varela, Thompson and Rosch1991), his preference being the latter. According to his view, all cognitive functions, particularly action and perception, are viewed as means to interact with the world, without the need to build internal representations of it. Neural activity during perception should be viewed as the result of the organism's interaction with the world, taking into account all possible influences, such as its internal state and its actions resulting in a given percept. Therefore, as the brain does not manipulate representations, it is senseless to try to decipher any code supposed to encrypt representations in neural activity. The results of three research fields focusing on proving that a particular neural code is at play should be addressed by Brette's review to strengthen his point.
First, in sensory physiology, research on tuning curves has been extended to naturalistic stimuli and is divided into two complementary approaches: encoding and decoding. Based on models of the stimulus-response function, these approaches rely on the idea that neural activity encodes some features of the external world. Successful reconstructions of complex stimuli based on neural responses (decoding), or successful predictions of responses to new stimuli (encoding) are viewed as proofs that the neural code has been cracked. Interpreting these results in the light of Brette's arguments seems necessary. Initially, the stimulus reconstruction method (decoding) was performed either with simple artificial stimuli (Bialek et al. Reference Bialek, Rieke, de Ruyter van Steveninck and Warland1991) or in peripheral sensory systems (Rieke et al. Reference Rieke, Bodnar and Bialek1995; Warland et al. Reference Warland, Reinagel and Meister1997). More recently, studies have reconstructed natural stimuli from cortical responses (Akbari et al. Reference Akbari, Khalighinejad, Herrero, Mehta and Mesgarani2019; Miyawaki et al. Reference Miyawaki, Uchida, Yamashita, Sato, Morito, Tanabe, Sadato and Kamitani2008; Naselaris et al. Reference Naselaris, Prenger, Kay, Oliver and Gallant2009), opening the spectacular expectation to read subjects’ percepts. In the auditory modality, encoding models were used to investigate neural selectivity to a variety of acoustic properties such as phonetic features (Mesgarani et al. Reference Mesgarani, Cheung, Johnson and Chang2014), pitch (Oxenham Reference Oxenham2018), and timbre and rhythm (Woolley et al. Reference Woolley, Gill, Fremouw and Theunissen2009). To achieve good performance, the stimulus/response models used in decoding/encoding approaches rely on features such as trial averaging, statistics of natural stimuli, and starting time of the stimulus. Thus, the right interpretation should be that an “ideal observer” with a priori knowledge of the experimental design can infer the stimulus (in the decoding approach) or the neural response (in the encoding approach). Noteworthy, this field has led to an interesting drift from the idea of a fixed relationship between stimulus and neural responses to a more dynamic model, and is now tackling the mechanisms by which sensory responses are modulated by learning, context, and history (Fritz et al. Reference Fritz, Elhilali and Shamma2005; Holdgraf et al. Reference Holdgraf, de Heer, Pasley, Rieger, Crone, Lin, Knight and Theunissen2016; Williamson et al. Reference Williamson, Ahrens, Linden and Sahani2016).
Second, the field of neuroprosthetic devices offers demonstrations of causal links between neural code and brain functions. The most successful of these devices, the cochlear implant (CI), operates with blunt stimulations of auditory nerve terminals. Despite a large current spread in the tympanic ramp, the CI allows implanted subjects to have percepts and recover speech understanding. Even though there are huge differences between the normal cochlea and the CI, the fact that CIs restore hearing can be viewed as a proof that the neural code at play in the periphery has been deciphered and is successfully implemented in a prosthetic device. However, the CI settings that lead to speech comprehension differ considerably from one subject to another, as do the strategies leading to the largest evoked responses in auditory cortex (Adenis et al. Reference Adenis, Gourévitch, Mamelle, Recugnat, Stahl, Gnansia, Nguyen and Edeline2018). Thus, in contrast to the genetic code that is invariant across cells and species, the neural code (understood as changes in neural activity in adaption to a CI) is probably specific for each individual and/or each type of neuron. In line with sensorimotor theories, the success of CIs shows that the brain is using a new input in a way it can interact again with the environment, which might be the basis of hearing restoration.
A third important field investigates the effect of disrupting a particular feature of neural activity on a cognitive skill. In the visual system, disruption of physiological activity in the primate middle temporal area during presentation of moving stimuli biases the perceptive judgment of a behaving animal (Salzman & Newsome Reference Salzman and Newsome1994; Salzman et al. Reference Salzman, Britten and Newsome1990), thus making the first link between neural code (understood as a pattern of activity of specific neurons) and behavioral performance. More recently, studies performed in the hippocampus have found that disrupting the replay of spiking patterns occurring across neuronal ensembles during the sharp wave ripples profoundly alters the memory of previously acquired information (Ego-Stengel & Wilson Reference Ego-Stengel and Wilson2010; Girardeau et al. Reference Girardeau, Benchenane, Wiener, Buzsáki and Zugaro2009). These data reinforce the notion that neuronal activity patterns do correlate with the acquired information. More importantly, associating a rewarding stimulation of the medial forebrain bundle with a hippocampal place cell activity induced a place preference at the place cell location (de Lavilléon et al. Reference de Lavilléon, Lacroix, Rondi-Reig and Benchenane2015), demonstrating causal links between a particular place cell's firing rate and a specific location memory. In all these examples, the exact neural activity feature (its firing rate or its temporal spike patterns) correlated with the animal's location is unknown, but causal relationships do exist. Yet, causality is not enough to define a neural code.
Clearly, more caution is necessary when discussing the neural code as overstatements made (Ferster & Spruston Reference Ferster and Spruston1995; Panzeri et al. Reference Panzeri, Harvey, Piasini, Latham and Fellin2017) tend to generate the illusions that (1) the same code operates in any sensory and motor system, which is obviously not the case; and (2) the brain's cognitive functions consist of manipulating encoded representations of the world, a theory that is controversial. Does this mean that the concept of neural code should be abandoned or should be used to describe studies linking neural activity to brain function? We believe that the neural code definition should be freed from the notion of representation, and we should clarify what we refer to when investigating the neural mechanism of brain functions.
You have
Access
By questioning the existence of the neural code, Romain Brette opens again a strong debate between representational views of the brain (cognitivism and computationalism) and sensorimotor/enaction theories (O'Regan and Noë Reference O'Regan and Noë2001; Varela et al. Reference Varela, Thompson and Rosch1991), his preference being the latter. According to his view, all cognitive functions, particularly action and perception, are viewed as means to interact with the world, without the need to build internal representations of it. Neural activity during perception should be viewed as the result of the organism's interaction with the world, taking into account all possible influences, such as its internal state and its actions resulting in a given percept. Therefore, as the brain does not manipulate representations, it is senseless to try to decipher any code supposed to encrypt representations in neural activity. The results of three research fields focusing on proving that a particular neural code is at play should be addressed by Brette's review to strengthen his point.
First, in sensory physiology, research on tuning curves has been extended to naturalistic stimuli and is divided into two complementary approaches: encoding and decoding. Based on models of the stimulus-response function, these approaches rely on the idea that neural activity encodes some features of the external world. Successful reconstructions of complex stimuli based on neural responses (decoding), or successful predictions of responses to new stimuli (encoding) are viewed as proofs that the neural code has been cracked. Interpreting these results in the light of Brette's arguments seems necessary. Initially, the stimulus reconstruction method (decoding) was performed either with simple artificial stimuli (Bialek et al. Reference Bialek, Rieke, de Ruyter van Steveninck and Warland1991) or in peripheral sensory systems (Rieke et al. Reference Rieke, Bodnar and Bialek1995; Warland et al. Reference Warland, Reinagel and Meister1997). More recently, studies have reconstructed natural stimuli from cortical responses (Akbari et al. Reference Akbari, Khalighinejad, Herrero, Mehta and Mesgarani2019; Miyawaki et al. Reference Miyawaki, Uchida, Yamashita, Sato, Morito, Tanabe, Sadato and Kamitani2008; Naselaris et al. Reference Naselaris, Prenger, Kay, Oliver and Gallant2009), opening the spectacular expectation to read subjects’ percepts. In the auditory modality, encoding models were used to investigate neural selectivity to a variety of acoustic properties such as phonetic features (Mesgarani et al. Reference Mesgarani, Cheung, Johnson and Chang2014), pitch (Oxenham Reference Oxenham2018), and timbre and rhythm (Woolley et al. Reference Woolley, Gill, Fremouw and Theunissen2009). To achieve good performance, the stimulus/response models used in decoding/encoding approaches rely on features such as trial averaging, statistics of natural stimuli, and starting time of the stimulus. Thus, the right interpretation should be that an “ideal observer” with a priori knowledge of the experimental design can infer the stimulus (in the decoding approach) or the neural response (in the encoding approach). Noteworthy, this field has led to an interesting drift from the idea of a fixed relationship between stimulus and neural responses to a more dynamic model, and is now tackling the mechanisms by which sensory responses are modulated by learning, context, and history (Fritz et al. Reference Fritz, Elhilali and Shamma2005; Holdgraf et al. Reference Holdgraf, de Heer, Pasley, Rieger, Crone, Lin, Knight and Theunissen2016; Williamson et al. Reference Williamson, Ahrens, Linden and Sahani2016).
Second, the field of neuroprosthetic devices offers demonstrations of causal links between neural code and brain functions. The most successful of these devices, the cochlear implant (CI), operates with blunt stimulations of auditory nerve terminals. Despite a large current spread in the tympanic ramp, the CI allows implanted subjects to have percepts and recover speech understanding. Even though there are huge differences between the normal cochlea and the CI, the fact that CIs restore hearing can be viewed as a proof that the neural code at play in the periphery has been deciphered and is successfully implemented in a prosthetic device. However, the CI settings that lead to speech comprehension differ considerably from one subject to another, as do the strategies leading to the largest evoked responses in auditory cortex (Adenis et al. Reference Adenis, Gourévitch, Mamelle, Recugnat, Stahl, Gnansia, Nguyen and Edeline2018). Thus, in contrast to the genetic code that is invariant across cells and species, the neural code (understood as changes in neural activity in adaption to a CI) is probably specific for each individual and/or each type of neuron. In line with sensorimotor theories, the success of CIs shows that the brain is using a new input in a way it can interact again with the environment, which might be the basis of hearing restoration.
A third important field investigates the effect of disrupting a particular feature of neural activity on a cognitive skill. In the visual system, disruption of physiological activity in the primate middle temporal area during presentation of moving stimuli biases the perceptive judgment of a behaving animal (Salzman & Newsome Reference Salzman and Newsome1994; Salzman et al. Reference Salzman, Britten and Newsome1990), thus making the first link between neural code (understood as a pattern of activity of specific neurons) and behavioral performance. More recently, studies performed in the hippocampus have found that disrupting the replay of spiking patterns occurring across neuronal ensembles during the sharp wave ripples profoundly alters the memory of previously acquired information (Ego-Stengel & Wilson Reference Ego-Stengel and Wilson2010; Girardeau et al. Reference Girardeau, Benchenane, Wiener, Buzsáki and Zugaro2009). These data reinforce the notion that neuronal activity patterns do correlate with the acquired information. More importantly, associating a rewarding stimulation of the medial forebrain bundle with a hippocampal place cell activity induced a place preference at the place cell location (de Lavilléon et al. Reference de Lavilléon, Lacroix, Rondi-Reig and Benchenane2015), demonstrating causal links between a particular place cell's firing rate and a specific location memory. In all these examples, the exact neural activity feature (its firing rate or its temporal spike patterns) correlated with the animal's location is unknown, but causal relationships do exist. Yet, causality is not enough to define a neural code.
Clearly, more caution is necessary when discussing the neural code as overstatements made (Ferster & Spruston Reference Ferster and Spruston1995; Panzeri et al. Reference Panzeri, Harvey, Piasini, Latham and Fellin2017) tend to generate the illusions that (1) the same code operates in any sensory and motor system, which is obviously not the case; and (2) the brain's cognitive functions consist of manipulating encoded representations of the world, a theory that is controversial. Does this mean that the concept of neural code should be abandoned or should be used to describe studies linking neural activity to brain function? We believe that the neural code definition should be freed from the notion of representation, and we should clarify what we refer to when investigating the neural mechanism of brain functions.