Christiansen & Chater's (C&C's) theory builds upon the notion that linguistic structures and processes are specific responses to the limitations of sensory and memory systems. Language relies on three main strategies – incrementality, hierarchical representation, and prediction – for coping with those limitations. We think this list is incomplete, and that it should also include inference, the ability to read and write, pragmatic devices for coordinating speaker and hearer, and the mutual tuning of speech comprehension and production systems in the brain. We aim to show that this is more than merely adding items to a list: Our argument points to a different balance between restrictions on storage and computation, and the full range of cognitive functions that have a mitigating effect on them. Indeed, C&C's concise inventory satisfies all constraints, but no more: Language processing remains a race against time. We argue instead that the moderating factors widely offset the constraints, suggesting a different picture of language than the one envisaged by C&C.
Hearing is the main input channel for language. C&C discuss constraints on auditory analysis, but not the mechanisms by which the brain recovers lost information. Sensory systems rely heavily on perceptual inference. A classic example is phonemic restoration (PhR) (Warren Reference Warren1970): Deleting auditory segments from speech reduces comprehension, but if the deleted segment is replaced with noise, comprehension is restored. PhR is not the creation of an illusory percept but the reorganisation of input: Because PhR arises in auditory cortices, it requires that energy be present at the relevant frequencies (Petkov et al. Reference Petkov, O'Connor and Sutter2007). Short segments of unprocessed speech are not necessarily lost but may often be reconstructed. Probabilistic and logical inference to richer structures based on sparse data is available at all levels of representation in language (Graesser et al. Reference Graesser, Singer and Trabasso1994; Swinney & Osterhout Reference Swinney and Osterhout1990).
Vision is next in line in importance. In C&C's theory, vision has largely a supporting role: It may provide cues that disambiguate speech, but it is itself subject to constraints like auditory processing. However, the human brain can translate information across modalities, such that constraints that apply to one modality are weaker or absent in another. This applies to some innovations in recent human evolutionary history, including reading and writing. By the nature of texts as static visual objects, the effects of temporal constraints on information intake may be reduced or abolished. This is not to say there are no temporal constraints on reading: Processing the fine temporal structure of speech is crucial for reading acquisition (Dehaene Reference Dehaene2009; Goswami Reference Goswami2015). Written information, though, is often freely accessible in ways that auditory information is not. We acquire a portion of our vocabulary and grammar through written language, and we massively use text to communicate. Therefore, it seems that C&C's premise that “language is fleeting” applies to spoken language only, and not to language in general.
But even auditory information can often be freely re-accessed. Misperception and the loss of information in conversation pose coordination problems. These can be solved by deploying a number of pragmatic devices that allow speaker and hearer to realign: echo questions are one example (A: “I just returned from Kyrgyzstan.” B: “You just returned from where?”). Information is re-accessed by manipulating the source of the input, with (implicit) requests to slow down production, or to deliver a new token of the same type. Language use relies on a two-track system (Clark Reference Clark1996): (1) communication about “stuff” and (2) communication about communication. Track 2 allows us to recover from failure to process information on Track 1, and to focus attention on what matters from Track 1. Signals from both tracks are subject to bottlenecks and constraints; nonetheless, Track 2 alleviates the effects of restrictions on Track 1 processing. Interestingly, infants are able to engage in repair of failed messages (Golinkoff Reference Golinkoff1986). This capacity develops early in childhood, in parallel with language growth (Brinton et al. Reference Brinton, Fujiki, Loeb and Winkler1986a; Saxton et al. Reference Saxton, Houston-Price and Dawson2005; Tomasello et al. Reference Tomasello, Conti-Ramsden and Ewert1990; Yonata Reference Yonata1999).
Finally, C&C claim that different levels of linguistic representation are mapped onto a hierarchy of cortical circuits. Each circuit chunks and passes elements at increasingly larger timescales. But research indicates the picture is rather more complicated. Most brain regions can work at multiple timescales. Frontal and temporal language cortices can represent and manipulate information delivered at different rates, and over intervals of different duration (Fuster Reference Fuster1995; Pallier et al. Reference Pallier, Devauchelle and Dehaene2011; Ding et al. Reference Ding, Melloni, Zhang, Tian and Poeppel2016). Furthermore, the left parietal lobe is a critical region for both temporal processing (e.g., Vicario et al. Reference Vicario, Martino and Koch2013; see Wiener et al. Reference Wiener, Turkeltaub and Coslett2010 for a review) and amodal (spoken and written) word comprehension (Mesulam Reference Mesulam1998). The left inferior parietal cortex is a core area for speech comprehension and production because of its connections with wide portions of Wernicke's (superior temporal cortex [STC]) and Broca's (left inferior frontal gyrus) areas (Catani et al. Reference Catani, Jones and ffytche2005). The temporal cortex processes speech at different scales: at shorter windows (25–50 ms) in the left STC, and at longer windows (150–250 ms) in the right STC (Boemio et al. Reference Boemio, Fromm, Braun and Poeppel2005; Giraud et al. Reference Giraud, Kleinschmidt, Poeppel, Lund, Frackowiak and Laufs2007; Giraud & Poeppel Reference Giraud and Poeppel2012). This asymmetry might result from mutual tuning of primary auditory and motor cortices in the left hemisphere (Morillon et al. Reference Morillon, Lehongre, Frackowiak, Ducorps, Kleinschmidt, Poeppel and Giraud2010). If speech production and perception indeed share some of the constraints described by C&C, then neither system should be expected to lag behind the speed or the resolution of the other.
A more comprehensive theory of language processing would arise from taking into account constraints of the kind discussed by C&C, plus a wider array of cognitive mechanisms mitigating the effect of these constraints, including (but not limited to) Chunk-and-Pass processing and its corollaries. The human brain's ubiquitous capacity to infer, recover, and re-access unprocessed, lost, or degraded information is as much part of the “design” of the language system as incrementality, hierarchical representation, and prediction. The joint effect of these strategies is to make language processing much less prone to information loss and much less subject to time pressures than C&C seem to imply.
Christiansen & Chater's (C&C's) theory builds upon the notion that linguistic structures and processes are specific responses to the limitations of sensory and memory systems. Language relies on three main strategies – incrementality, hierarchical representation, and prediction – for coping with those limitations. We think this list is incomplete, and that it should also include inference, the ability to read and write, pragmatic devices for coordinating speaker and hearer, and the mutual tuning of speech comprehension and production systems in the brain. We aim to show that this is more than merely adding items to a list: Our argument points to a different balance between restrictions on storage and computation, and the full range of cognitive functions that have a mitigating effect on them. Indeed, C&C's concise inventory satisfies all constraints, but no more: Language processing remains a race against time. We argue instead that the moderating factors widely offset the constraints, suggesting a different picture of language than the one envisaged by C&C.
Hearing is the main input channel for language. C&C discuss constraints on auditory analysis, but not the mechanisms by which the brain recovers lost information. Sensory systems rely heavily on perceptual inference. A classic example is phonemic restoration (PhR) (Warren Reference Warren1970): Deleting auditory segments from speech reduces comprehension, but if the deleted segment is replaced with noise, comprehension is restored. PhR is not the creation of an illusory percept but the reorganisation of input: Because PhR arises in auditory cortices, it requires that energy be present at the relevant frequencies (Petkov et al. Reference Petkov, O'Connor and Sutter2007). Short segments of unprocessed speech are not necessarily lost but may often be reconstructed. Probabilistic and logical inference to richer structures based on sparse data is available at all levels of representation in language (Graesser et al. Reference Graesser, Singer and Trabasso1994; Swinney & Osterhout Reference Swinney and Osterhout1990).
Vision is next in line in importance. In C&C's theory, vision has largely a supporting role: It may provide cues that disambiguate speech, but it is itself subject to constraints like auditory processing. However, the human brain can translate information across modalities, such that constraints that apply to one modality are weaker or absent in another. This applies to some innovations in recent human evolutionary history, including reading and writing. By the nature of texts as static visual objects, the effects of temporal constraints on information intake may be reduced or abolished. This is not to say there are no temporal constraints on reading: Processing the fine temporal structure of speech is crucial for reading acquisition (Dehaene Reference Dehaene2009; Goswami Reference Goswami2015). Written information, though, is often freely accessible in ways that auditory information is not. We acquire a portion of our vocabulary and grammar through written language, and we massively use text to communicate. Therefore, it seems that C&C's premise that “language is fleeting” applies to spoken language only, and not to language in general.
But even auditory information can often be freely re-accessed. Misperception and the loss of information in conversation pose coordination problems. These can be solved by deploying a number of pragmatic devices that allow speaker and hearer to realign: echo questions are one example (A: “I just returned from Kyrgyzstan.” B: “You just returned from where?”). Information is re-accessed by manipulating the source of the input, with (implicit) requests to slow down production, or to deliver a new token of the same type. Language use relies on a two-track system (Clark Reference Clark1996): (1) communication about “stuff” and (2) communication about communication. Track 2 allows us to recover from failure to process information on Track 1, and to focus attention on what matters from Track 1. Signals from both tracks are subject to bottlenecks and constraints; nonetheless, Track 2 alleviates the effects of restrictions on Track 1 processing. Interestingly, infants are able to engage in repair of failed messages (Golinkoff Reference Golinkoff1986). This capacity develops early in childhood, in parallel with language growth (Brinton et al. Reference Brinton, Fujiki, Loeb and Winkler1986a; Saxton et al. Reference Saxton, Houston-Price and Dawson2005; Tomasello et al. Reference Tomasello, Conti-Ramsden and Ewert1990; Yonata Reference Yonata1999).
Finally, C&C claim that different levels of linguistic representation are mapped onto a hierarchy of cortical circuits. Each circuit chunks and passes elements at increasingly larger timescales. But research indicates the picture is rather more complicated. Most brain regions can work at multiple timescales. Frontal and temporal language cortices can represent and manipulate information delivered at different rates, and over intervals of different duration (Fuster Reference Fuster1995; Pallier et al. Reference Pallier, Devauchelle and Dehaene2011; Ding et al. Reference Ding, Melloni, Zhang, Tian and Poeppel2016). Furthermore, the left parietal lobe is a critical region for both temporal processing (e.g., Vicario et al. Reference Vicario, Martino and Koch2013; see Wiener et al. Reference Wiener, Turkeltaub and Coslett2010 for a review) and amodal (spoken and written) word comprehension (Mesulam Reference Mesulam1998). The left inferior parietal cortex is a core area for speech comprehension and production because of its connections with wide portions of Wernicke's (superior temporal cortex [STC]) and Broca's (left inferior frontal gyrus) areas (Catani et al. Reference Catani, Jones and ffytche2005). The temporal cortex processes speech at different scales: at shorter windows (25–50 ms) in the left STC, and at longer windows (150–250 ms) in the right STC (Boemio et al. Reference Boemio, Fromm, Braun and Poeppel2005; Giraud et al. Reference Giraud, Kleinschmidt, Poeppel, Lund, Frackowiak and Laufs2007; Giraud & Poeppel Reference Giraud and Poeppel2012). This asymmetry might result from mutual tuning of primary auditory and motor cortices in the left hemisphere (Morillon et al. Reference Morillon, Lehongre, Frackowiak, Ducorps, Kleinschmidt, Poeppel and Giraud2010). If speech production and perception indeed share some of the constraints described by C&C, then neither system should be expected to lag behind the speed or the resolution of the other.
A more comprehensive theory of language processing would arise from taking into account constraints of the kind discussed by C&C, plus a wider array of cognitive mechanisms mitigating the effect of these constraints, including (but not limited to) Chunk-and-Pass processing and its corollaries. The human brain's ubiquitous capacity to infer, recover, and re-access unprocessed, lost, or degraded information is as much part of the “design” of the language system as incrementality, hierarchical representation, and prediction. The joint effect of these strategies is to make language processing much less prone to information loss and much less subject to time pressures than C&C seem to imply.