3.1. Implications of Strategy 1: Incremental processing

Chunk-and-Pass processing has important implications for comprehension and production: It requires that both take place incrementally. In incremental processing, representations are built up as rapidly as possible as the input is encountered. By contrast, one might, for example, imagine a parser that waits until the end of a sentence before beginning syntactic analysis, or that meaning is computed only once syntax has been established. However, such processing would require storing a stream of information at a single level of representation, and processing it later; but given the Now-or-Never bottleneck, this is not possible because of severe interference between such representations. Therefore, incremental interpretation and production follow directly from the Now-or-Never constraint on language.

To get a sense of the implications of Chunk-and-Pass processing, it is interesting to relate this perspective to specific computational principles and models. How, for example, do classic models of parsing fit within this framework? A wide range of psychologically inspired models involves some degree of incrementality of syntactic analysis, which can potentially support incremental interpretation (e.g., Phillips Reference Phillips, Camacho, Choueiri and Watanabe1996; Reference Phillips2003; Winograd Reference Winograd1972). For example, the sausage machine parsing model (Frazier & Fodor Reference Frazier and Fodor1978) proposes that a preliminary syntactic analysis is carried out phrase-by-phrase, but in complete isolation from semantic or pragmatic factors. But for a right-branching language such as English, chunks cannot be built left-to-right, because the leftmost chunks are incomplete until later material has been encountered. Frameworks from Kimball (Reference Kimball1973) onward imply “stacking up” incomplete constituents that may then all be resolved at the end of the clause. This approach runs counter to the memory constraints imposed by the Now-or-Never bottleneck. Reconciling right-branching with incremental chunking and processing is one motivation for the flexible constituency of combinatory categorial grammar (e.g., Steedman Reference Steedman1987; Reference Steedman2000; see also Johnson-Laird Reference Johnson-Laird1983).

With respect to comprehension, considerable evidence supports incremental interpretation, going back more than four decades (e.g., Bever Reference Bever and Hayes1970; Marslen-Wilson Reference Marslen-Wilson1975). The language system uses all available information to rapidly integrate incoming information as quickly as possible to update the current interpretation of what has been said so far. This process includes not only sentence-internal information about lexical and structural biases (e.g., Farmer et al. Reference Farmer, Christiansen and Monaghan2006; MacDonald Reference MacDonald1994; Trueswell et al. Reference Trueswell, Tanenhaus and Kello1993), but also extra-sentential cues from the referential and pragmatic context (e.g., Altmann & Steedman Reference Altmann and Steedman1988; Thornton et al. Reference Thornton, MacDonald and Gil1999) as well as the visual environment and world knowledge (e.g., Altmann & Kamide Reference Altmann and Kamide1999; Tanenhaus et al. Reference Tanenhaus, Spivey-Knowlton, Eberhard and Sedivy1995). As the incoming acoustic information is chunked, it is rapidly integrated with contextual information to recognize words, consistent with a variety of data on spoken word recognition (e.g., Marslen-Wilson Reference Marslen-Wilson1975; van den Brink et al. Reference van den Brink, Brown and Hagoort2001). These words are then, in turn, chunked into larger multiword units, as evidenced by recent studies showing sensitivity to multiword sequences in online processing (e.g., Arnon & Snider Reference Arnon and Snider2010; Reali & Christiansen Reference Reali and Christiansen2007b; Siyanova-Chanturia et al. Reference Siyanova-Chanturia, Conklin and Van Heuven2011; Tremblay & Baayen Reference Tremblay, Baayen and Wood2010; Tremblay et al. Reference Tremblay, Derwing, Libben and Westbury2011), and subsequently further integrated with pragmatic context into discourse-level structures.

Turning to production, we start by noting the powerful intuition that we speak “into the void” – that is, that we plan only a short distance ahead. Indeed, experimental studies suggest that, for example, when producing an utterance involving several noun phrases, people plan just one (Smith & Wheeldon Reference Smith and Wheeldon1999), or perhaps two, noun phrases ahead (Konopka Reference Konopka2012), and they can modify a message during production in the light of new perceptual input (Brown-Schmidt & Konopka Reference Brown-Schmidt and Konopka2015). Moreover, speech-error data (e.g., Cutler Reference Cutler1982) reveal that, across representational levels, errors tend to be highly local: Phonological, morphemic, and syntactic errors apply to neighboring chunks within each level (where material may be moved, swapped, or deleted). Consequently, speech planning appears to involve just a small number of chunks – the number of which may be similar across linguistic levels – but which covers different amounts of time depending on the linguistic level in question. For example, planning involving chunks at the level of intonational bursts stretches over considerably longer periods of time than planning at the syllabic level. Similarly, processes of reduction to facilitate production (e.g., modifying the speech signal to make it easier to produce, such as reducing a vowel to a schwa, or shortening or eliminating phonemes) can be observed across different levels of linguistic representation, from individual words (e.g., Gahl & Garnsey Reference Gahl and Garnsey2004; Jurafsky et al. Reference Jurafsky, Bell, Gregory, Raymond, Bybee and Hopper2001) to frequent multiword sequences (e.g., Arnon & Cohen Priva Reference Arnon and Cohen Priva2013; Bybee & Schiebman Reference Bybee and Scheibman1999).

Some may object that the Chunk-and-Pass perspective's strict notion of incremental interpretation and production leaves the language system vulnerable to the rather substantial ambiguity that exists across many levels of linguistic representation (e.g., lexical, syntactic, pragmatic). So-called garden path sentences such as the famous “The horse raced past the barn fell” (Bever Reference Bever and Hayes1970) show that people are vulnerable to at least some local ambiguities: They invite comprehenders to take the wrong interpretive path by treating raced as the main verb, which leads them to a dead end. Only when the final word, fell, is encountered does it become clear that something is wrong: raced should be interpreted as a past participle that begins a reduced relative clause (i.e., the horse [that was] raced past the barn fell). The difficulty of recovery in such garden path sentences indicates how strongly the language system is geared toward incremental interpretation.

Viewed as a processing problem, garden paths occur when the language system resolves an ambiguity incorrectly. But in many cases, it is possible for an underspecified representation to be constructed online, and for the ambiguity to be resolved later when further linguistic input arrives. This type of case is consistent with Marr's (Reference Marr1976) proposal of the “principle of least commitment,” that the perceptual system resolves ambiguous perceptual input only when it has sufficient data to make it unlikely that such decisions will subsequently have to be reversed. Given the ubiquity of local ambiguity in language, such underspecification may be used very widely in language processing. Note, however, that because of the severe constraints the Now-or-Never bottleneck imposes, the language system cannot adopt broad parallelism to further minimize the effect of ambiguity (as in many current probabilistic theories of parsing, e.g., Hale Reference Hale2006; Jurafsky Reference Jurafsky1996; Levy Reference Levy2008). Rather, within the Chunk-and-Pass account, the sole role for parallelism in the processing system is in deciding how the input should be chunked; only when conflicts concerning chunking are resolved can the input be passed on to a higher-level representation. In particular, we suggest that competing higher-level codes cannot be activated in parallel. This picture is analogous to Marr's principle of least commitment of vision: Although there might be temporary parallelism to resolve conflicts about, say, correspondence between dots in a random-dot stereogram, it is not possible to create two conflicting three-dimensional surfaces in parallel, and whereas there may be parallelism over the interpretation of lines and dots in an image, it is not possible to see something as both a duck and a rabbit simultaneously. More broadly, higher-level representations are constructed only when sufficient evidence has accrued that they are unlikely later to need to be replaced (for stimuli outside the psychological laboratory, at least).

Maintaining, and later resolving, an underspecified representation will create local memory and processing demands that may slow down processing, as is observed, for example, by increased reading times (e.g., Trueswell et al. Reference Trueswell, Tanenhaus and Garnsey1994) and distinctive patterns of brain activity (as measured by ERPs; Swaab et al. Reference Swaab, Brown and Hagoort2003). Accordingly, when the input is ambiguous, the language system may require later input to recognize previous elements of the speech stream successfully. The Now-or-Never bottleneck requires that such online “right-context effects” be highly local because raw perceptual input will be lost if it is not rapidly identified (e.g., Dahan Reference Dahan2010). Right-context effects may arise where the language system can delay resolution of ambiguity or use underspecified representations that do not require resolving the ambiguity right away. Similarly, cataphora, in which, for example, a referential pronoun occurs before its referent (e.g., “He is a nice guy, that John”) require the creation of an underspecified entity (male, animate) when he is encountered, which is resolved to be coreferential with John only later in the sentence (e.g., van Gompel & Liversedge Reference van Gompel and Liversedge2003). Overall, the Now-or-Never bottleneck implies that the processing system will build the most abstract and complete representation that is justified, given the linguistic input.Footnote ⁶

Of course, outside of experimental studies, background knowledge, visual context, and prior discourse will provide powerful cues to help resolve ambiguities in the signal, allowing the system rapidly to resolve many apparent ambiguities without incurring a substantial danger of “garden-pathing.” Indeed, although syntactic and lexical ambiguities have been much studied in psycholinguistics, increasing evidence indicates that garden paths are not a major source of processing difficulty in practice (e.g., Ferreira Reference Ferreira2008; Jaeger Reference Jaeger2010; Wasow & Arnold Reference Wasow, Arnold, Rohdenburg and Mondorf2003).Footnote ⁷ For example, Roland et al. (Reference Roland, Elman and Ferreira2006) reported corpus analyses showing that, in naturally occurring language, there is generally sufficient information in the sentential context before the occurrence of an ambiguous verb to specify the correct interpretation of that verb. Moreover, eye-tracking studies have demonstrated that dialogue partners exploit both conversational context and task demands to constrain interpretations to the appropriate referents, thereby side-stepping effects of phonological and referential competitors (Brown-Schmidt & Konopka Reference Brown-Schmidt and Konopka2011) that have otherwise been shown to impede language processing (e.g., Allopenna et al. Reference Allopenna, Magnuson and Tanenhaus1998). These dialogue-based constraints also mitigate syntactic ambiguities that might otherwise disrupt processing (Brown-Schmidt & Tanenhaus Reference Brown-Schmidt and Tanenhaus2008). This information may be further combined with other probabilistic sources of information such as prosody (e.g., Kraljic & Brennan Reference Kraljic and Brennan2005; Snedeker & Trueswell Reference Snedeker and Trueswell2003) to resolve potential ambiguities within a minimal temporal window. Finally, it is not clear that undetected garden path errors are costly in normal language use, because if communication appears to break down, the listener can repair the communication by requesting clarification from the dialogue partner.

3.2. Implications of Strategy 2: Multiple levels of linguistic structure

The Now-or-Never bottleneck forces the language system to compress input into increasingly abstract chunks that cover progressively longer temporal intervals. As an example, consider the chunking of the input illustrated in Figure 2. The acoustic signal is first chunked into higher-level sound units at the phonological level. To avoid interference between local sound-based units, such as phonemes or syllables, these units are further recoded as rapidly as possible into higher-level units such as morphemes or words. The same phenomenon occurs at the next level up: Local groups of words must be chunked into larger units, possibly phrases or other forms of multiword sequences. Subsequent chunking then recodes these representations into higher-level discourse structures (that may themselves be chunked further into even more abstract representational structures beyond that). Similarly, production requires running the process in reverse, starting with the intended message and gradually decoding it into increasingly more specific chunks, eventually resulting in the motor programs necessary for producing the relevant speech or sign output. As we discuss in section 3.3, the production process may further serve as the basis for prediction during comprehension (allowing higher-level information to influence the processing of current input). More generally, our account is agnostic with respect to the specific characterization of the various levels of linguistic representationFootnote ⁸ (e.g., whether sound-based chunks take the form of phonemes, syllables, etc.). What is central for the Chunk-and-Pass account: some form of sound-based level of chunking (or visual-based in the case of sign language), and a sequence of increasingly abstract levels of chunked representations into which the input is continually recoded.

Figure 2. Chunk-and-Pass processing across a variety of linguistic levels in spoken language. As input is chunked and passed up to increasingly abstract levels of linguistic representations in comprehension, from acoustics to discourse, the temporal window over which information can be maintained increases, as indicated by the shaded portion of the bars associated with each linguistic level. This process is reversed in production planning, in which chunks are broken down into sequences of increasingly short and concrete units, from a discourse-level message to the motor commands for producing a specific articulatory output. More-abstract representations correspond to longer chunks of linguistic material, with greater look-ahead in production at higher levels of abstraction. Production processes may further serve as the basis for predictions to facilitate comprehension and thus provide topdown information in comprehension. (Note that the names and number of levels are for illustrative purposes only.)

A key theoretical implication of Chunk-and-Pass processing is that the multiple levels of linguistic representation, typically assumed in the language sciences, are a necessary by-product of the Now-or-Never bottleneck. Only by compressing the input into chunks and passing them to increasingly abstract levels of linguistic representation can the language system deal with the rapid onslaught of incoming information. Crucially, though, our perspective also suggests that the different levels of linguistic representations do not have a true part–whole relationship with one another. Unlike in the case of SF, who learned strategies to perfectly unpack chunks from within chunks to reproduce the original string of digits, language comprehension typically employs lossy compression to chunk the input. That is, higher-level chunks will not in general contain complete copies of lower-level chunks. Indeed, as speech input is encoded into ever more abstract chunks, increasing amounts of low-level information will typically be lost. Instead, as in perception (e.g., Haber Reference Haber1983), there is greater representational underspecification with higher levels of representation because of the repeated process of lossy compression.Footnote ⁹ Thus, we would expect a growing involvement of extralinguistic information, such as perceptual input and world knowledge, in processing higher levels of linguistic representation (see, e.g., Altmann & Kamide Reference Altmann and Kamide2009).

Whereas our account proposes a lossy hierarchy across levels of linguistic representation, only a very small number of chunks are represented within a level: otherwise, information is rapidly lost due to interference. This has the crucial implication that chunks within a given level can interact only locally. For example, acoustic information must rapidly be coded in a non-acoustic form, say, in terms of phonemes; but this is only possible if phonemes correspond to local chunks of acoustic input. The processing bottleneck therefore enforces a strong pressure toward local dependencies within a given linguistic level. Importantly, though, this does not imply that linguistic relations are restricted only to adjacent elements but, instead, that they may be formed between any of the small number of elements maintained at a given level of representation. Such representational locality is exemplified across different linguistic levels by the local nature of phonological processes from reduction, assimilation, and fronting, including more elaborate phenomena such as vowel harmony (e.g., Nevins Reference Nevins2010), speech errors (e.g., Cutler Reference Cutler1982), the immediate proximity of inflectional morphemes and the verbs to which they apply, and the vast literature on the processing difficulties associated with non-local dependencies in sentence comprehension (e.g., Gibson Reference Gibson1998; Hawkins Reference Hawkins2004). As noted earlier, the higher the level of linguistic representation, the longer the limited time window within which information can be chunked. Whereas dealing with just two center-embeddings at the sentential level is prohibitively difficult (e.g., de Vries et al. Reference de Vries, Christiansen and Petersson2011; Karlsson Reference Karlsson2007), we are able to deal with up to four to six embeddings at the multi-utterance discourse level (Levinson Reference Levinson2013). This is because chunking takes place at a much longer time course at the discourse level compared with the sentence level, providing more time to resolve the relevant dependency relations before they are subject to interference.

Finally, as indicated by Figure 2, processing within each level of linguistic representation takes place in parallel – but with a clear temporal component – as chunks are passed between levels. Note that, in the Chunk-and-Pass framework, it is entirely possible that linguistic input can simultaneously, and perhaps redundantly, be chunked in more than one way. For example, syntactic chunks and intonational contours may be somewhat independent (Jackendoff Reference Jackendoff2007). Moreover, we should expect further chunking across different “channels” of communication, including visual input such as gesture and facial expressions.

The Chunk-and-Pass perspective is compatible with a number of recent theoretical models of sentence comprehension, including constraint-based approaches (e.g., MacDonald et al. Reference MacDonald, Pearlmutter and Seidenberg1994; Trueswell & Tanenhaus Reference Trueswell, Tanenhaus, Clifton, Frazier and Rayner1994) and certain generative accounts (e.g., Jackendoff's [2007] parallel architecture). Intriguingly, fMRI data from adults (Dehaene-Lambertz et al. Reference Dehaene-Lambertz, Dehaene, Anton, Campagne, Ciuciu, Dehaene, Denghien, Jobert, Lebihan, Sigman, Pallier and Poline2006a) and infants (Dehaene-Lambertz et al. Reference Dehaene-Lambertz, Hertz-Pannier, Dubois, Meriaux, Roche, Sigman and Dehaene2006b) indicate that activation responses to a single sentence systematically slows down when moving away from the primary auditory cortex, either back toward Wernicke's area or forward toward Broca's area, consistent with increasing temporal windows for chunking when moving from phonemes to words to phrases. Indeed, the cortical circuits processing auditory input, from lower (sensory) to higher (cognitive) areas, follow different temporal windows, sensitive to more and more abstract levels of linguistic information, from phonemes and words to sentences and discourse (Lerner et al. Reference Lerner, Honey, Silbert and Hasson2011; Stephens et al. Reference Stephens, Honey and Hasson2013). Similarly, the reverse process, going from a discourse-level representation of the intended message to the production of speech (or sign) across parallel linguistic levels, is compatible with several current models of language production (e.g., Chang et al. Reference Chang, Dell and Bock2006; Dell et al. Reference Dell, Burger and Svec1997; Levelt Reference Levelt2001). Data from intracranial recordings during language production are consistent with different temporal windows for chunk decoding at the word, morphemic, and phonological levels, separated by just over a tenth of a second (Sahin et al. Reference Sahin, Pinker, Cash, Schomer and Halgren2009). These results are compatible with our proposal that incremental processing in comprehension and production takes place in parallel across multiple levels of linguistic representation, each with a characteristic temporal window.

3.3. Implications of Strategy 3: Predictive language processing

We have already noted that, to be able to chunk incoming information as fast and as accurately as possible, the language system exploits multiple constraints in parallel across the different levels of linguistic representation. Such cues may be used not only to help disambiguate previous input, but also to generate expectations for what may come next, potentially further speeding up Chunk-and-Pass processing. Computational considerations indicate that simple statistical information gleaned from sentences provides powerful predictive constraints on language comprehension and can explain many human processing results (e.g., Christiansen & Chater Reference Christiansen and Chater1999; Christiansen & MacDonald Reference Christiansen and MacDonald2009; Elman Reference Elman1990; Hale Reference Hale2006; Jurafsky Reference Jurafsky1996; Levy Reference Levy2008; Padó et al. Reference Padó, Crocker and Keller2009). Similarly, eye-tracking data suggest that comprehenders routinely use a variety of sources of probabilistic information – from phonological cues to syntactic context and real-world knowledge – to anticipate the processing of upcoming words (e.g., Altmann & Kamide Reference Altmann and Kamide1999; Farmer et al. Reference Farmer, Monaghan, Misyak and Christiansen2011; Staub & Clifton Reference Staub and Clifton2006). Results from event-related potential experiments indicate that rather specific predictions are made for upcoming input, including its lexical category (Hinojosa et al. Reference Hinojosa, Moreno, Casado, Munõz and Pozo2005), grammatical gender (Van Berkum et al. Reference Van Berkum, Brown, Zwitserlood, Kooijman and Hagoort2005; Wicha et al. Reference Wicha, Moreno and Kutas2004), and even its onset phoneme (DeLong et al. Reference DeLong, Urbach and Kutas2005) and visual form (Dikker et al. Reference Dikker, Rabagliati, Farmer and Pylkkänen2010). Accordingly, there is a growing body of evidence for a substantial role of prediction in language processing (for reviews, see, e.g., Federmeier Reference Federmeier2007; Hagoort Reference Hagoort, Bickerton and Szathmáry2009; Kamide Reference Kamide2008; Kutas et al. Reference Kutas, Federmeier, Urbach, Gazzaniga and Mangun2014; Pickering & Garrod Reference Pickering and Garrod2007) and evidence that such language prediction occurs in children as young as 2 years of age (Mani & Huettig Reference Mani and Huettig2012). Importantly, as well as exploiting statistical relations within a representational level, predictive processing allows top-down information from higher levels of linguistic representation to rapidly constrain the processing of the input at lower levels.Footnote ¹⁰

From the viewpoint of the Now-or-Never bottleneck, prediction provides an opportunity to begin Chunk-and-Pass processing as early as possible: to constrain representations of new linguistic material as it is encountered, and even incrementally to begin recoding predictable linguistic input before it arrives. This viewpoint is consistent with recent suggestions that the production system may be pressed into service to anticipate upcoming input (e.g., Pickering & Garrod Reference Pickering and Garrod2007; Reference Pickering and Garrod2013a). Chunk-and-Pass processing implies that there is practically no possibility for going back once a chunk is created because such backtracking tends to derail processing (e.g., as in the classic garden path phenomena mentioned above). This imposes a Right-First-Time pressure on the language system in the face of linguistic input that is highly locally ambiguous.Footnote ¹¹ The contribution of predictive modeling to comprehension is that it facilitates local ambiguity resolution while the stimulus is still available. Only by recruiting multiple cues and integrating these with predictive modeling is it possible to resolve local ambiguities quickly and correctly.

Right-First-Time parsing fits with proposals such as that by Marcus (Reference Marcus1980), where local ambiguity resolution is delayed until later disambiguating information arrives, and models in which aspects of syntactic structure may be underspecified, therefore not requiring the ambiguity to be resolved (e.g., Gorrell Reference Gorrell1995; Sturt & Crocker Reference Sturt and Crocker1996). It also parallels Marr's (Reference Marr1976) principle of least commitment, as we mentioned earlier, according to which the perceptual system should, as far as possible, only resolve perceptual ambiguities when sufficiently confident that they will not need to be undone. Moreover, it is compatible with the fine-grained weakly parallel interactive model (Altmann & Steedman Reference Altmann and Steedman1988) in which possible chunks are proposed, word-by-word, by an autonomous parser and one is rapidly chosen using top-down information.

To facilitate chunking across multiple levels of representation, prediction takes place in parallel across the different levels but at varying timescales. Predictions for higher-level chunks may run ahead of those for lower-level chunks. For example, most people simply answer “two” in response to the question “How many animals of each kind did Moses take on the Ark?” – failing to notice the semantic anomaly (i.e., it was Noah's Ark, not Moses' Ark) even in the absence of time pressure and when made aware that the sentence may be anomalous (Erickson & Matteson Reference Erickson and Matteson1981). That is, anticipatory pragmatic and communicative considerations relating to the required response appear to trump lexical semantics. More generally, the time course of normal conversation may lead to an emphasis on more temporally extended higher-level predictions over lower-level ones. This may facilitate the rapid turn-taking that has been observed cross-culturally (Stivers et al. Reference Stivers, Enfield, Brown, Englert, Hayashi, Heinemann, Hoymann, Rossano, de Ruiter, Yoon and Levinson2009) and which seems to require that listeners make quite specific predictions about when the speaker's current turn will finish (Magyari & De Ruiter Reference Magyari and de Ruiter2012), as well as being able to quickly adapt their expectations to specific linguistic environments (Fine et al. Reference Fine, Jaeger, Farmer and Qian2013).

We view the anticipation of turn-taking as one instance of the broader alignment that takes place between dialogue partners across all levels of linguistic representation (for a review, see Pickering & Garrod Reference Pickering and Garrod2004). This dovetails with fMRI analyses indicating that although there are some comprehension- and production-specific brain areas, spatiotemporal patterns of brain activity are in general closely coupled between speakers and listeners (e.g., Silbert et al. Reference Silbert, Honey, Simony, Poeppel and Hasson2014). In particular, Stephens et al. (Reference Stephens, Silbert and Hasson2010) observed close synchrony between neural activations in speakers and listeners in early auditory areas. Speaker activations preceded those of listeners in posterior brain regions (including parts of Wernicke's area), whereas listener activations preceded those of speakers in the striatum and anterior frontal areas. In the Chunk-and-Pass framework, the listener lag primarily derives from delays caused by the chunking process across the various levels of linguistic representation, whereas the speaker lag predominantly reflects the listener's anticipation of upcoming input, especially at the higher levels of representation (e.g., pragmatics and discourse). Strikingly, the extent of the listener's anticipatory brain responses were strongly correlated with successful comprehension, further underscoring the importance of prediction-based alignment for language processing. Indeed, analyses of real-time interactions show that alignment increases when the communicative task becomes more difficult (Louwerse et al. Reference Louwerse, Dale, Bard and Jeuniaux2012). By decreasing the impact of potential ambiguities, alignment thus makes processing as well as production easier in the face of the Now-or-Never bottleneck.

We have suggested that only an incremental, predictive language system, continually building and passing on new chunks of linguistic material, encoded at increasingly abstract levels of representation, can deal with the onslaught of linguistic input in the face of the severe memory constraints of the Now-or-Never bottleneck. We suggest that a productive line of future work is to consider the extent to which existing models of language are compatible with these constraints, and to use these properties to guide the creation of new theories of language processing.

4.1. Implications of Strategy 1: Online learning

The Now-or-Never bottleneck implies that learning can depend only on material currently being processed. As we have seen, this implication requires a processing strategy according to which modification to current representations (in this context, learning) occurs right away; in machine-learning terminology, learning is online. If learning does not occur at the time of processing, the representation of linguistic material will be obliterated, and the opportunity for learning will be gone forever. To facilitate such online learning, the child must learn to use all available information to help constrain processing. The integration of multiple constraints – or cues – is a fundamental component of many current theories of language acquisition (see, e.g., contributions in Golinkoff et al. Reference Golinkoff, Hirsh-Pasek, Bloom, Smith, Woodward, Akhtar, Tomasello and Hollich2000; Morgan & Demuth Reference Morgan and Demuth1996; Weissenborn & Höhle Reference Weissenborn and Höhle2001; for a review, see Monaghan & Christiansen Reference Monaghan, Christiansen and Behrens2008). For example, second-graders' initial guesses about whether a novel word refers to an object or an action are affected by that word's phonological properties (Fitneva et al. Reference Fitneva, Christiansen and Monaghan2009); 7-year-olds use visual context to constrain online sentence interpretation (Trueswell et al. Reference Trueswell, Sekerina, Hill and Logrip1999); and preschoolers' language production and comprehension is constrained by pragmatic factors (Nadig & Sedivy Reference Nadig and Sedivy2002). Thus, children learn rapidly to apply the multiple constraints used in incremental adult processing (Borovsky et al. Reference Borovsky, Elman and Fernald2012).

Nonetheless, online learning contrasts with traditional approaches in which the structure of the language is learned offline by the cognitive system acquiring a corpus of past linguistic inputs and choosing the grammar or other model of the language that best fits with those inputs. For example, in both mathematical and theoretical analysis (e.g., Gold Reference Gold1967; Hsu et al. Reference Hsu, Chater and Vitányi2011; Pinker Reference Pinker1984) and in grammar-induction algorithms in machine learning and cognitive science, it is typically assumed that a corpus of language can be held in memory, and that the candidate grammar is successively adjusted to fit the corpus as well as possible (e.g., Manning & Schütze Reference Manning and Schütze1999; Pereira & Schabes Reference Pereira, Schabes and Thompson1992; Redington et al. Reference Redington, Chater and Finch1998). However, this approach involves learning linguistic regularities (at, say, the morphological level), by storing and later surveying relevant linguistic input at a lower level of analysis (e.g., involving strings of phonemes); and then attempting to determine which higher-level regularities best fit the database of lower-level examples. There are a number of difficulties with this type of proposal – for example, that only a very rich lower-level representation (perhaps combined with annotations concerning relevant syntactic and semantic context) is likely to be a useful basis for later analysis. But more fundamentally, the Now-or-Never bottleneck requires that information be retained only if it is recoded at processing time: Phonological information that is not chunked at the morphological level and beyond will be obliterated by oncoming phonological material.Footnote ¹²

So, if learning is shaped by the Now-or-Never bottleneck, then linguistic input must, when it is encountered, be recoded successively at increasingly abstract linguistic levels if it is to be retained at all – a constraint imposed, we argue, by basic principles of memory. Crucially, such information is not, therefore, in a suitably “neutral” format to allow for the discovery of previously unsuspected linguistic regularities. In a nutshell, the lossy compression of the linguistic input is achieved by applying the learner's current model of the language. But information that would point toward a better model of the language (if examined in retrospect) will typically be lost (or, at best, badly obscured) by this compression, precisely because those regularities are not captured by the current model of the language. Suppose, for example, that we create a lossy encoding of language using a simple, context-free phrase structure grammar that cannot handle, say, noun-verb agreement. The lossy encoding of the linguistic input produced using this grammar will provide a poor basis for learning a more sophisticated grammar that includes agreement – precisely because agreement information will have been thrown away. So the Now-or-Never bottleneck rules out the possibility that the learner can survey a neutral database of linguistic material, to optimize its model of the language.

The emphasis on online learning does not, of course, rule out the possibility that any linguistic material that is remembered may subsequently be used to inform learning. But according to the present viewpoint, any further learning requires reprocessing that material. So if a child comes to learn a poem, song, or story verbatim, the child might extract more structure from that material by mental rehearsal (or, indeed, by saying it aloud). The online learning constraint is that material is learned only when it is being processed – ruling out any putative learning processes that involve carrying out linguistic analyses or compiling statistics over a stored corpus of linguistic material.

If this general picture of acquisition as learning-to-process is correct, then we should expect the exploitation of memory to require “replaying” learned material, so that it can be re-processed. Thus, the application of memory itself requires passing through the Now-or-Never bottleneck – there is no way of directly interrogating an internal database of past experience; indeed, this viewpoint fits with our subjective sense that we need to “bring to mind” past experiences or rehearse verbal material to process it further. Interestingly, there is now also substantial neuroscientific evidence that replay does occur (e.g., in rat spatial learning, Carr et al. Reference Carr, Jadhav and Frank2011). Moreover, it has long been suggested that dreaming may have a related function (here using “reverse” learning over “fictional” input to eliminate spurious relationships identified by the brain, Crick & Mitchison Reference Crick and Mitchison1983; see Hinton & Sejnowki Reference Hinton, Sejnowski, Irwin and Sejnowski1986, for a closely related computational model). Deficits in the ability to replay material would, in this view, lead to consequent deficits in memory and inference; consistent with this viewpoint, Martin and colleagues have argued that rehearsal deficits for phonological pattern and semantic information may lead to difficulties in the long-term acquisition and retention of word forms and word meanings, respectively, and their use in language processing (e.g., Martin & He Reference Martin and He2004; Martin et al. Reference Martin, Shelton and Yaffee1994). In summary, then, language acquisition involves learning to process, and generalizations can only be made over past processing episodes.

4.2. Implications of Strategy 2: Local learning

Online learning faces a particularly acute version of a general learning problem: the stability-plasticity dilemma (e.g., Mermillod et al. Reference Mermillod, Bugaïska and Bonin2013). How can new information be acquired without interfering with prior information? The problem is especially challenging because reviewing prior information is typically difficult (because recalling earlier information interferes with new input) or impossible (where prior input has been forgotten). Thus, to a good approximation, the learner can only update its model of the language in a way that responds to current linguistic input, without being able to review whether any updates are inconsistent with prior input. Specifically, if the learner has a global model of the entire language (e.g., a traditional grammar), the learner runs the risk of overfitting that model to capture regularities in the momentary linguistic input at the expense of damaging the match with past linguistic input.

Avoiding this problem, we suggest, requires that learning be highly local, consisting of learning about specific relationships between particular linguistic representations. New items can be acquired, with implications for later processing of similar items; but learning current items does not thereby create changes to the entire model of the language, thus potentially interfering with what was learned from past input. One way to learn in a local fashion is to store individual examples (this requires, in our framework, that those examples have been abstractly recoded by successive Chunk-and-Pass operations, of course), and then to generalize, piecemeal, from these examples. This standpoint is consistent with the idea that the “priority of the specific,” as observed in other areas of cognition (e.g., Jacoby et al. Reference Jacoby, Baker and Brooks1989), also applies to language acquisition. For example, children seem to be highly sensitive to multiword chunks (Arnon & Clark Reference Arnon and Clark2011; Bannard & Matthews Reference Bannard and Matthews2008; see Arnon & Christiansen, Reference Arnon and Christiansensubmitted, for a reviewFootnote ¹³ ). More generally, learning based on past traces of processing will typically be sensitive to details of that processing, as is observed across phonetics, phonology, lexical access, syntax, and semantics (e.g., Bybee Reference Bybee2006; Goldinger Reference Goldinger1998; Pierrehumbert Reference Pierrehumbert2002; Tomasello Reference Tomasello1992).

That learning is local provides a powerful constraint, incompatible with typical computational models of how the child might infer the grammar of the language – because these models typically do not operate incrementally but range across the input corpus, evaluating alternative grammatical hypotheses (so-called batch learning). But, given the Now-or-Never bottleneck, the “unprocessed” corpus, so readily available to the linguistic theorist, or to a computer model, is lost to the human learner almost as soon as it is encountered. Where such information has been memorized (as in the case of SF's encoding of streams of digits), recall and processing is slow and effortful. Moreover, because information is encoded in terms of the current encoding, it becomes difficult to neutrally review that input to create a better encoding, and cross-check past data to test wide-ranging grammatical hypotheses.Footnote ¹⁴ So, as we have already noted, the Now-or-Never bottleneck seems incompatible with the view of a child as a mini-linguist.

By contrast, the principle of local learning is respected by other approaches. For example, item-based (Tomasello Reference Tomasello2003), connectionist (e.g., Chang et al. 1999; Elman Reference Elman1990; MacDonald & Christiansen Reference MacDonald and Christiansen2002),Footnote ¹⁵ exemplar-based (e.g., Bod Reference Bod2009), and other usage-based (e.g., Arnon & Snider Reference Arnon and Snider2010; Bybee Reference Bybee2006) accounts of language acquisition tie learning and processing together – and assume that language is acquired piecemeal, in the absence of an underlying Bauplan. Such accounts, based on local learning, provide a possible explanation of the frequency effects that are found at all levels of language processing and acquisition (e.g., Bybee Reference Bybee2007; Bybee & Hopper Reference Bybee and Hopper2001; Ellis Reference Ellis2002; Tomasello Reference Tomasello2003), analogous to exemplar-based theories of how performance speeds up with practice (Logan Reference Logan1988).

The local nature of learning need not, though, imply that language has no integrated structure. Just as in perception and action, local chunks can be defined at many different levels of abstraction, including highly abstract patterns, for example, governing subject, verb, and object; and generalizations from past processing to present processing will operate across all of these levels. Therefore, in generating or understanding a new sentence, the language user will be influenced by the interaction of multiple constraints from innumerable traces of past processing, across different linguistic levels. This view of language processing involving the parallel interaction of multiple local constraints is embodied in a variety of influential approaches to language (e.g., Jackendoff Reference Jackendoff2007; Seidenberg Reference Seidenberg1997).

4.3. Implications of Strategy 3: Learning to predict

If language processing involves prediction – to make the encoding of new linguistic material sufficiently rapid – then a critical aspect of language acquisition is learning to make such predictions successfully (Altmann & Mirkovic Reference Altmann and Mirkovic2009). Perhaps the most natural approach to predictive learning is to compare predictions with subsequent reality, thus creating an “error signal,” and then to modify the predictive model to systematically reduce this error. Throughout many areas of cognition, such error-driven learning has been widely explored in a range of computational frameworks (e.g., from connectionist networks, to reinforcement learning, to support vector machines) and has considerable behavioral (e.g., Kamin Reference Kamin, Campbell and Church1969) and neurobiological support (e.g., Schultz et al. Reference Schultz, Dayan and Montague1997).

Predictive learning can, in principle, take a number of forms: For example, predictive errors can be accumulated over many samples, and then modifications made to the predictive model to minimize the overall error over those samples (i.e., batch learning). But this is ruled out by the Now-or-Never bottleneck: Linguistic input, and the predictions concerning it, is present only fleetingly. But error-driven learning can also be “online” – each prediction error leads to an immediate, though typically small, modification of the predictive model; and the accumulation of these small modifications gradually reduces prediction errors on future input.

A number of computational models adhere to these principles: Learning involves creating a predictive model of the language, using online error-driven learning. Such models, limited though they are, may provide a starting point for creating an increasingly realistic account of language acquisition and processing. For example, a connectionist model which embodies these principles is the simple recurrent network (Altmann Reference Altmann2002; Christiansen & Chater Reference Christiansen and Chater1999; Elman Reference Elman1990), which learns to map from the current input on to the next element in a continuous sequence of linguistic (or other) input; and which learns, online, by adjusting its parameters (the “weights” of the network) to reduce the observed prediction error, using the back-propagation learning algorithm. Using a very different framework, in the spirit of construction grammar (e.g., Croft Reference Croft2001; Goldberg Reference Goldberg2006), McCauley and Christiansen (Reference McCauley, Christiansen, Carlson, Hölscher and Shipley2011) recently developed a psychologically based, online chunking model of incremental language acquisition and processing , incorporating prediction to generalize to new chunk combinations. Exemplar-based analogical models of language acquisition and processing may also be constructed, which build and predict language structure online, by incrementally creating a database of possible structures, and dynamically using online computation of similarity to recruit these structures to process and predict new linguistic input.

Importantly, prediction allows for top-down information to influence current processing across different levels of linguistic representation, from phonology to discourse, and at different temporal windows (as indicated by Fig. 2). We see the ability to use such top-down information as emerging gradually across development, building on bottom-up information. That is, children gradually learn to apply top-down knowledge to facilitate processing via prediction, as higher-level information becomes more entrenched and allows for anticipatory generalizations to be made.

In this section, we have argued that the child should not be viewed as a mini-linguist, attempting to infer the abstract structure of grammar, but as learning to process: that is, learning to alleviate the severe constraints imposed by the Now-or-Never bottleneck. Next, we discuss how chunk-based language acquisition and processing have shaped linguistic change and, ultimately, the evolution of language.

6.1. Explaining key properties of language

6.1.1. The bounded nature of linguistic units

In nonlinguistic sequential tasks, memory constraints are so severe that chunks of more than a few items are rare. People typically encode phone numbers, number plates, postal codes, and Social Security numbers into sequences of between two and four digits or letters; memory recall deteriorates rapidly for unchunked item-sequences longer than about four elements (Cowan Reference Cowan2000), and memory recall typically breaks into short chunk-like phrases. Similar chunking processes are thought to govern nonlinguistic sequences of actions (e.g., Graybiel Reference Graybiel1998). As we have argued previously in this article, the same constraints apply throughout language processing, from sound to discourse.

Across different levels of linguistic representation, units also tend to have only a few component elements. Even though the nature of sound-based units in speech is theoretically contentious, all proposals capture the sharply bounded nature of such units. For example, a traditional perspective on English phonology would postulate phonemes, short sequences of which are grouped into syllables, with multisyllabic words being organized by intonational or perhaps morphological groupings. Indeed, the tendency toward few-element units is so strong that long, nonsense words with many syllables such as supercalifragilisticexpialidocious is chunked successively, for example, as tentatively indicated:

$$[[[\hbox{Super}][\hbox{cali}]][[\hbox{fragi}][\hbox{listic}]][[\hbox{expi}][\hbox{ali}]][\hbox{docious}]]$$

Similarly, agglutinating languages, such as Turkish, chunk complex multimorphemic words using local grouping mechanisms that include formulaic morpheme expressions (Durrant Reference Durrant2013). Likewise, at higher levels of linguistic representation, verbs normally have only two or three arguments at most. Across linguistic theories of different persuasions, syntactic phrases typically consist of only a few constituents. Thus, the Now-or-Never bottleneck provides a strong bias toward bounded linguistic units across various levels of linguistic representations.

6.2. What is linguistic structure?

Chunk-and-Pass processing can be viewed as having an interesting connection with traditional linguistic notions. In both production and comprehension, the language system creates a sequence of chunking operations, which link different linguistic units together across multiple levels of structure. That is, the syntactic structure of a given utterance is reflected in its processing history. This conception is reminiscent of previous proposals, in which syntax is viewed as a control structure for guiding semantic interpretation (e.g., Ford et al. Reference Ford, Bresnan, Kaplan and Bresnan1982; Kempson et al. 2001; Morrill Reference Morrill2010). For example, in describing his incremental parser-interpreter, Pulman (Reference Pulman1985) noted, “Syntactic information is used to build up the interpretation and to guide the parse, but does not result in the construction of an independent level of representation” (p. 132). Steedman (Reference Steedman2000) adopted a closely related perspective when introducing his combinatory categorial grammar, which aims to map surface structure directly onto logic-based semantic interpretations, given rich lexical representations of words that include information about phonological structure, syntactic category, and meaning: “… syntactic structure is merely the characterization of the process of constructing a logical form, rather than a representational level of structure that actually needs to be built …” (p. xi). Thus, in these accounts, the syntactic structure of a sentence is not explicitly represented by the language system but plays the role of a processing “trace” of the operations used to create or interpret the sentence (see also O'Grady Reference O'Grady2005).

To take an analogy from constructing objects, rather than sentences, the process by which components of an IKEA-style flat-pack cabinet are combined provides a “history” (combine a board, handle, and screws to construct the doors; combine frame and shelf to construct the body; combine doors, body, and legs to create the finished cabinet). The history by which the cabinet was constructed may thus reveal the intricate structure of the finished item, but this structure need not be explicitly represented during the construction process. Thus, we can “read off” the syntactic structure of a sentence from its processing history, revealing the syntactic relations between various constituents (likely with a “flat” structure; Frank et al. Reference Frank, Bod and Christiansen2012). Syntactic representations are neither computed during comprehension nor in production; instead, there is just a history of processing operations. That is, we view linguistic structure as processing history. Importantly, this means that syntax is not privileged but is only one part of the system – and it is not independent of the other parts (see also Fig. 2).

In this view, a rather minimal notion of grammar specifies how the chunks from which a sentence is built can be composed. There may be several, or indeed many, orders in which such combinations can occur, just as operations for furniture assembly may be carried out somewhat flexibly (but not completely without constraints – it might turn out that the body must be screwed together before a shelf can be attached). In the context of producing and understanding language, the process of construction is likely to be much more constrained: Each new “component” is presented in turn, and it must be used immediately or it will be lost. Moreover, viewing Chunk-and-Pass processing as an aspect of skill acquisition, we might expect that the precise nature of chunks may change with expertise: Highly overlearned material might, for example, gradually come to be treated as a single chunk (see Arnon & Christiansen, Reference Arnon and Christiansensubmitted, for a review).

Crucially, as with other skills, the cognitive system will tend to be a cognitive miser (Fiske & Taylor Reference Fiske and Taylor1984), generally following a principle of least effort (Zipf Reference Zipf1949). As processing proceeds, there is an intricate interplay of top-down and bottom-up processing to alight on the message as rapidly as possible. The language system need only construct enough chunk structure so that, when combined with prior discourse and background knowledge, the intended message can be inferred incrementally. This observation relates to some interesting contemporary linguistic proposals. For example, from a generative perspective, Culicover (Reference Culicover2013) highlighted the importance of incremental processing, arguing that the interpretation of a pronoun depends on which discourse elements are available when it is encountered. This implies that the linear order of words in a sentence (rather than hierarchical structure) plays an important role in many apparently grammatical phenomena, including weak cross-over effects in referential binding. From an emergentist perspective, O'Grady (Reference O'Grady, MacWhinney and O'Grady2015) similarly emphasized the importance of real-time processing constraints for explaining differences in the interpretation of reflexive pronouns (himself, themselves) and plain pronouns (him, them). The former are resolved locally, and thus almost instantly, whereas the antecedents for the latter are searched for within a broader domain (causing problems in acquisition because of a bias toward local information).

More generally, our view of linguistic structure as processing history offers a way to integrate the formal linguistic contributions of construction grammar (e.g., Croft Reference Croft2001; Goldberg Reference Goldberg2006) with the psychological insights from usage-based approaches to language acquisition and processing (e.g., Bybee & McClelland Reference Bybee and McClelland2005; Tomasello Reference Tomasello2003). Specifically, we propose to view constructions as computational procedures Footnote ¹⁹  – specifying how to process and produce a particular chunk – where we take a broad view of constructions as involving chunks at different levels of linguistic representation, from morphemes to multiword sequences. A procedure may integrate several different aspects of language processing or production, including chunking acoustic input into sound-based units (phonemes, syllables), mapping a chunk onto meaning (or vice versa), incorporating pragmatic or discourse information, and associating a chunk with specific arguments (see also O'Grady Reference O'Grady2005; Reference O'Grady2013). As with other skills (e.g., Heathcote et al. Reference Heathcote, Brown and Mewhort2000; Newell & Rosenbloom Reference Newell, Rosenbloom and Anderson1981), there will be practice effects, where the repeated use of a given chunk results in faster processing and reduced demands on cognitive resources, and with sufficient use, leading to a high degree of automaticity (e.g., Logan Reference Logan1988; see Bybee & McClelland Reference Bybee and McClelland2005, for a linguistic perspective).

In terms of our previous forest track analogy, the more a particular chunk is comprehended or produced, the more entrenched it becomes, resulting in easier access and faster processing; tracks become more established with use. With sufficiently frequent use, adjacent tracks may blend together, creating somewhat wider paths. For example, the frequent processing of simple transitive sentences, processed individually as multiword chunks, such as “I want milk,” “I want candy,” might first lead to a wider track involving the item-based template “I want X.” Repeated use of this template along with others (e.g., “I like X,” “I see X”) might eventually give rise to a more abstract transitive generalization along the lines of N V N (a highway in terms of our track analogy). Similar proposals for the emergence of basic word order patterns have been proposed both within emergentist (e.g., O'Grady Reference O'Grady2005; Reference O'Grady2013; Tomasello Reference Tomasello2003) and generative perspectives (e.g., Townsend & Bever Reference Townsend and Bever2001). Importantly, however, just as with generalizations in perception and motor skills, the grammatical abstractions are not explicitly represented but result from the merging of item-based procedures for chunking. Consequently, there is no representation of grammatical structure separate from processing. Learning to process is learning the grammar.