Christiansen & Chater (C&C) focus on cognitive limitations in language understanding and production that force immediate decisions at multiple levels. Our experiments using rapid serial visual presentation (RSVP) of written words and of pictured scenes show that a large-capacity but short-lasting conceptual short-term memory (CSTM), consisting of associations from long-term memory, is retrieved in response to currently active stimuli and thoughts (Potter Reference Potter1993; Reference Potter2012). We “understand” when some structural connections are found between the current stimuli and CSTM. In visual perception of scenes and objects, as well as in language comprehension, we make quick use of knowledge that briefly exceeds the supposed limits of short-term memory. Consistent with C&C's core ideas, rich but unselective associations arise quickly but last only long enough for selective pattern recognition – chunking, in C&C's terms. Irrelevant associations never become conscious (or are immediately forgotten).
Three interrelated characteristics of CSTM support key ideas in C&C's target article. Demos of some of these effects can be seen on Scholarpedia (Potter Reference Potter2009).
1. There is rapid access to conceptual (semantic) information about a stimulus and its associations
Conceptual information about a word or a picture is available within 100–300 ms, as shown by experiments using semantic priming (Neely Reference Neely, Besner and Humphreys1991), including masked priming (Forster & Davis Reference Forster and Davis1984); eye tracking when reading (Rayner Reference Rayner1983; Reference Rayner1992) or looking at pictures (Loftus Reference Loftus and Rayner1983); measurement of event-related potentials during reading (Kutas & Hillyard Reference Kutas and Hillyard1980; Luck et al. Reference Luck, Vogel and Shapiro1996); and target detection in RSVP with letters and digits (Chun & Potter Reference Chun and Potter1995; Sperling et al. Reference Sperling, Budiansky, Spivak and Johnson1971), with pictures (Intraub Reference Intraub1981; Meng & Potter Reference Meng and Potter2008; Potter Reference Potter1976; Potter et al. Reference Potter, Wyble, Pandav and Olejarczyk2010 ), or with words (Davenport & Potter Reference Davenport and Potter2005; Lawrence Reference Lawrence1971b; Meng & Potter Reference Meng and Potter2011; Potter et al. Reference Potter, Staub and O'Connor2002). Conceptually defined targets can be detected in a stream of nontargets presented at rates of 8–10 items per second or faster (Potter et al. Reference Potter, Wyble, Hagmann and McCourt2014), showing that categorical information about a written word or picture is activated and then selected extremely rapidly. The converging evidence shows that semantic or conceptual characteristics of a stimulus have an effect on performance as early as 100 ms after its onset. This time course is too rapid for slower cognitive processes, such as intentional encoding, deliberation, or serial comparison in working memory.
2. New structures can be discovered or built out of the momentarily activated conceptual information, influenced by the observer's task or goal
Evidence for this claim comes from comparing responses to RSVP sentences, scrambled sentences, and lists of unrelated words. It is possible to process the syntactic and conceptual structure in a sentence and, hence, subsequently to recall it, when reading at a rate such as 12 words per second (Forster Reference Forster1970; Potter Reference Potter, Kieras and Just1984; Reference Potter1993; Potter et al. Reference Potter, Kroll, Harris and Nickerson1980; Reference Potter, Kroll, Yachzel, Carpenter and Sherman1986). In contrast, when short lists of unrelated words are presented at that rate, only two or three words can be recalled (see also Lawrence Reference Lawrence1971a). For sentences, the meaning and plausibility of the sentence, as well as the syntactic structure, are recovered as the sentence is processed. Words that do not fit the syntax or meaning are systematically misperceived (Potter et al. Reference Potter, Moryadas, Abrams and Noel1993). Syntactic and semantic choices are made online (Potter et al. Reference Potter, Stiefbold and Moryadas1998). Memory for the sentence may be reconstructed from meaning, rather than recalled word for word (Lombardi & Potter Reference Lombardi and Potter1992; Potter & Lombardi Reference Potter and Lombardi1990; Reference Potter and Lombardi1998). Because almost all of the sentences one normally encounters (and all of the experimental sentences) include new combinations of ideas, structure-building is not simply a matter of locating a previously encountered pattern in long-term memory: It involves the creation of a new relationship among existing concepts.
As with words, so with a new pictured scene: Not only must critical objects and the setting be identified, but also the relations among them – the gist of the picture (e.g., Davenport & Potter Reference Davenport and Potter2004). Associated long-term memory of visual scenes must be activated to recognize that one is looking at a picnic, or a bride and groom, or a ball game. As C&C suggest, structure-building presumably takes advantage of as much old structure as possible, using any preexisting associations and chunks of information to bind elements.
3. There is rapid forgetting of information that is not structured or that is not selected for further processing
Conceptual information is activated rapidly, but the initial activation is highly unstable and will be deactivated and forgotten within a few hundred milliseconds if it is not incorporated into a structure, consistent with C&C's proposal. As a structure is built – for example, as a sentence is being parsed and interpreted – the resulting interpretation can be held in memory and ultimately stabilized or consolidated in working or long-term memory as a unit, whereas only a small part of an unstructured sequence such as a string of unrelated words or an incoherent picture can be consolidated in the same time period.
Because similar principles seem to apply to language comprehension and to nonlinguistic visual understanding, I have proposed that understanding in both cases is abstractly conceptual rather than fundamentally language-based. For example, pictured objects and their names give equivalent and equally rapid information about meaning (Potter & Faulconer Reference Potter and Faulconer1975; Potter et al. Reference Potter, Valian and Faulconer1977). Other perceptual senses such as audition and touch also have rapid access to the same conceptual level.
If the CSTM hypothesis is correct, then the Now-or-Never bottleneck occurs after a rich set of associations from long-term memory has enabled conceptual chunking of incoming linguistic or visual information. At that point, the information can be passed through the bottleneck to a more abstract level of discourse or scene understanding. Moreover, the severe limitations of working memory seen for arbitrary lists of letters, numbers, or geometric figures are largely overcome when proactive interference from reuse of a small set of stimuli is eliminated (Endress & Potter Reference Endress and Potter2014a). The desperate speed of processing noted by C&C is not due solely to the limitations of short-term memory, but more generally reflects the pressure to think, see, understand, and act as fast as possible, in order to survive in a predatory world.
You have
Access
Christiansen & Chater (C&C) focus on cognitive limitations in language understanding and production that force immediate decisions at multiple levels. Our experiments using rapid serial visual presentation (RSVP) of written words and of pictured scenes show that a large-capacity but short-lasting conceptual short-term memory (CSTM), consisting of associations from long-term memory, is retrieved in response to currently active stimuli and thoughts (Potter Reference Potter1993; Reference Potter2012). We “understand” when some structural connections are found between the current stimuli and CSTM. In visual perception of scenes and objects, as well as in language comprehension, we make quick use of knowledge that briefly exceeds the supposed limits of short-term memory. Consistent with C&C's core ideas, rich but unselective associations arise quickly but last only long enough for selective pattern recognition – chunking, in C&C's terms. Irrelevant associations never become conscious (or are immediately forgotten).
Three interrelated characteristics of CSTM support key ideas in C&C's target article. Demos of some of these effects can be seen on Scholarpedia (Potter Reference Potter2009).
1. There is rapid access to conceptual (semantic) information about a stimulus and its associations
Conceptual information about a word or a picture is available within 100–300 ms, as shown by experiments using semantic priming (Neely Reference Neely, Besner and Humphreys1991), including masked priming (Forster & Davis Reference Forster and Davis1984); eye tracking when reading (Rayner Reference Rayner1983; Reference Rayner1992) or looking at pictures (Loftus Reference Loftus and Rayner1983); measurement of event-related potentials during reading (Kutas & Hillyard Reference Kutas and Hillyard1980; Luck et al. Reference Luck, Vogel and Shapiro1996); and target detection in RSVP with letters and digits (Chun & Potter Reference Chun and Potter1995; Sperling et al. Reference Sperling, Budiansky, Spivak and Johnson1971), with pictures (Intraub Reference Intraub1981; Meng & Potter Reference Meng and Potter2008; Potter Reference Potter1976; Potter et al. Reference Potter, Wyble, Pandav and Olejarczyk2010 ), or with words (Davenport & Potter Reference Davenport and Potter2005; Lawrence Reference Lawrence1971b; Meng & Potter Reference Meng and Potter2011; Potter et al. Reference Potter, Staub and O'Connor2002). Conceptually defined targets can be detected in a stream of nontargets presented at rates of 8–10 items per second or faster (Potter et al. Reference Potter, Wyble, Hagmann and McCourt2014), showing that categorical information about a written word or picture is activated and then selected extremely rapidly. The converging evidence shows that semantic or conceptual characteristics of a stimulus have an effect on performance as early as 100 ms after its onset. This time course is too rapid for slower cognitive processes, such as intentional encoding, deliberation, or serial comparison in working memory.
2. New structures can be discovered or built out of the momentarily activated conceptual information, influenced by the observer's task or goal
Evidence for this claim comes from comparing responses to RSVP sentences, scrambled sentences, and lists of unrelated words. It is possible to process the syntactic and conceptual structure in a sentence and, hence, subsequently to recall it, when reading at a rate such as 12 words per second (Forster Reference Forster1970; Potter Reference Potter, Kieras and Just1984; Reference Potter1993; Potter et al. Reference Potter, Kroll, Harris and Nickerson1980; Reference Potter, Kroll, Yachzel, Carpenter and Sherman1986). In contrast, when short lists of unrelated words are presented at that rate, only two or three words can be recalled (see also Lawrence Reference Lawrence1971a). For sentences, the meaning and plausibility of the sentence, as well as the syntactic structure, are recovered as the sentence is processed. Words that do not fit the syntax or meaning are systematically misperceived (Potter et al. Reference Potter, Moryadas, Abrams and Noel1993). Syntactic and semantic choices are made online (Potter et al. Reference Potter, Stiefbold and Moryadas1998). Memory for the sentence may be reconstructed from meaning, rather than recalled word for word (Lombardi & Potter Reference Lombardi and Potter1992; Potter & Lombardi Reference Potter and Lombardi1990; Reference Potter and Lombardi1998). Because almost all of the sentences one normally encounters (and all of the experimental sentences) include new combinations of ideas, structure-building is not simply a matter of locating a previously encountered pattern in long-term memory: It involves the creation of a new relationship among existing concepts.
As with words, so with a new pictured scene: Not only must critical objects and the setting be identified, but also the relations among them – the gist of the picture (e.g., Davenport & Potter Reference Davenport and Potter2004). Associated long-term memory of visual scenes must be activated to recognize that one is looking at a picnic, or a bride and groom, or a ball game. As C&C suggest, structure-building presumably takes advantage of as much old structure as possible, using any preexisting associations and chunks of information to bind elements.
3. There is rapid forgetting of information that is not structured or that is not selected for further processing
Conceptual information is activated rapidly, but the initial activation is highly unstable and will be deactivated and forgotten within a few hundred milliseconds if it is not incorporated into a structure, consistent with C&C's proposal. As a structure is built – for example, as a sentence is being parsed and interpreted – the resulting interpretation can be held in memory and ultimately stabilized or consolidated in working or long-term memory as a unit, whereas only a small part of an unstructured sequence such as a string of unrelated words or an incoherent picture can be consolidated in the same time period.
Because similar principles seem to apply to language comprehension and to nonlinguistic visual understanding, I have proposed that understanding in both cases is abstractly conceptual rather than fundamentally language-based. For example, pictured objects and their names give equivalent and equally rapid information about meaning (Potter & Faulconer Reference Potter and Faulconer1975; Potter et al. Reference Potter, Valian and Faulconer1977). Other perceptual senses such as audition and touch also have rapid access to the same conceptual level.
If the CSTM hypothesis is correct, then the Now-or-Never bottleneck occurs after a rich set of associations from long-term memory has enabled conceptual chunking of incoming linguistic or visual information. At that point, the information can be passed through the bottleneck to a more abstract level of discourse or scene understanding. Moreover, the severe limitations of working memory seen for arbitrary lists of letters, numbers, or geometric figures are largely overcome when proactive interference from reuse of a small set of stimuli is eliminated (Endress & Potter Reference Endress and Potter2014a). The desperate speed of processing noted by C&C is not due solely to the limitations of short-term memory, but more generally reflects the pressure to think, see, understand, and act as fast as possible, in order to survive in a predatory world.