I am in full agreement with Branigan & Pickering (B&P) that an experimental approach to linguistic representation is necessary. Despite the central role of grammaticality judgment in the field of linguistics, it does not suffice. The reason is that, in many cases, this procedure does not obey the same quantitative standards (including the proper statistics) that are state of the art in the rest of cognitive science (cf. Gibson & Fedorenko Reference Gibson and Fedorenko2010). When I made this argument in a panel discussion at a major linguistics conference, the following counterargument was presented in a commentary:
When linguists evaluate contrasts between two (or more) sentence types, they normally run several different examples in their heads, they look for potential confounds, and consult other colleagues (and sometimes naive participants), who evaluate the sentence types in the same fashion. The fact that this whole set of procedures (aka, experiments) is conducted informally does not mean it is not conducted carefully and systematically. (Almeida Reference Almeida2010, in Talking Brains blog debate, June 14, 2010)
Running sentences in your head and consulting a colleague is fine for discovering interesting phenomena and possible explanations (for the context of discovery, anything goes), but it does not suffice as the context of justification. We all are subject to confirmation bias. The fallibility of introspection is equally well known; it is a method that hence has fallen out of grace in psychology a long time ago. Therefore, to justify one's theory, empirical data have to be acquired and analyzed according to the quantitative standards of the other fields of cognitive science. In many circumstances, claims by an expert linguist of the form Sentence A is grammatical and sentence B is ungrammatical will not suffice as a valid empirical data point in support of a specific linguistic theory. That is why I endorse an experimental and quantitative approach to investigations of linguistic representations.
Moreover, I concur with B&P that the experimental results of the many sentence priming studies that they reviewed are in agreement with the Parallel Architecture that Jackendoff (Reference Jackendoff2002; Reference Jackendoff and Hagoortin press) has proposed and for which the Memory Unification and Control (MUC) model is a neurobiological specification (Hagoort Reference Hagoort2005; Reference Hagoort2014; Hagoort & Indefrey Reference Hagoort and Indefrey2014). The MUC model specifies non-overlapping neuronal network contributions for the unification of semantic, syntactic, and phonological information, in accordance with the separate generative capacities for semantics, syntax, and phonology that Jackendoff proposes (Reference Jackendoff2002; Reference Jackendoff and Hagoortin press; Culicover & Jackendoff Reference Culicover and Jackendoff2005).
I believe that, next to the behavioral methods that B&P advocate, it might be helpful to take experimental methods from neurobiology on board as well (cf., de Groot & Hagoort, in press). For instance, a direct correlate of priming in behavior is fMRI adaptation. This is the phenomenon that the blood-oxygen-dependent (BOLD) response in neuronal populations sensitive to a stimulus attribute is suppressed or enhanced when that attribute is repeated (Grill-Spector et al. Reference Grill-Spector, Henson and Martin2006). Using such an fMRI adaptation paradigm during speech comprehension and production, we found a clear segregation for areas involved in lexical processing, syntax, and semantics. However, for each of these domains, the same areas were recruited in speaking and listening (Menenti et al. Reference Menenti, Gierhan, Segaert and Hagoort2011; Segaert et al. Reference Segaert, Menenti, Weber, Petersson and Hagoort2012). This supports the claim that lexical, syntactic, and semantic representations are segregated but shared for production and comprehension. In general, repetition suppression of the BOLD signal can provide insight into the representational features that a particular population of neurons codes for. In recent years, the method of Representational Similarity Analysis has been developed (Kriegeskort et al. Reference Kriegeskorte, Mur and Bandettini2008) to determine how representational information might be encoded in distributed patterns of brain activity.
The toolkit of cognitive neuroscience has expanded in the last few decades (cf., De Groot & Hagoort, Reference De Groot and Hagoortin press). This does not take away the need for linguistic theory and theory-driven questions. However, the context of justification has to meet the standards of the other branches of cognitive science. Clearly, acceptability judgments can no longer be seen as the holy grail for testing the nature of linguistic representations. There is no excuse for leaving the available experimental and quantitative methods unused in this domain of research.
I am in full agreement with Branigan & Pickering (B&P) that an experimental approach to linguistic representation is necessary. Despite the central role of grammaticality judgment in the field of linguistics, it does not suffice. The reason is that, in many cases, this procedure does not obey the same quantitative standards (including the proper statistics) that are state of the art in the rest of cognitive science (cf. Gibson & Fedorenko Reference Gibson and Fedorenko2010). When I made this argument in a panel discussion at a major linguistics conference, the following counterargument was presented in a commentary:
When linguists evaluate contrasts between two (or more) sentence types, they normally run several different examples in their heads, they look for potential confounds, and consult other colleagues (and sometimes naive participants), who evaluate the sentence types in the same fashion. The fact that this whole set of procedures (aka, experiments) is conducted informally does not mean it is not conducted carefully and systematically. (Almeida Reference Almeida2010, in Talking Brains blog debate, June 14, 2010)
Running sentences in your head and consulting a colleague is fine for discovering interesting phenomena and possible explanations (for the context of discovery, anything goes), but it does not suffice as the context of justification. We all are subject to confirmation bias. The fallibility of introspection is equally well known; it is a method that hence has fallen out of grace in psychology a long time ago. Therefore, to justify one's theory, empirical data have to be acquired and analyzed according to the quantitative standards of the other fields of cognitive science. In many circumstances, claims by an expert linguist of the form Sentence A is grammatical and sentence B is ungrammatical will not suffice as a valid empirical data point in support of a specific linguistic theory. That is why I endorse an experimental and quantitative approach to investigations of linguistic representations.
Moreover, I concur with B&P that the experimental results of the many sentence priming studies that they reviewed are in agreement with the Parallel Architecture that Jackendoff (Reference Jackendoff2002; Reference Jackendoff and Hagoortin press) has proposed and for which the Memory Unification and Control (MUC) model is a neurobiological specification (Hagoort Reference Hagoort2005; Reference Hagoort2014; Hagoort & Indefrey Reference Hagoort and Indefrey2014). The MUC model specifies non-overlapping neuronal network contributions for the unification of semantic, syntactic, and phonological information, in accordance with the separate generative capacities for semantics, syntax, and phonology that Jackendoff proposes (Reference Jackendoff2002; Reference Jackendoff and Hagoortin press; Culicover & Jackendoff Reference Culicover and Jackendoff2005).
I believe that, next to the behavioral methods that B&P advocate, it might be helpful to take experimental methods from neurobiology on board as well (cf., de Groot & Hagoort, in press). For instance, a direct correlate of priming in behavior is fMRI adaptation. This is the phenomenon that the blood-oxygen-dependent (BOLD) response in neuronal populations sensitive to a stimulus attribute is suppressed or enhanced when that attribute is repeated (Grill-Spector et al. Reference Grill-Spector, Henson and Martin2006). Using such an fMRI adaptation paradigm during speech comprehension and production, we found a clear segregation for areas involved in lexical processing, syntax, and semantics. However, for each of these domains, the same areas were recruited in speaking and listening (Menenti et al. Reference Menenti, Gierhan, Segaert and Hagoort2011; Segaert et al. Reference Segaert, Menenti, Weber, Petersson and Hagoort2012). This supports the claim that lexical, syntactic, and semantic representations are segregated but shared for production and comprehension. In general, repetition suppression of the BOLD signal can provide insight into the representational features that a particular population of neurons codes for. In recent years, the method of Representational Similarity Analysis has been developed (Kriegeskort et al. Reference Kriegeskorte, Mur and Bandettini2008) to determine how representational information might be encoded in distributed patterns of brain activity.
The toolkit of cognitive neuroscience has expanded in the last few decades (cf., De Groot & Hagoort, Reference De Groot and Hagoortin press). This does not take away the need for linguistic theory and theory-driven questions. However, the context of justification has to meet the standards of the other branches of cognitive science. Clearly, acceptability judgments can no longer be seen as the holy grail for testing the nature of linguistic representations. There is no excuse for leaving the available experimental and quantitative methods unused in this domain of research.