Branigan & Pickering (B&P) claim that the success of structural priming as a method should “end the current reliance on acceptability judgments.” The basis of such a claim rests on the premise that, not only is structural priming a robust psychological phenomenon, but also can be used to adjudicate between finer points of syntactic theory. While we agree that structural priming is an interesting and useful phenomenon, we have reservations as to whether there is sufficient statistical evidence to support all of the detailed claims made about specific points of syntactic theory.
For example, as a case study of what they call the “clearest example,” B&P discuss what priming studies tell us about passive constructions and, specifically, how priming studies push back against the standard generative grammar accounts of passives. The details of this argument rely on priming studies that go beyond asking whether Structure X in a prime sentence leads to an increased likelihood of a participant producing Structure X in a target sentence. Rather, it relies on evidence that locatives prime passives (Bock & Lobell Reference Bock and Loebell1990), that unergatives prime unaccusatives (Flett Reference Flett2006), and that POs and DOs prime light verbs (Wittenberg Reference Wittenberg2014). Often, there is only one paper that investigates a particular research question, and B&P typically accept the results of that paper as delivering a statistically valid conclusion about the phenomenon in question.
In a meta-analysis in Mahowald et al. (Reference Mahowald, James, Futrell and Gibson2016b), we give evidence that structural priming is a robust and well-replicated phenomenon, but caution that studies of the type that B&P rely on for some of their conclusions (specifically, studies that ask whether some priming effects are greater than others or whether certain factors, such as age or L2 status, moderate priming effects) are statistically underpowered (with an average power of 53%, as determined by a p-curve analysis) and often do not use enough participants to warrant the conclusions drawn. Indeed, in studies of this sort, we recommend using several hundred participants – which almost no existing priming studies do.
While this is not evidence that the conclusions drawn in any one study that B&P referenced are misleading, we should be cautious not to assume that every individual study can be used as a building block in a larger syntactic theory. Rather, we should expect some studies to reach statistically significant conclusions (or fail to reach statistically significant conclusions) based on chance alone – and not just because of the experimental manipulation. This is of particular concern when there is only one unreplicated study on a particular phenomenon.
Of course, none of this is to say that structural priming should not be used alongside and sometimes instead of acceptability judgments (which have their own host of pitfalls). In the often messy world of empirical science, the availability of diverse, orthogonal methods that explore the same research question using different techniques is a feature, not a bug. So, insofar as B&P argue against the hegemony of one particular technique (acceptability judgments) in linguistics research, we agree. Insofar as they argue for the hegemony of a different technique (priming), we urge a healthy skepticism.
You have
Access
Branigan & Pickering (B&P) claim that the success of structural priming as a method should “end the current reliance on acceptability judgments.” The basis of such a claim rests on the premise that, not only is structural priming a robust psychological phenomenon, but also can be used to adjudicate between finer points of syntactic theory. While we agree that structural priming is an interesting and useful phenomenon, we have reservations as to whether there is sufficient statistical evidence to support all of the detailed claims made about specific points of syntactic theory.
For example, as a case study of what they call the “clearest example,” B&P discuss what priming studies tell us about passive constructions and, specifically, how priming studies push back against the standard generative grammar accounts of passives. The details of this argument rely on priming studies that go beyond asking whether Structure X in a prime sentence leads to an increased likelihood of a participant producing Structure X in a target sentence. Rather, it relies on evidence that locatives prime passives (Bock & Lobell Reference Bock and Loebell1990), that unergatives prime unaccusatives (Flett Reference Flett2006), and that POs and DOs prime light verbs (Wittenberg Reference Wittenberg2014). Often, there is only one paper that investigates a particular research question, and B&P typically accept the results of that paper as delivering a statistically valid conclusion about the phenomenon in question.
In a meta-analysis in Mahowald et al. (Reference Mahowald, James, Futrell and Gibson2016b), we give evidence that structural priming is a robust and well-replicated phenomenon, but caution that studies of the type that B&P rely on for some of their conclusions (specifically, studies that ask whether some priming effects are greater than others or whether certain factors, such as age or L2 status, moderate priming effects) are statistically underpowered (with an average power of 53%, as determined by a p-curve analysis) and often do not use enough participants to warrant the conclusions drawn. Indeed, in studies of this sort, we recommend using several hundred participants – which almost no existing priming studies do.
While this is not evidence that the conclusions drawn in any one study that B&P referenced are misleading, we should be cautious not to assume that every individual study can be used as a building block in a larger syntactic theory. Rather, we should expect some studies to reach statistically significant conclusions (or fail to reach statistically significant conclusions) based on chance alone – and not just because of the experimental manipulation. This is of particular concern when there is only one unreplicated study on a particular phenomenon.
Of course, none of this is to say that structural priming should not be used alongside and sometimes instead of acceptability judgments (which have their own host of pitfalls). In the often messy world of empirical science, the availability of diverse, orthogonal methods that explore the same research question using different techniques is a feature, not a bug. So, insofar as B&P argue against the hegemony of one particular technique (acceptability judgments) in linguistics research, we agree. Insofar as they argue for the hegemony of a different technique (priming), we urge a healthy skepticism.