Gestures are integral to communication (Kendon, Reference Kendon2004; McNeill, Reference McNeill2005), and have been found to play a facilitative role in the interpretation of structural ambiguities (e.g., Guellaiï, Langus, & Nespor, Reference Guellaï, Langus and Nespor2014; Prieto, Borràs-Comes, Tubau, & Espinal, Reference Prieto, Borràs-Comes, Tubau and Espinal2013; Tubau, González-Fuente, Prieto, & Espinal, Reference Tubau, González-Fuente, Prieto and Espinal2015). Specific gestural forms have been associated with the expression of negation in speech, with research on Open Hand Prone gestures (Kendon, Reference Kendon2004), head shakes (e.g., Calbris, Reference Calbris2011; Kendon, Reference Kendon2002), and their interaction and synchronization (Harrison, Reference Harrison2014a). The current study focuses on a context of negation in English in which the presence of quantification yields scopal ambiguities (see, e.g., Larrivée, Reference Larrivée, Ziegeler and Zhiming2017; Tottie & Neukom-Hermann, Reference Tottie, Neukom-Hermann and Horn2010).
Example (1) demonstrates how the combination of the quantifier all with the negator not yields two possible interpretations.
(1) All the students won’t go.
In this scenario, a speaker may intend to convey that no students will go. Conversely, the interpretation that some students will go and some will not go is also possible. In prior work on speech production, Syrett, Simon, and Nisula (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a) found that the majority of English speakers tested did not reliably produce the prosodic patterns at the ends of sentences (i.e., falling vs. rising intonational contours, respectively) predicted by Jackendoff (Reference Jackendoff1972) to differentiate their intended interpretation. Thus, in this partial replication of Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a), we examine whether associations exist between the interpretations intended by speakers and the specific gestural forms (e.g., articular used and type of movement) and timings (e.g., position and length of gesture in the utterance) that speakers produce to accompany these types of utterances. Such associations would imply that gestural cues have the potential to play a facilitative role in the interpretation of the scopal ambiguities arising from the interplay of negation and quantification.
BACKGROUND
Gestures and negation
Gestures are defined as “actions that have the feature of manifest deliberate expressiveness” (Kendon, Reference Kendon2004, p. 15), or communicatively relevant movements produced along with ongoing talk. Articulatory features of gestures denote the articulator involved in the movement (e.g., hand or head), its form and movement in space (e.g., open hand wave or head shake), and the components of the movement, labeled as the “gesture phrase” (e.g., the required “stroke phase,” the locus of effortful movement and communicative intent, and the optional “preparation,” “retraction,” “pre-” and “post-stroke hold” phases; Kendon, Reference Kendon and Key1980). Functional features of gestures describe the gesture’s purpose, generally within one of a number of classificatory systems (see Kendon, Reference Kendon2004). In McNeill’s (Reference McNeill1992) widely used system, representational movements encode referential content through iconic gestures (i.e., those that bear physical resemblance to actual entities); metaphoric gestures (i.e., those that symbolically represent abstract content); and deictic gestures (i.e., those that point to present or absent referents). Beat gestures (i.e., short, punctuated, and often repeated movements), align with the suprasegmental features of speech, while emblem gestures are defined as movements with standards of well-formedness that can replace verbal elements entirely and are comprehended by members of a given speech/cultural community (e.g., the “thumbs up” gesture). Relationships between gesture form and function are not absolute; thus, referential pointing can be implemented in a range of articulators (e.g., the index finger, foot, or lips), depending on the context or speech community (Kita, Reference Kita2003). Furthermore, gestures are often multifunctional, for example, an iconic gesture form reflecting conceptual content with a superimposed beat indicating discourse prominence (McNeill, Reference McNeill2005).
Gestures are semantically and temporally coordinated with co-occurring speech (Kendon, Reference Kendon, Siegman and Pope1972; McNeill, Reference McNeill1992; McNeill, Levy, & Pedelty, Reference McNeill, Levy, Pedelty and Hammond1990; Schegloff, Reference Schegloff, Atkinson and Heritage1984). They can repeat or highlight information encoded in speech or add information not present in the surrounding discourse, and speakers typically deploy the stroke phase of a gesture at the most semantically relevant moment, rendering the alignment of gesture and speech communicatively informative. With their label as a “window onto the mind” (McNeill, Reference McNeill1992), gestures can simultaneously provide insights on structural, semantic, and conceptual representations underlying speech as well as contribute in important ways to the structuring of discourse (Kendon, Reference Kendon and Key1980, Reference Kendon2004).
Gestures have been associated with spoken forms that express negation as well as the related concepts of denial, interruption, and stoppage (Kendon, Reference Kendon2004). Two main gestural forms have been identified in such contexts: Open Hand Prone gestures and Head Shakes. Open Hand Prone gestures comprise an open hand and a prone forearm, which can be moved vertically and horizontally, with the palm orientation up or down (Harrison, Reference Harrison, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Kendon, Reference Kendon2004). In such gestures, the preparation stage is particularly important as it enables speakers to begin and align their gesture with the negative particle and other elements associated with negation in speech (Harrison, Reference Harrison2013, Reference Harrison2014a), as demonstrated by the onset of such gestures with ne in the French ne … pas construction (Harrison & Larrivée, Reference Harrison, Larrivée, Larrivée and Lee2016). The gestures are often then held in post-stroke holds for the duration of the “scope” of negation (see below; Harrison, Reference Harrison2010, Reference Harrison2013). Head Shakes involve the head moving horizontally from side to side (e.g., Calbris, Reference Calbris2011; Harrison, Reference Harrison2014a, Reference Harrison, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Kendon, Reference Kendon2002), and have been attested before, during, or after negation in speech (Harrison, Reference Harrison2009), though as emblem gestures, they can also serve in place of speech (Calbris, Reference Calbris2011; Kendon, Reference Kendon2002). Harrison (Reference Harrison2014a) notes that the ways in which speakers coordinate the hand and head gestures associated with negation into “kinesic ensembles” (Calibris, Reference Calbris2011) is underexplored.
Harrison (Reference Harrison2014a) has described the intricate connection between gestures and spoken forms of negation. He argues that gesture ensembles are organized in relation to the so-called node, a negative particle in speech, and to the scope, a pragmatic/semantic domain, specifically the stretch of discourse covered by negation. Example (2) below from Harrison (Reference Harrison2014a, p. 132) illustrates this intertwining of speech and gesture, where numbering indicates the components of the gesture phrase and bold indicates speech aligned with gesture.
(2) Footnote 1I Footnote 2don’t Footnote 3have to pay Footnote 4for that night.
1: Preparation of Open Hand Prone, palm-down gesture
2: Stroke of Open Hand Prone across body (“PD Across”)
3: Post-stroke hold
4: Retraction
In (2), an Open Hand Prone gesture is prepared and then made across the body, which Harrison refers to as “PD Across” (palm down, horizontal, across body gesture), during the production of “don’t.” The gesture is then held through “have to pay,” indicating the scope of negation, and then retracted during the preposition phrase “for that night.”
Negation and quantification: A case of scopal ambiguity
While the contribution of gesture to expression of negation in general has been examined, one environment that warrants further exploration is the interaction between scope bearing items, specifically that between various types of quantification and negation. Such interactions pose a challenge in communication because they yield semantically ambiguous sentences with multiple interpretations, where the scope of negation differs based on the intended interpretation. An example follows in (3) with the negator and quantifier underlined.
(3) All the magnolias won’t bloom (Syrett, Simon, & Nisula, Reference Syrett, Simon and Nisula2014b, p. 466).
The relevant interpretations are captured by the logical representations in (3a) and (3b):
(3a) ∀x [magnolia (x) → ¬bloom (x)]
(3b) ¬∀x [magnolia (x) → bloom (x)]
In (3a), the universal quantifier all, described by Beghelli and Stowell (Reference Beghelli, Stowell and Szabolcsi1997) as a “group-denoting quantifier,” takes scope over verb phrase-level negation (all>not; see Syrett et al., Reference Syrett, Simon and Nisula2014b, p. 454). According to Jackendoff (Reference Jackendoff1972, pp. 356–357), in reading (3a) (“none of the magnolias”), negation is associated with the presupposition as shown in (3a’):
(3a’) Presupposition: λQ [Q of the magnolias won’t bloom] is well-formed/under discussion
Assertion: all ∈ λQ [Q of the magnolias won’t bloom]
In (3b), in contrast, negation is associated with the focused quantifier all and takes wide scope over the universal quantifier all (Horn, Reference Horn2001, p. 226; Syrett et al., Reference Syrett, Simon and Nisula2014b, p. 456) with the interpretation as in (3b) (not>all; “not all the magnolias”; see Jackendoff, Reference Jackendoff1972, pp. 356–357; Syrett et al., Reference Syrett, Simon and Nisula2014b, p. 454), as shown in (3b’):
(3b’) Presupposition: λQ [Q of the magnolias will bloom] is well-formed/under discussion
Assertion: all ∉ λQ [Q of the magnolias will bloom]
(see Syrett et al., Reference Syrett, Simon and Nisula2014b, pp. 456–457)
In (3), all is in the subject position, which has been argued to intensify the ambiguity in comparison to its placement in the object position, which would favor an interpretation of negation with wide scope (not>all), for example, I didn’t see ALL of the men (Jackendoff, Reference Jackendoff1972, p. 357), an asymmetry that has been further discussed by Marsden (Reference Marsden2009), who notes that “a property of English all is that it can take scope distributively if it is in subject position … but not in object position” (p. 138).
As noted by Jackendoff (Reference Jackendoff1972), the quantifiers “many” or “most,” described by Beghelli and Stowell (Reference Beghelli, Stowell and Szabolcsi1997) as “counting” or “proportional quantifiers,” behave similarly when combined with negation as in the following examples.
(4) Liam doesn’t know many alumni. (Syrett et al., Reference Syrett, Simon and Nisula2014b, p. 466).
Again, where negation is associated with the presupposition, the negator has narrow scope (i.e., many>not), which yields the interpretation as in (4a). Conversely, where negation is associated with focus or assertion, the negator has wide scope (i.e., not>many), yielding the interpretation as in (4b).
(4a) There are many alumni that Liam does not know. / Or Liam knows few alumni.
(4b) There are not many alumni that Liam knows. / Or Liam only knows a few alumni.
Following Jackendoff (Reference Jackendoff1972) and Syrett et al. (Reference Syrett, Simon and Nisula2014b), (4a) and (4b) are described below:
(4a’) Presupposition: λQ [Liam doesn’t know Q of alumni] is well- formed/under discussion
Assertion: many ∈ λQ [Liam doesn’t know Q of alumni]
(4b’) Presupposition: λQ [Liam knows Q of alumni] is well-formed/under discussion
Assertion: many ∉ λQ [Liam knows Q of alumni]
Kadmon and Roberts (Reference Kadmon, Roberts, Farley, Farley and McCullough1986) provide an example with most, shown in (5) with the interpretations as in (5a) and (5b):
(5) He doesn’t hate most of the songs.
(5a) Most of the songs are songs that he doesn’t hate. (“most>not” reading)
(5b) It is not the case that he hates most of the songs. (“not>most” reading) (Kadmon & Roberts 1986, p. 16)
In (5a), the presupposition contains the negation and asserts that the focus, most, is the proper portion; hence, most takes wide scope over the negation (Kadmon & Roberts, 1986, p. 17), following Jackendoff’s (Reference Jackendoff1972) notation formally:
(5a’) Presupposition: λQ [he doesn’t hate Q of the songs] is well-formed/under discussion
Assertion: most ∈ λQ [he doesn’t hate Q of the songs]
In contrast, in (5b), the presupposition does not contain the negation and asserts that most is not the proper proportion (Kadmon & Roberts, 1986, p. 17). Formally:
(5b’) Presupposition: λQ [he hates Q of the songs] is well-formed/under discussion
Assertion: most ∉ λQ [he hates Q of the songs]
In (4) and (5), the placement of the quantifier in the object position elicits a more ambiguous reading than if it were in the subject position, which would more strongly favor a many/most>not interpretation (e.g., Many/most students didn’t go).
Along with context, intonation has been argued to help serve as a disambiguating cue for speakers and hearers in instances of structural ambiguities (Nespor & Vogel, Reference Nespor and Vogel2007). In the case of the scopal interaction between negation and quantification, Jackendoff (Reference Jackendoff1972) proposed that when negation is associated with the presupposition (e.g., all>not), speakers may employ a sentence-final falling intonational contour (A Accent; Bollinger, Reference Bolinger, Abe and Kanekiyo1965) in the representation of (3a’). In contrast, where negation is associated with the focus of the utterance (e.g., not>all), speakers may employ a sentence-final rising (or fall-rise) intonational contour in the representation of (3b’; B Accent; Bollinger, Reference Bolinger, Abe and Kanekiyo1965). Syrett et al. (Reference Syrett, Simon and Nisula2014b) discuss how such prosodic cues are reliable in languages such as German, but that mixed empirical findings exist for English, in part perhaps because of methodological issues such as a lack of context presented with an ambiguous sentence.
To illustrate the lack of consistency in empirical findings for English, on the one hand, Syrett et al.’s (Reference Syrett, Simon and Nisula2014b) perception experiments demonstrated that when hearers were presented with utterances containing the prosodic cues described above for the disambiguation of scopally ambiguous sentences (i.e., sentence-final falling intonation for, e.g., all>not sentences, and sentence-final rising intonation for, e.g., not>all sentences), they could (a) interpret the intended meaning with a relatively high degree of success (above 63% of the time depending on sentence type, which was significantly above chance), and (b) select an appropriately produced response based on a given context (above 70% of the time depending on sentence type, which was significantly above chance). On the other hand, Syrett et al.’s (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a) initial production experiment revealed that such prosodic cues are rather unreliably produced by speakers in a seminaturalistic context (i.e., when reading scripted ambiguous sentences embedded within disambiguating paragraph-length texts). Specifically, in the production study, speakers tended to render the majority of negation plus quantification utterances with falling intonational contours. Syrett et al. (Reference Syrett, Simon and Nisula2014b) concluded that, for language production at least, there was no one-to-one mapping between intonation contour and scopal interpretation and thus that intonation patterning was considered neither a “necessary nor sufficient condition” for the disambiguation of scopally ambiguous sentences (p. 461). In further analyses, they did find, however, that the duration of the final word, which was neither the negator nor the quantifier, was potentially a cue more often used by speakers for disambiguation. Specifically, with narrow-scope negation (i.e., all/many/most>not), the final word in the sentence tended to be lengthened by speakers, in comparison to sentences produced in contexts when negation had wide scope (i.e., not>all/many/most). Nonetheless, the researchers noted considerable variability in production by speaker and item, and overall, only 3 of the 19 participants in Syrett et al.’s (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a) production study produced prosodic cues for disambiguation reliably enough for their renderings to be used as stimuli in Syrett et al.’s (Reference Syrett, Simon and Nisula2014b) subsequent perception study.
Gesture in the context of structural ambiguities
Gestures can be used to clarify meaning. In language acquisition, for example, children employ gestures to help specify a referent that they may underspecify in their accompanying speech, for example, in the case of unlicensed null arguments (So, Demir, & Goldin-Meadow, Reference So, Demir and Goldin-Meadow2010; see also So, Lim, & Tan, Reference So, Lim and Tan2014). There is also evidence that gesture is used as a cue in the context of structural ambiguities among adults, though research has generally focused on language perception as opposed to production. Guellaï et al. (Reference Guellaï, Langus and Nespor2014), for example, examined spoken prosody with and without gestures in the context of ambiguous sentences in Italian involving features such as pronominal reference. In constructing the stimuli for the study, native speakers read target sentences and were specifically instructed to produce accompanying gestures as naturally as possible. Study participants were then provided with those renderings of ambiguous sentences in one of three conditions: audio alone, audio with congruent matching gestures, or audio with incongruent mismatching gestures, where the gestures from one sentence interpretation were overlaid onto the audio of the alternative sentence interpretation. Statistically comparable levels of accuracy in interpretation were generated from the audio alone (84%) and audio with matching gestures (79%) conditions, but in the audio with mismatching gestures condition, accuracy was significantly lower (69%), with participants increasingly likely to choose the sentence interpretation indicated by the gesture. On the basis of this finding, the researchers concluded that listeners use gestures to help interpret the meaning of ambiguous sentences, though when prosodic cues are strong, the presence of gestures may impair interpretation. As gesture production per se was not a part of Guellaï et al.’s research question, the gestures produced by speakers in construction of their stimuli were not separately analyzed.
Related to the expression of negation, Tubau et al. (Reference Tubau, González-Fuente, Prieto and Espinal2015) examined the impact of prosodic and gestural cues in the interpretation of yes answers to negative yes/no questions in Catalan. In the sentence Isn’t John coming? a positive answer of yes could indicate confirmation (i.e., that John is not coming), or contradiction (i.e., that John is coming). An initial experiment tested the interpretation of such yes/no responses in written stimuli, and confirmed that readers were unsure of the intended meaning specifically of yes responses to negative questions. The second experiment began with the construction of audiovisual stimuli. For this, four Catalan speakers were asked to render yes/no responses to negative questions within disambiguating contexts. Unlike Guellaï et al. (Reference Guellaï, Langus and Nespor2014), speakers were not specifically instructed to gesture, but asked to “sound and act in a natural way” (p. 125). In examining productions, the researchers found that when speakers expressed a confirmatory yes, they always produced slight or moderate head nods (100% of cases) and sometimes slight eyebrow raising (50% of cases). In contrast, when speakers expressed a contradictory yes, gestures were more pronounced, with “more intense nodding” (100% of cases) and eyebrow raising (50%), wide hand/arm movements (100%), and shoulder shrugs (75%; p. 126). In the subsequent perception portion of the second experiment, representative renderings of the yes/no responses to positive and negative questions were played for study participants first in an audio-only condition and then in an audiovisual condition. Results revealed less uncertainty by listeners for yes responses to negative questions in the audiovisual context (i.e., when they both heard the prosodic marking and saw the accompanying gestures). The researchers concluded that “gesture enhances the effect of intonation in the interpretation of yes-answers to negative yes/no questions” (pp. 130–131). A related comparison of production data from Catalan and Russian in this same syntactic area (González-Fuente, Tubau, Espinal, & Prieto, Reference González-Fuente, Tubau, Espinal and Prieto2015) revealed different lexicosyntactic and prosodic strategies cross-linguistically, but “common denial gestures associated with the notion of reject … in both languages” (p 12).
Prieto et al. (Reference Prieto, Borràs-Comes, Tubau and Espinal2013) used a similar research design in their investigation of interpretations of the double negative construction in Spanish and Catalan. Again, a small group of native speakers created stimuli for a later comprehension task by providing responses to scripted contexts. In their rendering of a total of 32 instances of the target negative word (Catalan ningú and Spanish nadie, “nobody”), 8 speakers produced 75 gestures across the two languages. The authors reported descriptions of individual examples of speech-gesture combinations as well as raw frequencies of gesture types, finding that shoulder shrugging was limited to contexts of double negative readings, head shakes were found in both negative and double negative readings, but head nods were more common with the double negative. Regarding hand gestures, a “palms down across” movement was associated only with negative readings, while a “palms up across” movement was more common with double negatives and a “palm open toward center movement” was restricted to a double negative reading. A subset of the gestures considered representative were subsequently shown to 60 participants across both languages in auditory-only, visual-only, and audiovisual conditions, the latter crossed with congruent and incongruent prosody-gesture matches, and participants indicated their interpretations (single or double negation). In both the auditory-only and visual-only conditions, prosodic and gestural patterns associated with double negative readings elicited accurate interpretations from participants in both languages, indicating that both prosody and gesture can function as critical and independent cues to comprehension cross-linguistically. In the audiovisual condition, in line with Guellaï et al. (Reference Guellaï, Langus and Nespor2014), only the congruent prosody-gesture matches triggered accurate interpretations, supporting the multimodal nature of language comprehension (though in addition to Guellaï et al., Reference Guellaï, Langus and Nespor2014, also see Borràs-Comes & Prieto, Reference Borràs-Comes and Prieto2011, for evidence of the supremacy of gesture when visual and auditory cues clash).
CURRENT STUDY
A review of the literature pertinent to this study supports the existence of relationships between the expression of negation in speech and the articulatory and functional properties of gesture such that Open Hand Prone gestures and Head Shakes are commonly produced simultaneously with words such as no, not, and nothing and can serve emblematically in place of speech (Calbris, Reference Calbris2011; Harrison, Reference Harrison2009, Reference Harrison2010, Reference Harrison2013, Reference Harrison2014a, Reference Harrison, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Harrison & Larrivée, Reference Harrison, Larrivée, Larrivée and Lee2016; Kendon, Reference Kendon2002, Reference Kendon2004). In addition, work on language production, though generally from relatively few participants for the purpose of creating stimulus materials for comprehension tests, has described gesture patterns associated with specific readings in contexts of structural or semantic/pragmatic ambiguity (Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013; Tubau et al., Reference Tubau, González-Fuente, Prieto and Espinal2015), as well as gesture strategies that are universal even in the context of lexicosyntactic and prosodic differences across languages (González-Fuente et al., Reference González-Fuente, Tubau, Espinal and Prieto2015). In the larger body of research on language comprehension, gestures congruent with prosody have generally been found to be facilitative to interpretation (Guellaï et al., Reference Guellaï, Langus and Nespor2014; Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013; Tubau et al., Reference Tubau, González-Fuente, Prieto and Espinal2015).
In the domain of scopal ambiguities arising from the interaction between negation and quantification, empirical findings on the production of prosodic patterning are mixed (Syrett et al., Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a). In other words, speakers do not reliably produce specific prosody when attempting to convey particular interpretations of sentences involving negation and quantification. Therefore, this exploratory study investigated possible associations between gestural forms, lengths, and timings and interpretations intended by speakers in contexts of scopal ambiguity, specifically in interactions between the quantifiers all, most, and many and the negator not, using a data set of language production large enough to employ inferential statistical analyses.
METHOD
Participants
A total of 28 native English-speaking adults were recruited for participation in the study. Of those, 3 participants did not produce any gestures at all; thus, data from 25 gesturing participants (10 males, 15 females) were analyzed.
Stimuli
Stimuli published in Syrett at al. (Reference Syrett, Simon and Nisula2014b) to investigate prosodic patterns were repurposed in this study for examining gesture patterns. These contained 15 sentences with various types of structural ambiguities. Sentences were preceded by two possible contexts, each of which forced a different interpretation of the ambiguous sentence. The 5 sentences targeted for analysis contained negation and a quantifier, and the remaining 10 sentences served as distractors. Three sentences employed the group-denoting quantifier “all” along with negation, while two of the sentences employed the counting/proportional quantifiers “many” or “most” with negation. The target and distractors sentences are presented in (6) and (7).
(6) Target quantifier + negation sentences (Syrett at al., Reference Syrett, Simon and Nisula2014b):
All the magnolias won’t bloom.
All the moms didn’t allow eyeliner.
All the wool lining wasn’t worn.
Liam doesn’t know many alumni.
Neil doesn’t enjoy most musicals.
(7) Distractor sentences (Syrett et al., Reference Syrett, Simon and Nisula2014b):
Mary admires Arianna, but she doesn’t like her.
Alan punched Owen, and then he kicked him.
Ryan passed Nolan, and then he drove off the road.
Larry only elbowed Riley.
Mary only ran one mile.
They’re not late because of his driving.
Georgia isn’t singing because she’s preparing for an audition.
Omar isn’t in shape because he runs outdoors.
She even composts her newspapers.
She even painted the garage.
Procedure
In the first stage of the procedure, participants were informed that the study was about confusing sentences and told that they would read some sentences that were ambiguous, purportedly in order to construct a comprehension test for second language learners of English.
Following procedures in Syrett at al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a), participants were initially provided with all 15 sentences, each with two associated, disambiguating contexts (total 30 contexts). The pair of contexts associated with a given ambiguous sentence were first presented side by side, followed by the ambiguous sentence, and finally by a pair of comprehension questions. The comprehension questions asked either about the intended interpretation of a referent or for selection of the most appropriate sentence to follow naturally from the context and ambiguous sentence. These questions were designed by Syrett at al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a) to ensure that the participants understood each of the possible interpretations of the ambiguous sentences. Examples of the initial stimulus presentation for each of two types of negation plus quantification sentences along with a distractor sentence are shown in (8)–(10) with appropriate answers to the comprehension questions indicated in bold. The full stimulus set can be found in Syrett et al. (Reference Syrett, Simon and Nisula2014b).
(8) Target sentence with negation + Group-Denoting Quantifier (Beghelli & Stowell, Reference Beghelli, Stowell and Szabolcsi1997) “all” (Syrett at al., Reference Syrett, Simon and Nisula2014b, p. 482).
(9) Target sentence with negation + Counting quantifiers (Beghelli & Stowell, Reference Beghelli, Stowell and Szabolcsi1997) “many/most” (Syrett at al., Reference Syrett, Simon and Nisula2014b, p. 483).
(10) Distractor sentence with ambiguous pronominal reference (Syrett at al., Reference Syrett, Simon and Nisula2014b, p. 488).
All study participants indicated their understanding of the target sentence interpretations by selecting the appropriate response to the comprehension questions; thus, all participants proceeded to the second stage of the procedure. With a planned focus on spontaneous gestures in the context of a tightly controlled experimental design, the second stage of the procedure posed the largest methodological challenge. In the original Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a) study, the elicitation stimuli were used to examine prosody in language production; thus, participants in that study were asked to read the target sentences aloud as expressively as possible. They were also asked to read the preceding contexts aloud to ensure a smooth and natural delivery of the target sentences, though individual variability was still high despite the scripted reading. However, the same procedure could not be extended to this study because speakers generally do not produce spontaneous gestures while reading aloud unless they have had considerable exposure to a text almost to the point of memorization, and it was not feasible to require this level of familiarity with 15 syntactically ambiguous sentences and 30 associated contexts from our participants. Yet at the same time, complete accuracy in production of the target sentences by speakers was needed in order to allow investigation of the gestural patterns associated with specific interpretations of ambiguous sentences.
After extensive piloting, the most effective elicitation method entailed participants first reviewing each context silently, then memorizing the ambiguous sentence, and finally delivering only the ambiguous sentence to a video camera “as expressively as possible to convey the intended interpretation,” following Prieto et al. (Reference Prieto, Borràs-Comes, Tubau and Espinal2013) and Tubau et al. (Reference Tubau, González-Fuente, Prieto and Espinal2015). Given the purported context of creation of a comprehension test for second language speakers and thus the absence of a real interlocutor (participants were aware that the experimenter was only there to issue instructions and operate the recording equipment), this design was considered the most naturalistic design possible within the experimental constraints (i.e., delivery of a specific scripted utterance without active reading, e.g., from paper or a teleprompter). Further, in order to increase the naturalness and spontaneity of the gestures produced and to minimize participants’ attention to their own gestures, participants were not specifically instructed to gesture while delivering the ambiguous sentences (cf. Guellaï et al., Reference Guellaï, Langus and Nespor2014).
Two sequences of stimuli were constructed, and the order of presentation was counterbalanced across participants. Participants first delivered each ambiguous sentence associated with one context, then repeated the entire set based on the second context. Participants were assured that repetition of the recordings of target sentences was perfectly acceptable during production of the “comprehension test,” and repetitions were given from a total of nine participants in a total of 18 target sentence renditions, either at the request of the experimenter because of some kind of error in the initial rendition, such as deviations from the scripted sentence or a disfluency, or prompted by the participant because of dissatisfaction with a prior rendition. In both cases, only the rendition that was free of errors and accepted by the participant as his or her final version was submitted for analysis.
Coding and analyses
Participant renditions of each of the five target sentences containing negation (not) and quantification (all, many/most) were examined for the production of a co-occurring gesture. The digital video tagging software program, ELAN (Wittenburg, Brugman, Russel, Klassmann, & Sloetjes, Reference Wittenburg, Brugman, Russel, Klassmann and Sloetjes2006), which enables a frame-by-frame analysis (at 40-ms intervals) of movement and sound, was used to code the gestures. Following the literature on gestures associated with negation (e.g., Calbris, Reference Calbris2011; Harrison, Reference Harrison2014a; Kendon, Reference Kendon2002, Reference Kendon2004), each gesture was classified by articulator (head, hand/arm, or other), form (e.g., beat, shake, or other), length (measured in the number of overlapping words, with verb+negator contractions classified as one word), overlap with the quantifier, and finally overlap with the negator. Representative examples of these coding processes are shown in (11) and (12) with the direction of gestures indicated by arrows and speech overlapping the gesture shown in bold.
(11) Participant rendition of “Neil doesn’t enjoy most musicals” with narrow-scope negation (most>not) reading
Neil doesn’t enjoy most musicals.
Gesture articulator: head
Gesture form: shake
Gesture length: 5 words
Quantifier overlap: Yes
Negator overlap: Yes
(12) Participant rendition of “All the magnolias won’t bloom” with narrow-scope negation (all>not) reading
All the magnolias won’t bloom.
Gesture articulator: hands
Gesture form: other (palm up, lifting, outward)
Gesture length: 3 words
Quantifier overlap: Yes
Negator overlap: No
In this scripted, experimental context, the gestures observed tended to be rather subtle in general and, if the hands were involved (see the analysis in the Gesture Articulator section below), fairly low in the gesture space in comparison to the gestures produced in other more naturalistic contexts (e.g., spontaneous narratives); thus, two forms of coder reliability testing were conducted to assess the replicability of the coding scheme. First, in intra-coder reliability testing, the same coder coded the entire data set on two separate occasions separated by a period of several months. A total of 263 gestures were identified in the corpus during the initial round of coding, while 317 gestures were identified during the second round of coding, representing 83% agreement on the identification of gestures. The discrepancy in gesture identification may be understood in the context of the subtlety of the gestures in the corpus and the associated benefit of preliminarily coding the entire data set before conducting a second round of coding. Of the 263 gestures identified in the first and second rounds of coding, 100% agreement was reached on the articulator used, 100% agreement was reached on the gestural form, and 95% agreement was reached on the length of the gesture in number of words. Second, in inter-coder reliability testing, a new coder coded 20% of the data set (i.e., 5 participants). Of the 65 gestures identified by the first coder, 54 gestures were identified by the second coder, also representing 83% agreement on the identification of gestures. Of the 54 gestures identified by both coders, 100% agreement was reached on the articulator used, and 100% agreement was reached on the gestural form. However, 56% agreement was reached on the length of the gesture in number of words. In cases of disagreement, the second coder generally determined that the gesture was shorter, overlapping with fewer words, than that determined by the first coder. In the vast majority of such instances, the second coder treated gestures with repeated components (e.g., repeated head nods or beat gestures), as separate gesture strokes, which reflects a general area of methodological challenge in gesture work. Thus, overall, the coding scheme was determined to be reliable relative to the field, with some difficulty posed in assessing the length of gestures containing repeated components such as beats or shakes. Decisions made by the original coder in the second round of coding were adopted for analysis.
Finally, with respect to analysis, sentence type constituted a categorical independent variable with several levels. Target sentences were organized according to scopal interaction of the quantifier and negator, yielding four sentence types:
all>not – narrow-scope negation, e.g., All the magnolias won’t bloom = The number of magnolias that did not bloom is all / No magnolias bloomed.
not>all – wide-scope negation, e.g., All the magnolias won’t bloom = The number of magnolias that bloomed is not all / Some magnolias bloomed.
many/most>not - narrow-scope negation, e.g., Liam doesn’t know many alumni = The number of alumni that Liam does not know is many / Few alumni are known.
not>many/most - wide-scope negation, e.g., Liam doesn’t know many alumni = The number of alumni that Liam knows is not many / A few alumni are known.
A variety of gesture features served as the dependent variables. Gesture articulator, gesture form, gesture overlap with quantifier, and gesture overlap with negator were analyzed as binary, nominal dependent variables. Gesture length was classified as a continuous dependent variable. The data points were not considered to be independent as each speaker could produce varying numbers of gestures and the production of one gesture (e.g., a head gesture), could affect the production of another (e.g., a second gesture produced simultaneously which could then potentially be a hand gesture but crucially not a head gesture). In addition, as there were more items containing all than many/most, it was not clear that the items would behave similarly. Thus, participant and item were added to the analyses as random variables (Raaijmakers, Schrijnemakers, & Gremmen, Reference Raaijmakers, Schrijnemakers and Gremmen1999). For the categorical variables, a series of binomial mixed-effects logistic regression analyses were conducted to test for associations between nominal features of gestures and sentence type. For the continuous variable, a mixed-effects linear regression analysis examined the relationship between the numerical measure of gesture length and sentence type.
RESULTS
Gesture articulator
Figure 1 displays the percentage of all gestures using different articulators by sentence type.Footnote 5 Figure 1 indicates that gestures were distributed across sentence types in the corpus. Not>all sentences, where negation has a wide scope, attracted the highest percentage of gestures (32%), while many/most>not sentences, in which negation has a narrow scope, attracted the lowest percentage of gestures (18%). Further, descriptively more head gestures than hand/arm gestures were produced for all sentence types. A binomial probability analysis revealed that the frequency of observed head gestures was significantly greater than chance (p < .001). A further logistic regression with four levels of the independent variable showed partial significant relationships (with all>not as the intercept, p < .001; many/most>not, p = .08; not>all, p = .44; not>many/most, p = .92). Thus, head gestures were produced significantly more often than hand gestures with utterances combining quantifiers and negators, and were to some extent associated particularly with contexts of narrow-scope negation. Output from these analyses is displayed in Appendix A.
Figure 1. Gesture articulator by sentence type.
Gesture form
The second analysis examined gesture forms. With the exception of one gesture, all shakes employed the head as the articulator, while beats were enacted in both the head and the hands. Figure 2 displays the percentage of gestures of different forms by sentence type. Figure 2 indicates again that all sentence types elicited different gestural forms: shakes, beats, and other movements. For three out of four interpretation types (all>not, not>all, and not>many/most), beat gestures were descriptively the most common, followed by shakes and then other movements. For many/most>not items, shakes were descriptively the most common, followed by beats and other movements. In the statistical analyses, as “other” movements were relatively infrequent (less than 10% of the total number of gestures), two separate binary, mixed-effects logistic regressions were conducted: one examining shakes versus nonshakes (collapsing beats and other movements) and the other examining beats versus nonbeats (collapsing shakes and other movements). In an analysis with many/most>not coded as the reference variable, significant negative relationships were found between the production of shakes and all>not (p < .001), not>many/most (p < .001), and not>all (p < .001) interpretations. With all>not coded as the reference variable, a significant positive relationship was found between the production of shakes and many/most>not interpretations (p < .001). The results from the analysis of shakes was confirmed by the analysis of beat gestures. With many/most>not coded as the reference variable, significant negative relationships were found between the production of nonbeat gesturesFootnote 6 (the majority of which were shakes) and all>not (p < .001), not>many/most (p < .001), and many/most>not (p < .001) interpretations, while a significant positive relationship was found between the production of nonbeat gestures (again, the majority of which were shakes) and the intercept (many/most>not, p < .001). Thus, the statistical analyses supported the descriptive findings such that beats were statistically more common than other gestures with all>not, not>all, and not>many/most interpretations, while shakes were statistically more common than other gestures in many/most>not interpretations. Output from these analyses is displayed in Appendix B.
Figure 2. Gesture form by sentence type.
Gesture alignment with negator
Figure 3 displays the percentage of gesture strokes whose alignment in the sentence included the negator not. In this analysis, the gestures were of all types (e.g., head, hand, beat, shake, other, etc.). In addition, in the case of the group-denoting all items, the negator followed the universal quantifier, appearing toward the end of the sentence, while for the counting many/most items, the negator preceded the quantifier, appearing toward the middle of the sentence. In the analysis of gesture stroke alignment, gestures produced in sentences with narrow-scope negation interpretations (all>not and many/most>not) were descriptively more often aligned with speech content that included the negator not than were gesture strokes produced in sentences with wide-scope negation interpretations (not>all and not>many/most), where gestures were more often aligned with content in the target sentences that did not include the negator (e.g., the quantifier; see Gesture Alignment With Quantifier in the following section). The binary mixed-effects logistic regression with all>not set as the reference variable partially supported these descriptive findings. A statistically negative relationship was found in the alignment of gestures between the intercept, all>not items, and speech content without the negatorFootnote 7 (p < .01) and a statistically positive relationship between (not>many/most items) and speech content without the negator (p < .01). In other words, in narrow-scope negation with all, the gesture was more likely to be aligned with the negator than in wide-scope negation with many/most. Output from this analysis is shown in Appendix C.
Figure 3. Alignment of gesture with negator not.
Gesture alignment with quantifier
Figure 4 displays the percentage of all gesture types whose stroke alignment in the sentence included the quantifier all, many, or most. As above, in the case of the group-denoting all items, the quantifier preceded the negator, appearing at the start of the sentence, while for the counting many/most items, the quantifier followed the negator, appearing toward the end of the sentence. As Figure 4 demonstrates, for three of the item types, there was relatively little difference in whether the gesture stroke aligned with the quantifier, though slightly more gestures spanned text without the quantifier. The exception was not>many/most items, in which gestures aligned more often with the quantifier than without. The binary mixed-effects logistic regression with all>not set as the reference variable supported these descriptive findings. A statistically positive relationship was found in the alignment of gesture strokes between not>most/many items and speech content with the quantifier (p = .01).Footnote 8 No other statistical relationships were found. Output from this analysis is shown in Appendix D.
Figure 4. Alignment of gesture with quantifier all, many, or most.
Gesture length
Finally, Figure 5 displays the mean length of gesture strokes, as measured in number of overlapping words, by sentence type. Figure 5 demonstrates that gestures associated with narrow-scope negation sentences (all>not and many/most>not) were slightly longer than gestures associated with wide-scope negation sentences (not>all and not>many/most), though the data were rather variable as indicated by the error bars. A mixed-effects linear regression, log-transformed for better model fit, with all>not set as the reference variable, supported the descriptive results showing a significant positive relationship between the intercept (all>not items) and the length of the gesture (p < .001), a significant negative relationship between not>all items and the length of the gesture (p = .029), and a marginally significant negative relationship between not>many/most items and the length of the gesture (p = .07). Thus, gestures produced for sentences with wide-scope negation interpretations tended to be significantly shorter than gestures produced for sentences with narrow-scope negation interpretations. Output from this analysis is shown in Appendix E.
Figure 5. Mean gesture length by sentence type.
Representative example gestures
Examples of the various gestural features follow in Figures 6 and 7, all from the same participant. The left pane in Figure 6 illustrates the frequency of head gestures and the longer head shake gesture produced when the speaker communicated a sentence involving narrow-scope negation, implying that no moms allowed eyeliner. In contrast, the shorter head shake in the right pane was associated with wide-scope negation, implying that some moms allowed eyeliner, while others did not. For both of these examples, the gesture stroke overlapped with the negator though not the quantifier.
Figure 6. Participant production of head shakes with an all+not item.
Similarly, the left pane in Figure 7 again shows a head gesture: a longer head shake accompanied production of a sentence involving narrow-scope negation, implying that The number of musicals that Neil does not like are many / Neil enjoys few musicals. Here, the gesture stroke overlapped with both the quantifier and the negator. The shorter head shake in the right pane accompanied production of a sentence involving wide-scope negation, implying that The number of musicals that Neil likes are not many / Neil enjoys a few musicals. In this case, the gesture stroke only overlapped with the quantifier and not the negator.
Figure 7. Participant production of head shakes in a most+not item.
Discussion
This study investigated associations between gestural forms and timings and the interpretations intended by speakers in sentences involving scopal interactions between negation and quantification, specifically in contexts of narrow-scope negation (all>not and many/most>not) versus wide-scope negation (not>all and not>many/most). In analyses of speaker production of five target sentences, each including either all+not or many/most+not with two possible interpretations distinguished by an accompanying disambiguating context (Syrett et al., Reference Syrett, Simon and Nisula2014b), results revealed a number of gestural features that associated with one of the two scopal interpretations.
First, speakers produced significantly more head than hand gestures, especially for sentences involving narrow-scope negation. While head gestures, specifically head shakes, have been associated with the expression of negation (Calbris, Reference Calbris2011; Harrison, Reference Harrison2014a, Reference Harrison, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Kendon, Reference Kendon2002; Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013), Open Hand Prone gestures have also been associated with negation (Harrison, Reference Harrison2014a, Reference Harrison, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Kendon, Reference Kendon2004; Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013), yet they constituted a rather small percentage of gestures in this data set. While head gestures were also more common than hand gestures in Prieto et al. (Reference Prieto, Borràs-Comes, Tubau and Espinal2013), their marked preponderance here may have been an artifact of the experimental setup, which required speakers to deliver scripted sentences and may have yielded more subtle head gestures than would have been elicited in more naturalistic data elicitation procedures or in deliveries of one-word responses (e.g., Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013).
In the second analysis, significantly more head shakes as opposed to beats and other types of gestures were observed when speakers delivered many/most>not sentences. This statistical pattern did not extend to sentences with all+not; however, one may reconcile the apparent difference in two possible ways. Differences exist in the interpretations of sentences containing all versus many/most with negation. In all>not interpretations, negation is at the verb phrase level, and none of the discourse entities mentioned in the sentence have the property (e.g., none of the magnolias bloomed). In not>all interpretations, negation is propositional, and the quantity of discourse entities mentioned in the sentence that have the property is not all (e.g., not all of the magnolias bloomed). With many/most sentences, regardless of scopal relation, the quantity does not vary; what varies is the focus on the quantity. For example, in (7), the number of alumni that Liam knows is always small. With many>not interpretations, emphasis is placed on the number of alumni that Liam does not know (many). However, with not>many interpretations, emphasis is placed on how many alumni he does know, which is a few (see Baltazani, Reference Baltazani, Andronis, Debenport, Pycha and Yoshimura2002). We may infer that in many/most>not contexts, speakers may try to emphasize the number of entities without the attribute (i.e., many alumni not known, most musicals not enjoyed) through the use of a semantic as opposed to prosodic gesture, specifically an emblem gesture that expresses negation. Further, Syrett et al. (Reference Syrett, Simon and Nisula2014b) explain their findings for the more accurate perceptions of prosody in many/most sentences in comparison to all sentences by appealing to their different syntactic positioning relative to the negator, proposing that for many/most sentences, “when a hearer is processing the sentence incrementally, they may have accumulated enough relevant information in parsing the sentence before they hit the quantifier … later in the sentence that they are in a better position to integrate the information and access the correct interpretation” (p. 478). While our speakers had been familiarized with the target sentences and were not naively processing incrementally, if they did have a clearer understanding of the narrow scope of negation in many/most>not contexts, they may have been better able to communicate emphasis on the large quantity of items that did not display the attribute.
The subsequent analysis of the alignment of gestures with the negative particle not in speech sheds further light on the issue of gestural highlighting. In not>many/most sentences, gestures were less likely to be aligned with content including the negator than in many/most>not sentences, where gestures were likely to be associated with the negative particle. While statistically significant relationships were not found for all+not items, the direction of relationships was consistent with those for many/most+not items. These results support the analysis of head shakes above, suggesting further that speakers emphasize narrow-scope negation by deploying their gesture strokes to coincide with the most semantically relevant part of the utterance (i.e., the negative particle). They are also in line with general claims regarding the semantic and temporal alignment of gestures with speech (Kendon, Reference Kendon, Siegman and Pope1972; McNeill, Reference McNeill1992; McNeill et al., Reference McNeill, Levy, Pedelty and Hammond1990; Schegloff, Reference Schegloff, Atkinson and Heritage1984) and Harrison’s (Reference Harrison2010) assertion that the “node” of negation attracts the deployment of a gesture. To support this conclusion, a descriptive post hoc analysis of gesture stroke alignment with negation examined the percentage of shakes versus beats aligned with not. These results are displayed in Tables 1 and 2.
Table 1 demonstrates that the vast majority of shakes, all but one of which were head shakes, were aligned more with the negator than with alternative speech content, regardless of sentence type, which validates the proposal that not in speech may be semantically supported by co-occurring emblematic gestures that typically communicate negation. When broken down by sentence type, a slightly higher percentage of head shakes were associated with the negator in sentences with narrow-scope negation (all>not and many/most>not), which one might expect, than were associated with sentences with wide-scope negation (not>all and not>many/most). These findings support Calbris (Reference Calbris2011), who argues that “the head shake is simultaneously an emblem of negation and one of the co-speech signs of totality” (p. 175), and suggests that the use of head shakes may be one visual cue to the disambiguation of scopally ambiguous sentences.
Table 1. Alignment of shakes with negator “not”
In the corresponding descriptive analysis of beat gestures in Table 2, fewer than half were aligned with the negator than with alternative speech content, regardless of sentence type. By sentence type, relatively few beats were associated with the negator in sentences in which the quantifier was many or most, regardless of scopal interpretation. Over half of the beat gestures were associated with the negator in all>not interpretations, and fewer than half were associated with the negator in not>all interpretations. The results may indicate that prosodic gestures also have a role to play, and may emphasize the negator in cases of narrow-scope negation. Taken together, we conclude that the use of the negative particle not is more likely to co-occur with a symbolically congruent gesture (i.e., an emblematic head shake), that this is especially true in cases of narrow-scope negation, and that prosodic gestures (i.e., beats) may also serve to emphasize negative particles in such cases.
Table 2. Alignment of beats with negator “not”
The fourth main analysis focused on the alignment of gesture with the quantifier. Here, the findings were somewhat less robust that those for alignment of gesture with the negator. For not>many/most interpretations, gestures in general aligned significantly more often with the quantifier than without, though no statistical relationships were found for all+not sentences of either interpretation. Further descriptive post hoc analyses, parallel to those above, examined the percentage of shakes versus beats aligned with the quantifiers all/many/most. These results are displayed in Tables 3 and 4.
Table 3. Alignment of shakes with quantifiers “all/many/most”
Table 4. Alignment of beats with quantifiers “all/many/most”
Fewer than half of the total number of shakes were aligned with the quantifier than with alternative speech content, regardless of sentence type. When broken down by sentence type, the picture was rather variable. Fewer than half of the shakes were associated with the quantifier in all>not interpretations, and more than half were associated with the quantifier in not>all interpretations. The results for many/most sentences were not consistent with those for all sentences. For both interpretations, fewer than half of the shakes were associated with the quantifier, and this was more pronounced in not>many/most interpretations. This difference between quantifier types was somewhat unexpected considering their relative syntactic positions. If gestures of negation (e.g., head shakes) commence at the negator as the “node” of negation and potentially continue through the rest of the sentence as the “scope” of the negation (Harrison, Reference Harrison2010), a simple prediction would be that head shakes are less likely to co-occur with all, which appears at the start of the sentence and before the negative particle, and more likely to co-occur with many/most, which appear toward the end of the sentence, crucially after the negative particle (though see also Harrison, Reference Harrison2009, for discussion of more variable positioning of head shakes). However, this is not what we observe, as 66% of head shakes aligned with all in not>all sentences. These results could imply that the relationship between node/scope of negation and gesture phrase may be attenuated in the case of scopally ambiguous sentences involving negation, specifically that in contexts of wide-scope negation, head shakes may begin somewhere other than the negator, including earlier, though the same might not be true of other semantically oriented gestures of negation (e.g., Open Hand Prone gestures; see Harrison & Larrivée, Reference Harrison, Larrivée, Larrivée and Lee2016).
In the post hoc analysis of the alignment of beat gestures with the quantifier in Table 4, the reverse pattern to that of shakes was largely apparent. More than half of the total number of beats were aligned with the quantifier than with alternative speech content, regardless of sentence type. When broken down by sentence type, beats aligned with the quantifier in just over half of the all>not, not>all, and many/most>not interpretations, while a majority of beats aligned with the quantifier in not>many/most interpretations. These results suggest that, in general, gestures of prosodic emphasis (i.e., beats) are more likely to co-occur with quantifiers, regardless of their syntactic position, than are gestures of a more symbolic nature (i.e., emblematic head shakes), especially in cases of wide-scope negation. More research including an examination of the relationship between gesture and prosody in this area is needed to shed more light on these patterns.
The final main analysis examined gesture length. Here gestures associated with narrow-scope negation (all>not) were significantly more likely to be longer, spanning more words, than those associated with wide-scope negation (not>all). While the results for many/most sentences were not statistically significant, they were consistent with the direction of relationships for all sentences, and the lack of statistical finding may have in part been related to the fact that there were fewer many/most items than all items, and the former were shorter, a maximum of five words, while the latter were longer, at six words. The relationship between gesture length and scopal interpretation bears an intriguing resemblance to Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a), who found that speakers tend to lengthen the final words of sentences where negation has narrow scope (all>not), though individual production was varied. Whether these longer gestures coincide with longer final words remains to be seen. The findings for gesture length are somewhat in line with Harrison (Reference Harrison2010). Despite the less interpretable findings above for alignment of gesture with the quantifier and the fact that only one negative particle, not, was under consideration here and it always modified verbs, the longer gesture length for cases of narrow-scope negation may offer partial support for Harrison’s association between negation and “scope” of negation.
Limitations and suggestions for further research
Space limitations only permitted the analysis of hand/arm and head gestures for this paper, but other articulators are also involved in the expression of negation (e.g., shoulder shrugs, facial expressions, and eyebrow movements; see Krahmer & Swerts, Reference Krahmer and Swerts2007, Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013; Tubau et al., Reference Tubau, González-Fuente, Prieto and Espinal2015). Further analyses should take these into consideration as additional cues might also associate with sentences involving narrow-scope negation. An examination of how relationships between gesture and scopal ambiguities involving negation play out across languages is also needed to see if the potential cues described here are universal (see González-Fuente et al., Reference González-Fuente, Tubau, Espinal and Prieto2015; Harrison & Larrivée, Reference Harrison, Larrivée, Larrivée and Lee2016; also Cirillo, Reference Cirillo, Baauw, Drijkoningen, Meroni and Pinto2013). In addition, it would be preferable to control for the syntactic position of the quantifier, something that was not possible in this replication of Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a), as well as to examine differences between many and most regarding potential for focus.Footnote 9 Furthermore, as is conventional in studies of the interpretation of specific syntactic constructions (see, e.g., investigations of the prosody–syntax interface in the double negative construction in Espinal, Tubau, Borras-Comes, & Prieto, Reference Espinal, Tubau, Borras-Comes, Prieto, Larrivée and Lee2016; Prieto et al., Reference Prieto, Borràs-Comes, Tubau and Espinal2013), scripted stimuli were necessary in this study in order to target the often awkward expression of negation with quantification. While every effort was made to keep the data elicitation as naturalistic as possible, participant rendering of the target sentences to a video camera under the pretext of making a comprehension test for second language speakers limited the interactional nature of the context and compromised the ecological validity of the study, which may have impacted the naturalness of the gestures and idealized production.
This paper takes a highly quantitative approach in order to offer a broad, distributional view of multimodal communication. However, the data should also be subjected to the microanalytic approach available elsewhere in order to examine the internal features and organization of gestures, such as the repetitive nature of some gestures (e.g., shakes), which can yield intercoder variation, and critical features and phases such as the form/shape of beat gestures as well as preparations and post-stroke holds, which have been found to be highly relevant to the expression of negation (e.g., Harrison, Reference Harrison2010, Reference Harrison2014a; though note that such features are far less discernable in head vs. hand gestures). It is important to note that the analyses conducted here did not extend to prosody or to the relationship between prosody and gesture. Although prior work as described in Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a, Reference Syrett, Simon and Nisula2014b) emphasizes mixed findings on prosodic patterning in relation to the expression of quantification and negation, a close relationship between prosody and gesture certainly exists. In a series of three experiments, Krahmer and Swerts (Reference Krahmer and Swerts2007) demonstrated that a speaker’s production of beat gestures is associated with acoustic changes in their prosody; that the production of beat gestures, an eyebrow movement, or a head nod, is associated with the production of prosodic emphasis on target words in a sentence; and that the presence of visual cues result in a hearer’s perception of elevated prosodic prominence. Thus, the lengthening of gestures for narrow-scope negation may be associated with the lengthening of final words found in Syrett et al. (Reference Syrett, Simon, Nisula, Huang, Poole and Rysling2014a). Finally, prior research on prosodic and gestural cues for the resolution of ambiguity has focused more heavily on language comprehension, with production from relatively few individuals examined in limited detail primarily for the purpose of stimulus construction. This study attempted to provide a rigorous and robust analysis of gesture production as an initial step, and additional analyses relating prosody to gesture in comprehension of language involving scopal interactions is clearly an important area for further study in order to more accurately characterize the multimodal nature of communication.
Conclusion
This study emerged from several bodies of research: (a) compelling associations between the expression of negation and use of gesture in communication, (b) the existence of ambiguity in the interpretation of sentences with an interaction between the scope bearing items of quantification and negation, (c) mixed empirical findings on the use of prosody by speakers when conveying a specific interpretation in such contexts, and (d) the possibility of gesture patterns associating with interpretations of scopal interactions. From the data presented, relationships were found between gestural features and the communication of specific interpretations of scopally ambiguous sentences involving negation and quantification, and those gesture features appeared to some extent to be sensitive to the quantifier used.
Overall, head gestures significantly dominated in this data set. Beat gestures were the most common gesture type in almost all contexts, but emblematic head shakes, which semantically express negation, were statistically associated with many/most>not sentences. In contexts of narrow scope negation (all>not and many/most>not), gesture strokes (especially semantic head shakes but also prosodic beat gestures) were statistically more likely to align with speech content that included the negator. Fewer patterns were observed in analyses of gesture stroke alignment with the quantifier, but the relatively large proportion of gestures (>25%) that aligned with sentence-initial all, which preceded the negator not, raises further questions about the basis for gesture stroke onset, for which testing in truly naturalistic contexts would be the most illuminating. Perhaps the most striking finding was the association between gesture length and sentence interpretation such that in contexts of narrow-scope negation (e.g., all>not), gesture strokes were significantly longer than in contexts of wide-scope negation (e.g., not>all).
We conclude in support of the claim that gestures comprise audiovisual prosody (Granström & House, Reference Granström and House2005; Swerts & Krahmer, Reference Swerts and Krahmer2005) and may play an important role in enabling interlocutors to “see what we mean” (cf. Kellerman, Reference Kellerman1992). Specifically, speakers may manipulate the features of gestural form, placement, and length potentially to help listeners resolve the ambiguities arising from scopal interactions between quantification and negation. Whether manipulation of these specific gestural features actually does facilitate interpretation remains a question for further research.
Acknowledgments
This work was supported by a grant from the Syracuse University Small Scale Grant Program. We acknowledge, with grateful thanks, the participants in this research as well as the anonymous Applied Psycholinguistics reviewers who provided constructive feedback, and Carmel O’Shannessy for consultation on statistical analysis.
Appendix A. Output of Analyses in 5.1 (modified)
Appendix B. Output of Analyses in 5.2 (modified)
Shakes versus nonshakes
Beats versus nonbeats
Appendix C. Output of Analyses 5.3 (modified)
Appendix D. Output of Analyses 5.4 (modified)
Appendix E. Output of Analyses 5.5 (modified)
