At several points in their target article, Pickering & Garrod (P&G) suggest that prediction by simulation is based on determining a conversational partner's intention through perceiving the unfolding action or speech, potentially combined with background knowledge. On this basis, an efference copy of the intended act is generated, enabling the prediction of upcoming behavior and the production of behavior or speech that complements it. But how explicit do the attributed intentions need to be to allow such prediction, and how might comprehenders derive them?
Unfortunately, P&G do not define clearly what they mean by “intention”: Whereas on the part of the actor or speaker, an intention seems to represent nothing more than an “action command” (e.g., in the legend to Figure 3; target article, sect. 2.2), recognizing intentions on the part of the comprehender is more complicated. According to P&G, it can involve considerations of past and present behavior and of the speaker's perceived state of mind, as well as ongoing modifications of this interpretation. In the literature, identifying others' intentions is generally taken to imply an additional step beyond action recognition, that of identifying the goal of this action (see e.g., Levinson Reference Levinson, Enfield and Levinson2006; Tomasello et al. Reference Tomasello, Carpenter, Call, Behne and Moll2005). A similar view underlies the HMOSAIC architecture (Wolpert et al. Reference Wolpert, Doya and Kawato2003), which contains symbolic representations of the task in the form of goals or intentions.
Elaborate attributions of intention based on the interlocutor's potential motivations in the current situation and on general world knowledge are certainly useful for understanding speech. They help to generate expectations about how the conversation is likely to develop and can be used for what P&G call “offline prediction.” Yet although such expectations have, in turn, been shown to influence moment-by-moment language comprehension (e.g., Kamide et al. Reference Kamide, Altmann and Haywood2003; Van Berkum et al. Reference Van Berkum, van den Brink, Tesink, Kos and Hagoort2008), they are presumably not computed on a moment-by-moment basis and remain relatively constant across extended periods of time.
The time-critical online simulations in P&G's account of real-time conversation must require intentions of a more basic kind – some form of heuristic for anticipating others' upcoming actions. To use their example: It is unquestionably valuable if I am quick to predict that someone is preparing to punch me rather than to shake my hand, so I can prepare to move appropriately. But if I start considering why they might wish to hurt me, I will probably be too late in responding.
For real-time online prediction of upcoming words and sentences, I would like to suggest that it may often be possible to rely on contextual clues to a speaker's upcoming actions that are directly perceivable: A particular tone of voice, a facial expression, a hand gesture, or a shift in the speaker's gaze direction can all be informative about how the speaker plans to continue a sentence. In this sense, such cues are closely connected to the speaker's intentions. At the same time, they are often produced unintentionally on the part of the speaker, and can be readily perceived on the part of the comprehender. This is exactly what makes them so efficient: They can help to disambiguate the linguistic signal without requiring deliberate consideration of intentions (cf. Shintel & Keysar Reference Shintel and Keysar2009, who refer to such processes as “nonstrategic generic-listener adaptations”; p. 269).
A prime example of this type of contextual cue is the direction of other people's gaze. Gaze is a salient attentional cue that reliably causes viewers to shift their own attention in the same direction (Emery Reference Emery2000). This occurs even in the visual periphery and without requiring conscious awareness (Langton et al. Reference Langton, Watt and Bruce2000; Xu et al. Reference Xu, Zhang and Geng2011). Additionally, because speakers tend to look at objects they are preparing to mention (e.g., Griffin & Bock Reference Griffin and Bock2000; Meyer et al. Reference Meyer, Sleiderink and Levelt1998), gaze will often directly reflect the speaker's action plan. Such referential gaze is therefore both easy to detect and informative about upcoming sentence content. In fact, comprehenders can and do make rapid use of the speaker's gaze direction to anticipate upcoming referents (Hanna & Brennan Reference Hanna and Brennan2007; Staudte & Crocker, Reference Staudte and Crocker2011) and even to assign thematic role relations (Nappa et al. Reference Nappa, Wessel, McEldoon, Gleitman and Trueswell2009; Knoeferle & Kreysa Reference Knoeferle and Kreysa2012).
These benefits of gaze-following in comprehension can be conceived of as a form of prediction about what will be mentioned next, similar to anticipatory fixations of objects in the visual world in eye tracking studies of spoken sentence processing (e.g., Altmann & Kamide Reference Altmann and Kamide1999; Knoeferle & Crocker Reference Knoeferle and Crocker2006; for recent reviews see Altmann Reference Altmann, Liversedge, Gilchrist and Everling2011 and Huettig et al. Reference Huettig, Rommers and Meyer2011). It is interesting to consider P&G's classification and to speculate on whether this might be prediction by association (e.g., “people who look at objects often mention them shortly thereafter”) or even prediction by simulation (using one's own gaze behavior as a proxy, e.g., “if I had just looked at the kite, I'd refer to it next”). Alternatively, it might be sufficient that speaker gaze attracts the comprehender's attention to a location that is relevant for understanding the unfolding speech utterance. In all three cases, the end result is a coordination of the interlocutors' attention on the same objects in the visual world. This is known to benefit problem solving (Grant & Spivey Reference Grant and Spivey2003; Knoblich et al. Reference Knoblich, Öllinger, Spivey and Underwood2005) and conversation in general (Richardson & Dale Reference Richardson and Dale2005; Richardson et al. Reference Richardson, Dale and Kirkham2007). Such benefits may well be due to a shared perspective and aligned representations of the situation, but they need not imply awareness of the interlocutor's intentions.
You have
Access
At several points in their target article, Pickering & Garrod (P&G) suggest that prediction by simulation is based on determining a conversational partner's intention through perceiving the unfolding action or speech, potentially combined with background knowledge. On this basis, an efference copy of the intended act is generated, enabling the prediction of upcoming behavior and the production of behavior or speech that complements it. But how explicit do the attributed intentions need to be to allow such prediction, and how might comprehenders derive them?
Unfortunately, P&G do not define clearly what they mean by “intention”: Whereas on the part of the actor or speaker, an intention seems to represent nothing more than an “action command” (e.g., in the legend to Figure 3; target article, sect. 2.2), recognizing intentions on the part of the comprehender is more complicated. According to P&G, it can involve considerations of past and present behavior and of the speaker's perceived state of mind, as well as ongoing modifications of this interpretation. In the literature, identifying others' intentions is generally taken to imply an additional step beyond action recognition, that of identifying the goal of this action (see e.g., Levinson Reference Levinson, Enfield and Levinson2006; Tomasello et al. Reference Tomasello, Carpenter, Call, Behne and Moll2005). A similar view underlies the HMOSAIC architecture (Wolpert et al. Reference Wolpert, Doya and Kawato2003), which contains symbolic representations of the task in the form of goals or intentions.
Elaborate attributions of intention based on the interlocutor's potential motivations in the current situation and on general world knowledge are certainly useful for understanding speech. They help to generate expectations about how the conversation is likely to develop and can be used for what P&G call “offline prediction.” Yet although such expectations have, in turn, been shown to influence moment-by-moment language comprehension (e.g., Kamide et al. Reference Kamide, Altmann and Haywood2003; Van Berkum et al. Reference Van Berkum, van den Brink, Tesink, Kos and Hagoort2008), they are presumably not computed on a moment-by-moment basis and remain relatively constant across extended periods of time.
The time-critical online simulations in P&G's account of real-time conversation must require intentions of a more basic kind – some form of heuristic for anticipating others' upcoming actions. To use their example: It is unquestionably valuable if I am quick to predict that someone is preparing to punch me rather than to shake my hand, so I can prepare to move appropriately. But if I start considering why they might wish to hurt me, I will probably be too late in responding.
For real-time online prediction of upcoming words and sentences, I would like to suggest that it may often be possible to rely on contextual clues to a speaker's upcoming actions that are directly perceivable: A particular tone of voice, a facial expression, a hand gesture, or a shift in the speaker's gaze direction can all be informative about how the speaker plans to continue a sentence. In this sense, such cues are closely connected to the speaker's intentions. At the same time, they are often produced unintentionally on the part of the speaker, and can be readily perceived on the part of the comprehender. This is exactly what makes them so efficient: They can help to disambiguate the linguistic signal without requiring deliberate consideration of intentions (cf. Shintel & Keysar Reference Shintel and Keysar2009, who refer to such processes as “nonstrategic generic-listener adaptations”; p. 269).
A prime example of this type of contextual cue is the direction of other people's gaze. Gaze is a salient attentional cue that reliably causes viewers to shift their own attention in the same direction (Emery Reference Emery2000). This occurs even in the visual periphery and without requiring conscious awareness (Langton et al. Reference Langton, Watt and Bruce2000; Xu et al. Reference Xu, Zhang and Geng2011). Additionally, because speakers tend to look at objects they are preparing to mention (e.g., Griffin & Bock Reference Griffin and Bock2000; Meyer et al. Reference Meyer, Sleiderink and Levelt1998), gaze will often directly reflect the speaker's action plan. Such referential gaze is therefore both easy to detect and informative about upcoming sentence content. In fact, comprehenders can and do make rapid use of the speaker's gaze direction to anticipate upcoming referents (Hanna & Brennan Reference Hanna and Brennan2007; Staudte & Crocker, Reference Staudte and Crocker2011) and even to assign thematic role relations (Nappa et al. Reference Nappa, Wessel, McEldoon, Gleitman and Trueswell2009; Knoeferle & Kreysa Reference Knoeferle and Kreysa2012).
These benefits of gaze-following in comprehension can be conceived of as a form of prediction about what will be mentioned next, similar to anticipatory fixations of objects in the visual world in eye tracking studies of spoken sentence processing (e.g., Altmann & Kamide Reference Altmann and Kamide1999; Knoeferle & Crocker Reference Knoeferle and Crocker2006; for recent reviews see Altmann Reference Altmann, Liversedge, Gilchrist and Everling2011 and Huettig et al. Reference Huettig, Rommers and Meyer2011). It is interesting to consider P&G's classification and to speculate on whether this might be prediction by association (e.g., “people who look at objects often mention them shortly thereafter”) or even prediction by simulation (using one's own gaze behavior as a proxy, e.g., “if I had just looked at the kite, I'd refer to it next”). Alternatively, it might be sufficient that speaker gaze attracts the comprehender's attention to a location that is relevant for understanding the unfolding speech utterance. In all three cases, the end result is a coordination of the interlocutors' attention on the same objects in the visual world. This is known to benefit problem solving (Grant & Spivey Reference Grant and Spivey2003; Knoblich et al. Reference Knoblich, Öllinger, Spivey and Underwood2005) and conversation in general (Richardson & Dale Reference Richardson and Dale2005; Richardson et al. Reference Richardson, Dale and Kirkham2007). Such benefits may well be due to a shared perspective and aligned representations of the situation, but they need not imply awareness of the interlocutor's intentions.