R1. Rebooting a second-person neuroscience

What may have irritated a number of commentators is that our proposal is simultaneously less and more extreme than some assume or deduce from our target article, as in the following three respects: First, we are less extreme, in the sense that we do not claim that second-person engaged interaction is the only way to understand others, although we defend our claim that it is ontogenetically prior. Second, we are more extreme in a methodological sense, in that we claim that crucial aspects of social understanding remain under-studied as long as we assume that spectatorial paradigms are the only approach that can tell us meaningful things about how we go about in the social world. Third, and most importantly, we are extreme in our theoretical proposal to start looking in a different way at concepts like “minds” or “intentionality,” in that these are not just “properties” to be read into behavior. The neurobiological substrates that can be found in such instances may be real, but it is unclear what their fine-grained functional role might be in the field of social cognition. To us this seems comparable to earlier stages of the search for human psychology, in which logical operators and symbolic processing proved to be fruitful in computing aspects of conscious human thinking and decision-making, but not in describing the basic nature of human cognition, in that it failed to capture the subsymbolic processes that underlie logic and symbols. However, in the case of social understanding, our approach is more radical in that it not only questions the mechanism by which a cognitive process comes into being, but it also questions whether it is at all meaningful to speak of a mechanism and a cognitive process when it is confined to intra-agent space.

Before advancing to the discussion of more specific points raised by the commentaries, we will briefly touch upon two general concerns: First, as stated in the previous paragraph, we do not, at any point, exclude the possible existence of an observation mode of social cognition, a way of theorizing about people from a third-person stance, in terms of a “we-mode” (Gallotti) or by means of “simulations” of what others feel: We can certainly infer things about others when we are observing them through a one-way mirror. Rather than denying first- and third-person social cognition, we want to emphasize the status of second-person engagement and direct interaction. While we acknowledge that within interaction there is a continuum (Simpson & Ferrari), we also hold that there is something fundamentally different about direct interactions with the world or with others. Specifically, we do not think that approaches that give priority to first- and third-person modes can satisfactorily describe how these interaction-related faculties come into existence. Importantly, we do not think that they are necessary prerequisites for successful social interaction, whereas we propose that a second-person mode is a prerequisite for coming to know others. From a developmental point of view – rather than assuming the existence of a gap between self and other – we understand the subjective experience of such a “gap” as a result of development through interaction and engagement with others, a process in which, for instance, contingencies related to my own volition and that of others may play an important role (see target article sect. 4). In other words, it seems to us that it is the emergence of the capacity for reasoning about people for inference and simulation that needs to be investigated and understood as resulting from our prior ability to interact with them, and not how social actions and interactions emerge from our capacity to represent knowledge about other people.

Second, the focus of our article falls on neuroscientific methods, and more precisely, on possible solutions for the problems or potential incompatibilities that seem to exist between a neuroscientific approach that focuses on an individual's brain, and a social and cultural world in which “information processing” does not seem to be at all limited to the confines and characteristics of particular individuals. One of the core insights here is that neuroscience should not content itself with a spectatorial view of social cognition. This is why we advocate the adoption of paradigms that allow a person to engage or interact with someone in a meaningful manner. Indeed, some commentaries (e.g., Vermeulen, Pleyers, & Mermillod [Vermeulen et al.]) identify paradigms which they assume could be considered as belonging to the category of “second-person” studies such as Olsson et al. (Reference Olsson, Ebert, Banaji and Phelps2005), Hein et al. (Reference Hein, Silani, Preuschoff, Batson and Singer2010), or Kitayama et al. (Reference Kitayama, Snibbe, Markus and Suzuki2004), whereas in fact these studies are purely observational. So is the concept of social immersion (Krach et al.), whereby interaction is merely used to prime a different way of reasoning. According to our proposal, these paradigms are insufficient to approach, explore, and explain the core processes that go on and constitute everyday-life social interactions.

On the other hand, the need for a second-person neuroscience that looks beyond the individual brain as an explanans does not imply that looking at an individual brain is incompatible or meaningless in a second-person context (as suggested by Gallagher et al. and Sameen et al.), or that specific brain activations cannot be an adequate explanans for a well-defined explanandum. The core message of a second-person neuroscience is that in social cognition studies, the part that is often left out, in an attempt to isolate an explanandum, a scientific phenomenon, or a system (which has, in psychology, been confused with isolating a person), might not just be part of the explanandum, but might well be the crucial explanans. In other words, we should not simply abandon any specific way of probing parts of the system, but we should be aware that we are only dealing with parts of a causally complex dynamic framework out of which we lifted a specific and perhaps non-essential part. Consequently, we should not draw conclusions on how people interact or what parts of the brain are responsible for a function in interaction, if we have never taken such measurements during interaction. Otherwise, the functional role we ascribe to specific brain areas is confined to a spectatorial account, which sits comfortably behind the one-way mirror.

R2. Contextualizing and extending the second-person account

The fact that we do not seek to replace all other alternative theoretical accounts, but instead advocate the primary role of second-person engagement and interaction for social cognition and describe its sufficient empirical characterization as a key target for the future of social neuroscience, does not make the implications of our approach less wide-ranging. As Cummins suggests in his commentary “the ‘dark matter’ may be much larger.” In fact, as Cummins points out, the overarching problem lies in the occidental tradition of focusing on the individual, rather than on the context. Indeed, this is also reflected in the issue of free will in western philosophy, and the controversy surrounding experiments questioning free will (e.g., Haggard Reference Haggard2008; Libet Reference Libet1985; Soon et al. Reference Soon, Brass, Heinze and Haynes2008), which revolve around the question of whether our actions are determined by our own volition or are instead decisions caused by “something else.” In this context we can also mention studies on unconscious determinants of behavior and unconscious behavioral priming or goal activation (Bargh & Earp Reference Bargh and Earp2009; Custers & Aarts Reference Custers and Aarts2010; Doyen et al. Reference Doyen, Klein, Pichon and Cleeremans2012). In all of the above described cases, the fact that we perceive as exotic the idea that individuals might be mere pawns in an environment that “controls us” outside of awareness, only stems from an artificial split between individual agents and their environment, and between individuals. On the contrary (as suggested by Dominey), one could argue for a starting point, which holds that individual agents are intertwined with and part of their environment. Consequently, the question should be how we come to perceive ourselves as intentional agents that are separate from others and how this develops and results from our innate capacity for (inter-) action and engagement. What is the minimal requirement, not only of the individual brain, but of the dynamic system, that can account for the fact that I am able to perceive other people as different from myself, without taking this “gap” between self and other as epistemologically given?

Both Cummins and Froese et al. point towards the importance of focusing on dynamic systems, an idea that Dominey extends to robotics. Crucially, this includes an interaction between “A's brain” and “A's body,” as well as “B's brain” and “B's body” (see Froese et al., but also Longo & Tsakiris), with the important point being that these cannot be isolated from their shared environment. This harks back to what we mentioned earlier, namely that taking into account the dynamic system does not imply that one cannot assume sub-systems. It just means that these cannot be studied in isolation. As Longo & Tsakiris suggest for agency, Cummins points towards outcome prediction as an important mechanism. For instance, predictive coding mechanisms might constitute one of the brain's primary functions, which could be related to subcortical bottom-up processes and reward mechanisms (Hamon-Hill & Gadbois; Swain, Konrath, Dayton, Finegood, & Ho [Swain et al.]). One interesting question is whether, at the system level, there is something like “shared predictive coding” – how the coupled system anticipates its own future states as well as those of the emergent, overt system behavior as co-constructed by the agents meeting each other in a social encounter. This ties in with Cummins' suggestion to go a step further, not only investigating brain activities in interactive settings and relating brain activities of different agents, but also to relate these to emergent phenomena in the interaction process. Approaches to study dynamic patterns of interpersonal gaze adjustment by making use of virtual characters in interactive settings have been introduced by Bente et al. (Reference Bente, Eschenburg and Krämer2007b; Reference Bente, Rüggenberg, Krämer and Eschenburg2008b). These studies, however, have been restricted to the behavioral aspect of gaze coordination and have neglected the social cognitive dimension and its neural correlates. Our ongoing efforts are targeting the integration of both behavioral and neural data captured during human interactions, and utilize newly established dual eye-tracking methodology to connect experimental variations to participants' responses (Barisic et al. Reference Barisic, Timmermans, Pfeiffer, Bente, Vogeley and Schilbach2013).

As highlighted in the target article, another important field in which interpersonal dynamics come into play is language research. This is also emphasized by the insightful comments provided by Evans, Gambi & Pickering, and Syal & Anderson, who suggest that a second-person neuroscience can learn from language research as well as it should itself “assume a second-person perspective” (Gambi & Pickering) because – historically – the majority of work in this field has considered isolated examples of language production or comprehension without focusing on pairs of participants engaged in dialogue. We are thankful to these authors for pointing towards this interesting parallel in research development and are in full agreement with their view that investigations of real-time dialogue are likely to advance our knowledge of processes of interpersonal prediction and adaptation in joint activities. We are also intrigued by the comment provided by Evans, who draws attention to the person category of “first-person inclusive” to denote the union of speaker and addressee in non-Indo-European languages and thereby reminds us to also raise questions about cultural differences, as well as the intricate relationship of culture and social interaction.

Furthermore, the issue of language-based communication also ties in with the insightful commentary provided by Kevin Moore who discusses similarities between the second-person approach and a Wittgensteinian analysis of psychological concepts. According to Moore both approaches converge in their argument for a priority of interaction in grounding language and providing meaning that is not reliant on “ontologically private” states. Moore is also right in suggesting that our interest in the transformation of implicit to explicit social cognition stems from the idea that first- and third-person perspective arise from ontogenetically prior second-person experiences. We are also most sympathetic to Moore's suggestion that on a Vygotskian view social interactions serve as the motor, that drives developmental processes (“maturing functions”) and may lead to their “internalization,” thereby allowing for the emergence of explicit forms of social cognition.

R3. How far and wide: Situating the second-person account within current theories of social cognition

Some commentaries (Hamilton; Gallagher et al.; Sameen et al.) seem to suggest that we do not take the second-person idea far enough, whereas others seem to suggest that we take it too far (Moore & Iacoboni; Overgaard & Krueger). We take this disparity in judgment as an opportunity to spell out further where we situate our account.

Hamilton argues that the target article lacks a sufficiently detailed theoretical model of how social cognition works in order to lead empirical investigation of the neural mechanisms enabling it. We agree that – although the underlying philosophical ideas are only sketched in broad strokes – Hamilton is right to place the authors beside proponents of the enactive approach like de Jaegher et al. (Reference De Jaegher, Di Paolo and Gallagher2010), who emphasize interaction dynamics as an important feature and – in part – constitutive aspect of social understanding. Hamilton goes on to discuss (i) computational models derived from game theory, (ii) enactive approaches based on dynamical systems theory, and (iii) information processing models, which emphasize the idea of the brain being involved in generating predictions about future events. Hamilton clearly favors the latter kind of approach. She also hints at the possibility that these theories might not be mutually exclusive, but may converge.

Traditionally, enactive approaches have been formulated as being massively anti-representational, thus engendering a paradigm shift in the cognitive sciences (Chemero Reference Chemero2009; Hutto Reference Hutto2008; Thompson Reference Thompson2007; Varela et al. Reference Varela, Thompson and Rosch1991). Yet, it should be emphasized that the main idea of enactive cognition is not incompatible with the traditional idea that the brain is in the business of processing information by way of representations. This should already be clear when we trace back the use of the term “enactive” to its origins in the work of Jerome Bruner (Reference Bruner1964). Apparently unbeknownst to Varela et al. (Reference Varela, Thompson and Rosch1991), Bruner introduced the notion in the context of his distinction of three ways of storing the knowledge that is acquired consecutively during the cognitive growth of the infant: enactive, imagistic, and linguistic representations.

The first kind of representation is, according to Bruner, essentially connected to and inseparable from the execution of certain actions. While the latter idea is compatible with Varela's later usage of the term, the idea that it is fundamentally a kind of representation is not. Yet, even though we cannot elaborate this thought further here, it demonstrates that simply placing a theory alongside enactive approaches does not yet determine the details of the theory. Enactive theories range from providing very modest to very strong claims, and some of the basic ideas are actually deemed compatible with the idea of predictive coding (cf. Clark Reference Clark2013). So we agree with Hamilton's assessment that the theoretical options discussed are not mutually exclusive; and as mentioned above, the goal was not to replace the existing theories altogether. As in the development of new experimental paradigms, further work needs to be done in order to support empirical research with an appropriate theoretical background.

Overgaard & Krueger defend the social perception view (as proposed, e.g., by Gallagher Reference Gallagher2008) against our criticism that it remains committed to a spectator theory. They present us with a dilemma between the (implausible) claim that social perception is impossible and acknowledgment of its possibility, which would show that our approach and the social perception view cannot be distinguished. But this appears as a dilemma only if we had proposed that there is merely one means of coming to understand others, namely, by way of interacting with them. However, this is not what we claimed. We claim that paradigmatically social situations are characterized by interaction and engagement and that in such situations, social cognition is fundamentally different from situations where we are mere observers. This, of course, implies that the latter situations are also possible. The problem for the social perception approach, in our view, is of a conceptual nature, because it is typically cashed out in terms of direct perception. Yet, it is just not clear whether direct social perception is possible without involvement or interaction. As recent debates have shown, the notion of “direct perception” is not unproblematic. This is acknowledged by Gallagher (Reference Gallagher2001) and Hutto (Reference Hutto2008) who supplement their view with a further “narrative” hypothesis, according to which young children are exposed to stories in which people act for reasons based on their beliefs and desires. Such narratives are supposed to provide the necessary scaffolding for the acquisition of the concepts of belief and desire. So, even in these terms, the social perception view on its own is not enough to explain the full range of social cognitive abilities. Second, the notion of perception at play here must be a rich one. Gallagher (Reference Gallagher2008) calls it “smart perception,” which is conceptual (epistemic seeing) and includes a wide range of background assumptions and world knowledge when, for example, it is claimed that even young children directly perceive affordances. This appears problematic, because this notion of perception lines up with a weak notion of “theory” as in the “theory theory” approach. Thirdly, not only in light of philosophical debates about the epistemology of perception, but also in light of recent developments in the neuroscience of perception, it is questionable whether there is such a capacity or process as direct perception. According to the predictive coding hypothesis, brains are in the business of predicting future sensory events. In turn, these hypotheses are tested by sensory feedback and by action (Friston Reference Friston2010; Frith Reference Frith2007; Hohwy Reference Hohwy2012). Clark (Reference Clark2013) has recently given an overview of the explanatory strength of this general approach to brain functions while leaving no room for direct perception. On this, we agree with Rietveld et al., who adopt this view with regard to human beings' responsiveness to affordances. So, the defender of the social perception approach faces the dilemma of enriching her notion of perception so much that it is in danger of collapsing into a version of “theory theory.”

This discussion leads naturally to the comments put forward by Gallagher et al. and Sameen et al. who criticize our account of containing an unresolved tension by maintaining that social neuroscience is supposed to be in the business of locating neural correlates of social cognition, which they suggest is at odds with the main thrust of the enactive approach to cognition that we seem to endorse. More specifically, these authors point out that our approach is still dealing in representations, whereas the enactive approach rejects this notion altogether. First, we have already elaborated above that only extreme versions of the enactive approach (e.g., Hutto & Myin, in press), are really at odds with the idea that neural/mental representations play a special role in the explanation of cognition. In fact, representations can be a fruitful conceptual instrument if they are understood as eliciting the functional role of neural processes in a three-place concept of representation as opposed to a simple causal-correlative understanding of representation (e.g., Vogeley & Bartels Reference Vogeley, Bartels, Newen, Bartels and Jung2011). Second, and this point specifically addresses Sameen et al., if the task of cognitive neuroscience is not supposed to be the investigation of the neural mechanisms that enable cognitive activities, then the critics must determine an alternative task for cognitive neuroscience. Surely, the idea of a neural correlate is coherent enough if it is not understood as a neo-phrenological attempt to isolate brain regions, which are responsible for specific cognitive functions. Instead, the relevant neural mechanisms must be conceived in terms of the context of dynamics that include body and (physical as well as social) environment. Since the main thrust of our target article is to provide new ideas for research in the neurosciences that deals with the nature of social cognition, emphasis was placed on what neuroscientists could do to investigate social cognition in an ecologically valid way. While this will continue to include measuring brain activity, we are in full agreement with Gallagher et al. that “the question is, what brains do in the complex and dynamic mix of interactions.” Finally, whether (some or all) embodied phenomena relevant to social cognition can (or should) be characterized in representational terms is an open and important question. Simply assuming that they cannot (or should not), like the authors do, is surely inappropriate (Vogeley & Bartels Reference Vogeley, Bartels, Newen, Bartels and Jung2011). Here, it is important to distinguish different claims associated with embodied, embedded, enactive, and extended cognition. While it is not clear whether Gallagher et al. and Sameen et al. would subscribe to all of these different theses, the role played by the various cultural, evolutionary, and personal factors in shaping not only cognition, but also the cognition-enabling brain circuitry, has to be investigated in the context of the more general debate about the so-called extended mind. That is, the background question of this debate is whether these factors can be said to be constitutive elements of cognitive processes or merely important scaffolding or shaping factors. Only if one were to hold a strong extended mind thesis (Clark & Chalmers Reference Clark and Chalmers1998), would the role of brain circuitry for cognition be diminished. But whether such a strong claim can be vindicated against the more modest scaffolding-claim (Adams & Aizawa Reference Adams and Aizawa2008) is also an open question (Menary Reference Menary2010). In sum, the important role that body and environment play in the dynamics of social cognition has been already sufficiently emphasized in the target article (e.g., see sect. 2). Moreover, the general idea of the enactive approach, namely that cognition is an activity of the whole organism, is compatible with an explanatory framework that makes use of mental representations, even if these have to be understood in a more action-oriented sense (Clark Reference Clark2001; Wheeler Reference Wheeler2005). Whether a radical enactivism or the extended mind thesis can be vindicated is an open question.

R4. Agency, intentionality, and development

Another focus of commentaries dealt with how our second-person account may be seen to fall short with respect to explaining the experience of agency and intentionality. Specifically, several authors suggest a role of perspectives and triadic interaction in the development of ascriptions of agency and intentions, which partially stems from a misinterpretation of our proposal.

Longo & Tsakiris suggest that second-person experiences presuppose first-person (singular) experiences and that the former cannot be investigated independently of the latter. Their main point is that contingencies also play an important role in the first-person experience of embodiment, ownership, and agency, which can all be experimentally manipulated. Of course, our proposal does not imply that investigations of first-person experience (e.g., by means of the famous rubber hand illusion) cannot inform the investigation of social cognition in interesting ways. In their argument for the claim that first-person experiences are necessary for second-person experiences, Longo & Tsakiris focus on the case where I form an intention, which leads first to my own action and only then elicits the action of another. But one of the main points of the target article was to emphasize cases where “being addressed as you” (sect. 3.1.1) in social interaction may elicit a reaction on my part that can lead to reciprocal intentional relations. It is not clear how Longo & Tsakiris's discussion of body ownership and bodily agency bears on the issue, unless they want to argue that first-person agency experiences are ontogenetically prior to second-person agency experiences (in addition to the question whether they are logically prior): We think that this is not the case. In fact, as suggested by, for instance, Dominey and Froese et al., the gap between self and other may be largely artificial and may only develop at a later ontogenetic stage. One can, for example, imagine that differences in contingencies, with respect to my own intentions and action outcomes as compared to my intentions and others' action outcomes, can lead to a differentiation between self- and other-agency. As Lewis & Stack write, “infants become naturally attentive to gestures that are intentional because they are repeated and become predictable.” Importantly however, in our proposal, contingencies as described by Longo & Tsakiris are only a part of what it means to be in a second-person interaction with someone.

Another example of reading our proposal as more radical than it is, are the commentaries of Gallotti, Krach et al., and Lewis & Stack. Gallotti argues – in line with ideas developed by Chris Frith (see Frith Reference Frith2012b) – that the emphasis on the second-person perspective should be replaced by a first-person plural perspective or “we-mode.” Similarly, Lewis & Stack suggest that second-person understanding may emerge from a more foundational “first-person plural experience.” Here, too, it seems important to repeat that we never argued that it is impossible to represent ourselves as being in a “we-mode.” Persuasively, Gallotti emphasizes that the we-mode may constitute an irreducible mode of cognition vis-à-vis cognitive states in the I-mode. As Searle (Reference Searle, Cohen, Morgan and Pollack1990) and others have shown, we-intentions (e.g., “we are playing this game together”) cannot be analyzed in terms of or reduced to a sum of the individual intentions of the agents and that the we-mode has considerable explanatory power. We can completely agree with this claim and have actually said nothing to undermine it. In addition to Gallotti's claim, our account can actually tell a persuasive story about the origin of our we-intentions. Surely, we-intentions cannot simply precede social interaction. Neither should we take we-intentions as brute, inexplicable facts. The only viable explanation seems to be that joint engagement and activities may lead to intentions in the we-mode. That is, the we-mode presupposes the features emphasized in our approach and is thus no replacement for it. Even if the we-mode was irreducible to two I-modes, it would still assume that within an interaction, the actions of an agent can be causally explained in terms of representations that reside in that agent alone. In this sense, it would be clearly spectatorial and would downplay sensorimotor accounts (see, e.g., Lewis & Stack). Rather than downgrading ideas of collective intentionality, our approach can lead towards an answer to the question of how collective intentions arise from interaction dynamics and emotional engagement.

In the same vein as Gallotti, commentators Lewis & Stack suggest that “knowledge of ‘you’” emerges from a previously shared context in which infant and adult form a (proto-) conversational unit that can be described as a very early first-person plural experience. As mentioned before, we suggest that it may be the other way around. Implicitly, Lewis & Stack acknowledge this when they contend that these shared activities are “largely stage-managed by the adult.” Without needing to discuss whether it is actually genuinely participatory and jointly “managed,” it is clear that this implies that someone takes the initiative by emotionally engaging with the other in a communication loop that is characterized by reciprocity, and is supported by social affordances. Furthermore, they ask what may fill the “spectatorial gap.” Even from a spectatorial perspective, one valid suggestion might be that it is essentially a form of embodied sensorimotor know-how (McGeer Reference McGeer2001; Schlicht, forthcoming). This is actually what Lewis & Stack themselves seem to suggest. The earliest forms of awareness are arguably “sensorimotor and take place within practical activities.” One important characteristic of know-how (with respect to some ability) is that its development goes hand in hand with the ability to recognize the execution of this ability in someone else's actions. Knowing how to swim enables one to recognize when someone else executes her swimming ability successfully (McGeer Reference McGeer2001). Similarly, the suggestion would be that we develop social know-how in the context of scenes of mutual engagement and interaction. But, as mentioned before, the crucial difference in our approach is that this development of social know-how is not a faculty we develop to “bridge a gap.” Rather, it constitutes our primary way of experiencing others. This is in line with Bruner's (Reference Bruner1964) suggestion that the developmentally primary form of representation is “enactive” in the sense that it is embodied and inextricably tied to (inter-) action. It provides the “basis for reflective forms of social understanding,” as Lewis & Stack emphasize. As argued above, the first-person plural perspective does not emerge from nowhere, but instead may be seen to arise from second-person experiences. Our point is not that all infant attentiveness already and necessarily involves complex awareness of othe's minds. Our point is that it could. Lewis & Stack are very clearly ruling out any reference to awareness of mind in early infancy on the grounds that this is too rich an interpretation. But this is precisely the point. Why should mentality be assumed to be too rich? Because it is non-perceivable and needs inference is the usual answer in cognitive developmental psychology. It is not clear whether this is Lewis & Stack's answer, too, but if so, they fall prey to the very same methodological behaviorism that according to our argumentation neuroscience and psychology should try to avoid.

Krach et al. present a methodological equivalent of Gallotti's argument, in that they suggest that having direct interactions is not necessary, but that instead priming interactions by social immersion might suffice to put people in some social state of mind. In our view this proposal runs into the same trap as the we-mode proposal by suggesting that our brain may simply switch to an “interaction mode,” which represents ourselves in some interactive context. But from a methodological point of view, putting people in such a state does not seem like an acceptable substitute for actually having them interact. One might observe differences with non-social immersion situations, but these would still bear no relationship to the experience of being engaged in ongoing interaction.

An interesting issue put forward by Moore & Paulus and Cleret de Langavant et al. is related to differences between dyadic and triadic interactions. Even though we focus on dyadic interactions in our article, we, of course, do not claim that social cognition stops there. Triadic intentional relations clearly are complex interactions, and we are happy to agree upon the view that joint attention enriches the kind of social encounters that go on in the first few months of life. In fact, we specifically refer to the importance of looking at how triadic interactions develop out of dyadic ones and make suggestions about how to investigate their neural bases in the target article (sect. 3). For instance, it might be that triadic interactions hold the key in the development of a distinction between how my own and someone else's actions are coupled to an effect in the outside world, something that the dyad may not provide. Unfortunately, Moore & Paulus do not substantiate or present evidence for their strong claim that the third element is a necessary component for social cognition. They are claiming that “it is unclear how the properties of dyadic interactions alone […] could reveal intentionality at all” simply because, as they state, “intentionality is tied inextricably to actions on objects.” Based on this premise, they hold that “the problem with purely dyadic interactions is that there is no obvious way for the intentionality of action – its object directedness – to be manifest.” Hereby, the authors neglect the possibility that an action performed by another may be directed at oneself (as is often the case in the first 7–8 months of life, viewed from the perspective of the infant) and that this may itself reveal intentionality and elicit a basic form of intentional understanding. Furthermore, Moore & Paulus do not demonstrate that their stronger claim is true. Indeed, the quote above seems to reveal a relatively narrow conception of intentionality, which appears to be confined to intentional actions towards objects, excluding many other manifestations of intentionality, as in love (emphasized by Brentano [Reference Brentano, Rancurello, Terrell and McAlister1874/1973]) or attention, which are both dyadic intentional relations. This is surprising in light of their contention that intentionality comes in many different flavors. Recent philosophical debates have demonstrated that a more viable notion of intentionality also includes – in addition to propositional attitudes like belief and desire – more basic manifestations of intentionality like sensorimotor or bodily intentionality (cf. Merleau-Ponty Reference Merleau-Ponty1958; Schlicht, forthcoming; Sinigaglia Reference Sinigaglia, Morganti, Carassa and Riva2008). This basic form of intentionality is crucial in the early stages of cognitive development and reveals itself in dyadic intentional relations. The same line of argument holds for their criticism of our emphasis on emotional engagement. The point, again, was not to downgrade other features of social cognition, but to emphasize the role of emotional engagement in contrast to traditional approaches.

Furthermore, Moore & Paulus charge the authors of the target article for a lack of a positive account of “what it means to understand mind.” Possibly, the philosophical approach behind this was underemphasized in the target article. Yet, we tried to make clear that an approach in line with enactive and embodied approaches, based on the notion of embodied sensorimotor know-how might be one way to go (McGeer Reference McGeer2001; Schlicht, forthcoming). This approach emphasizes that there is a form of social understanding that is more basic, different in nature and below (propositional) mentalizing as traditionally understood. We are thankful to Rietveld et al. for elaborating the relevant kind of “skilled intentionality” that is important for such engagement and the pick-up of social affordances in context. Rietveld et al. attempt to align social cognition with “nonsocial engagements with the environment” by embedding social affordances in a richer context of affordances in general, based on the plausible claim that any affordance has to be conceived of in a whole field of quite different affordances. However, although this is true, social affordances are still quite different from “object affordances” because they are much more complex and transient elusive during an ongoing social interaction. This is due to the fact that objects of the environment are indifferent to our attempts to understand them, while other people immediately react to such attempts and thereby modulate the field of social and other affordances.

Cleret de Langavant et al. argue that a second-person neuroscience should be able to explain communicative triadic intentional relations. In order to emphasize the difference between dyadic and triadic intentional relations, they point to the pathological condition of heterotopagnosia, which suggests that brain networks for dyadic and triadic relations may be different. We agree that this is an important issue that deserves further investigation and in our target article we have discussed studies of joint attention, which use virtual characters (Schilbach et al. Reference Schilbach, Wilms, Eickhoff, Romanzetti, Tepest, Bente, Shah, Fink and Vogeley2010b) and which have the potential of directly comparing the neural correlates of dyadic and triadic relations. We are thankful for the pointer provided by Cleret de Langavant et al. that in cases such as heterotopagnosia, the use of anthropomorphic virtual characters may be of limited use.

R5. Theoretical accounts of the second-person and the role of the “mirror neuron system” (MNS)

Some commentaries emphasize the need for strong theoretical accounts that could promote the development of a second-person neuroscience. We have already mentioned Hamilton in this respect, who compares the applicability of computational models such as game theory (see also Gariépy et al.), dynamic models (see also Froese et al.), and cognitive models. Furthermore, Hamilton advances her proposal of the so-called STORM (social top-down response modulation) model, which she describes as a “socially engaged information processing model” based upon the idea of the brain's visuo-motor stream being modulated by other “social brain systems.” As mentioned earlier, all these models describe only part of the social reality; for instance, computational models and cognitive models are very useful at a non-dynamic level (see the target article's Fig. 2). However, these models necessarily assume a causality reducible to one agent, which may be the case in observational situations, but not in interaction. This said, we acknowledge that there is a component within a single individual that can tell us something about how non-interactive social cognition works. The “modulation” that Hamilton describes in STORM assumes that there is some sort of “generator” that generates social behavior, which is then streamlined according to context. Again, this falls into the trap of assuming agent-internal drives that somehow must compete with the context that has to be integrated in a unified account. We suggest the reverse, namely that, despite some intrinsic reward-related motivational component, the social context is the initial drive in this process, making use of brain areas that have developed sensitivities to contexts over prolonged social learning. We are not denying such “modulation,” in as much as it is simply the recruitment of stored information on ongoing information processing. But the core is that, in interaction, such modulations may only make sense in the context of the other.

Gariépy et al. suggest game theoretical conceptions as a suitable theoretical framework. We found it surprising that the authors call on Gintis (Reference Gintis2009) to overcome critiques on game theory, as Gintis (Reference Gintis2009) himself states that game theory alone is not a satisfactory account for a couple of things, such as belief sharing. Specifically, Gintis (Reference Gintis2009, p. 243) writes:

This sounds very much like a proposal that is in favor of an expansion of game theory, rather than a justification of its use in all contexts. Gintis (Reference Gintis2009) goes on:

This, in fact, is exactly where we want to be heading (cf. Froese et al.), although we acknowledge that at this point a formal model including dynamics is lacking. Furthermore, game theory is essentially about decision-making, preferably in a “game” context – that is, in a context in which cooperation and/or competition play a central role. Many human interactions are not about decision making. In fact, they often do not contain a cooperative or competitive context (Pfeiffer et al. Reference Pfeiffer, Timmermans, Bente, Vogeley and Schilbach2011). It is important to look at what motivates people outside of external reward or otherwise “optimal decisions” (Engemann et al. Reference Engemann, Bzdok, Eickhoff, Vogeley and Schilbach2012).

Dezecache, Conty, & Grèzes (Dezecache et al.) question whether “mirror neurons” are involved in the processing of social affordances, understood as action possibilities in the context of social interactions. Yet, they simply assume a canonical interpretation of the function of mirror neurons, namely, as facilitating simulation. This is clear when they write that the affordance interpretation is in conflict with “what MNs are known to do, that is, to simulate an observed motor pattern.” In the light of recent controversies about the “correct” interpretation of the mirror mechanism (de Bruin & Gallagher Reference De Bruin and Gallagher2012; Gallagher Reference Gallagher2007; Gallese & Sinigaglia Reference Gallese and Sinigaglia2011), that is, whether they facilitate simulation or direct social perception, it is surprising – at least at this stage – that Dezecache et al. presume that anything is supposedly “known” about the function of mirror neurons. One may, however, suggest an alternative interpretation of the “mirror neuron” activation that has less to do with the understanding of another person's mental states. Several studies suggest that “mirror neurons” may encode one's own action possibilities in the light of an observed action. Buccino et al. (Reference Buccino, Lui, Canessa, Patteri, Lagravinese, Benuzzi, Porro and Rizzolatti2004) showed that such neurons do not fire upon observation of a dog barking, presumably because barking is not in one's human motor repertoire. More to the point, Gazzola et al. (Reference Gazzola, van, Worp, Mulder, Wicker, Rizzolatti and Keysers2007) investigated aplasic patients, born without either arms or hands, while they observed healthy subjects perform hand actions. In addition, they recorded neural activity correlated with the performance of actions with mouth or feet. They found that the patients' “mirror neurons” fired upon their observation of hand actions, yet it was the set of neurons that was typically activated when the patients' themselves performed actions with their mouth or feet. That suggests that they encode or transform perceptual information about an intentional action in terms of what they themselves can do. This interpretation is also not in conflict with other existing studies. In contrast to canonical neurons, “mirror neurons” are social in the sense that they require observation or execution of an action, they are not activated merely upon perception of an object without an action being performed on it (Schlicht, forthcoming).

Simpson & Ferrari similarly seem to conflate the empirical discovery of the “mirror neuron” network with ways of interpreting it when they suggest that the simulation interpretation of the “mirror neuron” discovery is “tightly linked” to the discovery. Against this way of seeing things, it should be emphasized that the “simulation theory” is not part of the discovery of “mirror neurons” even though this may still be a widespread view. Moreover, taken as a theoretical option in the explanation of social cognition, simulation theory is patently circular because it presupposes what it aims to explain, namely, an understanding of another person's mental states (Newen & Schlicht Reference Newen and Schlicht2009). At the core of the “simulation theory” are pretend mental states that are thought to be used as input for a decision mechanism in order to attribute its output to the other person. But in order to produce such pretend mental states in oneself that bear a sufficient resemblance with the other person's mental states, one already needs to know what the other person feels or thinks. This objection led Goldman (Reference Goldman2006) to defend a hybrid account containing elements of theory-theory. Simpson & Ferrari claim that “nearly all work on single cell recordings of mirror neurons involves second-person interactions,” but they do not demonstrate that this is actually so. The example of perceiving someone else's emotional facial expression is clearly not an example characterized by reciprocity and interaction dynamics. But it suggests that the activation of “mirror neurons” is first and foremost tied to perception and the selection of adaptive behavioral responses, not to simulation, which is in line with Dezecache et al.'s interpretation of data from a recent study by Conty et al. (Reference Conty, Dezecache, Hugueville and Grèzes2012). This is in accordance with studies of our group that suggest that the human mirror neuron system is involved in early detection of potentially socially salient signals, whereas the evaluation of actually socially salient signals is then handed over to the medial prefrontal cortex as part of the mentalizing system (Kuzmanovic et al. Reference Kuzmanovic, Georgescu, Eickhoff, Shah, Bente, Fink and Vogeley2009; Santos et al. Reference Santos, Kuzmanovic, David, Rotarska-Jagiela, Eickhoff, Shah, Fink, Bente and Vogeley2010). It is also in accordance with the interpretation of “mirror neuron” activation as underlying the execution and observation of embodied skills. Whether, in light of all this, one would like to continue calling these sets of neurons, mirror neurons, is an entirely different question.

In the context of accounts of “embodied simulation” often closely associated with the discussion of “mirror neurons,” Vermeulen et al. draw attention to the recently described Simulation of Smiles (SIMS) model by Niedenthal et al. (Reference Niedenthal, Mermillod, Maringer and Hess2010) and argue that we fail to discuss this proposal in spite of it being “a direct and detailed second-person theoretical model.” While we are sympathetic to the ambitious SIMS account and the emphasis it puts on personal involvement, for example, by means of eye-contact, we also note important differences in comparison to our account. As discussed at great length in the target article, we, again, do not subscribe to the notion of “simulation” both on conceptual as well as empirical grounds. With regard to the putative “mirror neuron system” (MNS) – often thought to underlie the respective “simulations” – it seems plausible to us to assume that neural activity in the relevant brain regions is modulated by sensorimotor experience, which is often obtained through interactions with others. Rather than providing “simulations,” activity changes in these brain regions might, therefore, be more closely related to perceiving possibilities for (inter-) action (cf. Rietveld et al.). Furthermore, we believe that the use of ecologically valid experimental paradigms will help to further elucidate the contributions of both the MNS and the mentalizing network of the brain during social interaction (Becchio et al. Reference Becchio, Cavallo, Begliomini, Sartori, Feltrin and Castiello2012; Schilbach Reference Schilbach2010; Wang et al. Reference Wang, Ramsey and Hamilton2011).

R6. Clinical applications and individual differences

Finally, we address the set of commentaries which focuses on the individual in terms of disorders (Nephew; Redcay et al.; Rietveld et al.), in terms of the importance of individual differences (Chakrabati; Moore & Iacoboni), and in terms of the investigation of motivational, reward-related components of cognition (Hamon-Hill & Gadbois; Swain et al.).

Moore & Iacoboni argue for a residual utility of non-interactive experimental scenarios, in the context of establishing individual differences, whereby one needs to be able to separate one agent's characteristics from the other's. While we acknowledge the importance of individual differences (see sect. 6 of the target article), we think that looking at individual differences does not require abandoning interaction. As suggested by Chakrabarti, it is plausible to assume that social interactions, which have characteristics in addition to those of the individual interactors, are influenced by individual differences; for example, the degree to which interactors may exhibit autistic traits or carry genetic polymorphisms that can modulate cerebral responses to different reward stimuli (e.g., Chakrabarti & Baron-Cohen Reference Chakrabarti and Baron-Cohen2011). We are thankful to Chakrabarti for raising this important point and his confirmation that interactive paradigms might be particularly well suited for the investigation of social reward sensitivities, both in healthy controls, as well as cases of pathology.

Similarly, Nephew and Vermeulen et al. point out that standardized investigations of social interaction could be a useful “tool” for the identification and differential diagnosis of psychiatric disorders (cf. Vogeley & Newen Reference Vogeley, Newen, Wood, Allen and Pantelis2009). We use the example of high-functioning autism as a case in point, but it is certainly true that disorders such as schizophrenia, and possibly also chronic depression and personality disorders, can be construed as disorders of social interaction, rather than social observation. With regard to autism and the question of reward-related neurocircuitry contributing to an intrinsic motivation during social interaction, it is also noteworthy that prominent accounts of autism suggest specific “social motivation” deficits without resorting – to the best of our knowledge – to interaction-based paradigms to test these and their underlying neural bases. Importantly, neural networks that do not involve reward-related neurocircuitry, but have been implicated in social cognition, appear to be relatively intact in autism (Marsh & Hamilton Reference Marsh and Hamilton2011). In this line of thought, we are grateful for the commentary provided by Redcay et al., who carefully describe relevant differences between interaction and observation and the challenges associated with isolating the interaction component. Redcay et al. also present their own experimental approach of using a live face-to-face video feed in conjunction with neuroimaging methodology, which resonates well with our proposal of truly interactive paradigms. Also, the authors are in agreement with our suggestion that the investigation of differences between interaction and observation could be particularly relevant to the advancement of our understanding of autism, as spectatorial tests often fail to find differences between patients and controls (e.g., Nation & Penny Reference Nation and Penny2008). Also, Redcay et al. refer to exciting recent neuroimaging evidence in autism, which capitalizes on this distinction and demonstrates hypoactivity in patients' brains for a real-time joint attention task and hyperactivity during a control condition, which suggests a failure to modulate brain responses according to whether the task required a social interaction or not (Redcay et al. Reference Redcay, Dodell-Feder, Mavros, Kleiner, Pearrow, Triantafyllou, Gabrieli and Saxe2012). This is completely in line with our suggestion of the translational potential of the second-person neuroscience approach.

R7. Concluding remarks

In writing the target article our goal has been to highlight that in spite of the remarkable progress made in the young and emerging field of social neuroscience, the neural mechanisms that underlie real-time social encounters – which should be a the very heart of the field's research interests – are only beginning to be studied. In order to take on this challenge, social neuroscience may be helped by drawing upon a second-person approach to knowing other minds, which is based on interaction and emotional engagements between people, rather than mere observation. The commentaries we have received in response to this proposal have provided important and fruitful new aspects for the critical discussion and the possible conceptual improvement of the account, most of which we have hopefully been able to engage with, more or less directly. None of the challenges raised appear to be fatal to our proposal, but, instead, may have helped to enrich the sketched “landscape” of a second-person neuroscience both in terms of relevant theoretical and empirical questions.