The target article argues for an approach to coalitional psychology involving a focus on the question of how a robot could be made to “see” groups (Pietraszewski). We agree that considering representations is central to that objective. We would like, however, to propose a different approach, which nicely complements that of the target article as it is aimed at a different level of analysis.

Philosophically, our approach accords with the extension to Marr's levels of analysis proposed in Poggio (Reference Poggio2012). Its key idea is that in addition to Marr's three classical levels, we can also describe a system at the level of the principles of learning needed for the system to self-organize into a solution to the problem. Recent successes in artificial intelligence show that it is possible to solve difficult problems without a computational or algorithmic understanding of how the system comes to solve them (e.g., language models which do not need linguistics, such as Brown et al., Reference Brown, Mann, Ryder, Subbiah, Kaplan and Dhariwal2020). On this level of analysis, it is not the representation itself that needs to be understood, but rather how it can be learned.

Another way in which our approach differs from that of the target article is that we think that the data the brain use to train its representation of groups likely includes action as well as perception. This is because agents must learn representations that not only help them perceive the world, but also act appropriately in it. Thus, we base our approach on deep reinforcement learning (Mnih et al., Reference Mnih, Kavukcuoglu, Silver, Rusu, Veness and Bellemare2015): A framework where agents receive observations of their environment and process them into internal representations, which they then use to select actions. The environment has a state (which is often known only imperfectly by the agent). Agents receive rewards when they take certain actions in certain states, and learning proceeds by systematically tweaking representations and decision rules (encoded in a neural network) in order to further an objective of maximizing expected future rewards.

A common misconception of learning-based approaches to cognitive science is that they entail a blank slate perspective wherein data alone induce all the mind's structure. In practice, this could not be further from the truth. For instance, when neural networks are trained to represent natural images, convolutional architectures learn Gabor-like receptive fields (resembling primary visual cortex), but fully connected architectures do not (e.g., Saxe, Bhand, Mudur, Suresh, & Ng, Reference Saxe, Bhand, Mudur, Suresh and Ng2011). All network architectures feature inductive bias of one form or another. Therefore, the question always boils down to: What is the right architectural inductive bias to solve the problem at hand?

To capture coalitional psychology, we propose the latent-variable model shown in Figure 1. It aims to decompose social interactions into recognizable and reproducible behavioral primitives – called options (Sutton, Precup, & Singh, Reference Sutton, Precup and Singh1999). The recognition part of the model could be learned in an unsupervised way, for instance by optimizing network weights so as to maximize mutual information between latent variables and observed behavior of other agents, as in Vezhnevets, Wu, Eckstein, Leblond, and Leibo (Reference Vezhnevets, Wu, Eckstein, Leblond and Leibo2020). Some latent variables may come to encode the options of others (e.g., attack and defend), whereas others could be informative ancillary attributes – such as clothing style, language, and so on. The behavior reproduction part of the model could be learned with a method resembling that of Vezhnevets et al. (Reference Vezhnevets, Osindero, Schaul, Heess, Jaderberg, Silver and Kavukcuoglu2017) – the choice of option specifying a desired change in the (social) environment that the network must learn to produce through actions. Decomposing behavior in this way simultaneously induces a representation of the social world (what is going on), and presents a set of implementable options to take in response to it (what to do about it).

Figure 1. Proposed architecture for a learning agent that can acquire social group representations from experience.

The network layers between the recognition and reproduction representations can be interpreted as decision-making circuitry, and are trained by reinforcement learning (e.g., Mnih et al., Reference Mnih, Kavukcuoglu, Silver, Rusu, Veness and Bellemare2015). These layers should come to represent and use decision-relevant information such as group membership. After learning, the result is an agent that can select appropriate response options for its current context (e.g., encountering ingroup vs. outgroup individuals).

Note that this model dispenses with many of the explicit information-processing functions mentioned in the target article. It does not need any explicit machinery for stringing together chains of triadic primitives, modifiers of defaults, or generating counterfactuals. If these mechanisms are necessary then they will emerge implicitly, just as the perceptual representation and option-production circuitry emerge (Botvinick et al., Reference Botvinick, Barrett, Battaglia, de Freitas, Kumaran and Leibo2017). By adopting the architecture of Figure 1, the system designer is specifying only that the agent will aim to decompose its social world into options, but not precisely what those options will be, or how to accomplish the decomposition. Instead, the agent learns all that for itself, using the data generated from its interactions with other individuals in its environment. In engineering terms, this kind of approach is thought to be more robust and scalable than one that relies on explicit engineering of each information processing function (LeCun, Bengio, & Hinton, Reference LeCun, Bengio and Hinton2015).

Once we have such a learning agent then we can study it in silico with neuroscience-inspired analysis methods (e.g., Zhuang et al., Reference Zhuang, Yan, Nayebi, Schrimpf, Frank, DiCarlo and Yamins2021). The key question of where and how the agent comes to represent social group assignments becomes one that we can answer empirically. In particular, one could vary the social environment (e.g., conflict vs. cooperation) and probe how different representations emerge as a result. Methods such as representational similarity analysis (Kriegeskorte, Mur, & Bandettini, Reference Kriegeskorte, Mur and Bandettini2008) can be applied to explicitly compare these emergent representations to the explicit representation proposed in the target article.

Modern deep reinforcement learning methods enable an alternative approach to the question of how agents represent social groups. Where the target article explicitly enumerates information processing functions, the approach we propose instead involves neural networks that self-organize to solve reinforcement learning problems and softer forms of inductive bias. This level of understanding complements the more explicit level described in the target article.