Richerson et al. argue that cultural systems, such as prosocial religion and institutions that maintain social order, play a vital role in enabling the cooperation of both small- and large-scale human societies. We agree. But we are unconvinced that cultural group selection (CGS) is the only hypothesis able to account for the existence of such phenomena. CGS leads to group-functional culture by selecting on cultural variation (Boyd & Richerson Reference Boyd and Richerson2010), but such selection presupposes that group-functional equilibria exist already, suggesting that other processes, including within-group dynamics, contribute to the emergence of group-functional culture. Since researchers have long recognized that individuals with the requisite power coercively and consensually create rules, we propose a complementary “rule-making hypothesis” for group-adaptive institutions. Understanding the contribution of such agent-based social processes is essential for evaluating the value and limits of CGS theory.

Focusing specifically on rules and institutions that control social deviance (one of Richerson et al.'s three examples of group-adaptive culture; see sect. 6.2), we describe two manifestations of this rule-making hypothesis. The first emerges when asymmetries in power or influence allow parties to create and impose rules that satisfy their own interests. Depending on whether the interests of these coercive authorities coincide with those of the group, these individuals will sometimes but not always create and maintain rules with group-level benefits. An alternative form of this hypothesis develops when power is more evenly distributed. In these cases, coordinated groups of individuals can consensually create, modify, and enforce rules that control defection.

Individuals can have key roles in producing group-functional rules if they have sufficient power or influence to institute cultural changes and enforce new rules. Schapera (Reference Schapera1970) observed that Tswana tribal chiefs drove cultural change by introducing innovations ranging from single rules, such as the banning of beer, to infrastructural changes, like the establishment of a police system. The chiefs' objectives in creating rules included quelling social disturbance, reducing poverty, and collectivizing otherwise divergent interests – goals with explicit group-level benefits. Functional rules can also stem from leaders dealing with novel forms of conflict. Llewellyn and Hoebel (Reference Llewellyn and Hoebel1941), in their ethnography of the Cheyenne legal system, described how tensions arose after one man borrowed another individual's horse without permission. The military chiefs eased the friction between the parties and then established a rule: “Now we shall make a new rule. There shall be no more borrowing of horses without asking … [If] the taker tries to keep them, we will give him a whipping” (p. 128). Considering the role of coercive rule-makers offers an alternative explanation not only for the origin of group-functional culture, but also for its maintenance, because powerful rule-makers should continue to enforce group-functional rules as long as the rules satisfy their own perceived self-interest. Moreover, the coercive authority dynamic accounts for the ubiquity of institutions that asymmetrically benefit the powerful, spanning etiquette norms (e.g., Trobriand Islanders: Malinowski Reference Malinowski1926) and food taboos (e.g., the Etoro: Kelly Reference Kelly1980; the Aranda: Spencer & Gillen Reference Spencer and Gillen1927; and the Sanumá: Taylor Reference Taylor, Kensinger and Kracke1981).

Coercive rule-makers who have the power to change rules should create group-functional culture when their own interests coincide with those of their groups. In some cases, these overlapping interests stem from coercive rule-makers preferring groups with cooperative, compliant individuals. Elders of the Kurnai of Australia demanded that young boys “listen to, and obey the old men” and “live peaceably with their friends” (Howitt Reference Howitt1885, p. 316), rules that improve the welfare of the elders while also creating group-level benefits. Studies of punishment in animal societies show that CGS is not necessary for the coercive enforcement of group-adaptive behavior (Singh and Boomsma Reference Singh and Boomsma2015). Reeve (Reference Reeve1992) found that naked mole rat queens harass lazy subordinates, while Flack et al. (Reference Flack, de Waal and Krakauer2005; Reference Flack, Girvan, de Waal and Krakauer2006) noted that dominant macaques police conflicts among subordinates, stabilizing their own high status while reducing within-group conflict.

To this point, we have illustrated the ability of powerful individuals to create group-adaptive rules. However, the development of group-functional culture does not require that novelty originate with coercive authorities. Parties of similarly powerful individuals can also establish and maintain rules to control each other's behavior. Systems of rules that emerged across camps during the California Gold Rush represent a well-studied example of self-interested individuals coordinating to create and enforce novel institutions of social order. Despite the absence of organized state law, miners residing at promising digs convened and instituted majority-approved rules to protect mining rights and safety (McDowell Reference McDowell2004). Violations of those rules spurred injured parties to seek redress through the support of the community, who used coordinated punishment to enforce rules. Ostrom's (Reference Ostrom1990) review of small-scale institutions that manage common pool resources similarly exemplifies how the coordination of self-interested agents can produce group-functional culture. Fishermen in Alanya, Turkey, for example, took 10 years to perfect an institution to control overharvesting and conflict, with a built-in means of motivating individuals to monitor and enforce violations of mutually recognized rules. Last, group consent can interact with authorities to produce group-functional culture. Among the Enga of New Guinea, leaders presented cultural innovations to their groups, though group consensus ultimately governed adoption. As Wiessner (Reference Wiessner2002) summarized, “The innovations promoted were ones that leaders felt could be played to their own advantage; the innovations that stuck were those that worked for the individual and the group” (p. 251).

Here we briefly reviewed evidence that individuals motivated by self-interest can create group-functional culture through coercion, consensus, or both. Although our discussion is largely restricted to rules controlling deviance, these arguments apply to the development of other domains of group-functional culture as well, such as institutions that reward otherwise costly behaviors (Glowacki & Wrangham Reference Glowacki and Wrangham2013) and prosocial religion (Norenzayan et al. Reference Norenzayan, Shariff, Gervais, Willard, McNamara, Slingerland and Henrich2016). Only after recognizing complementary and alternative hypotheses for group-functional culture, as well as the criteria for testing among them, will we be able to assess Richerson et al.'s claim that CGS is uniquely important in generating the cultural systems underlying human cooperation.