I. Prisoner’s Dilemma as Model

In “Morality and Advantage,” the PD is invoked as a model, showing how it is possible for two seemingly inconsistent properties central to a system of moral rules to be jointly consistent:

Baier’s Thesis: Morality is (at least partly) a system, S, of principles such that:

• B1. It is advantageous for everyone if everyone accepts and acts on S, and

• B2. Acting on S requires at least some to perform disadvantageous acts. ^{Footnote 3}

To model these conditions in the PD, one simply identifies acting on S as the cooperation move and violating S as defection. Since defection dominates cooperation, condition B2 is clearly satisfied. Since mutual cooperation is preferred to mutual defection, B1 is satisfied as well. Indeed, dominance of defection and unanimous preference of universal cooperation to universal defection are generally taken to be defining characteristics of the PD game.

It is worth noting that Gauthier’s explanation of how the PD reconciles the seemingly paradoxical conditions in Baier’s Thesis requires conditions that are not always associated with the game, and indeed, cannot even be properly stated for the version of the game with ordinal payoffs that Gauthier considers in “Morality and Advantage”:

Following common conventions, let us label the four possible payoffs in a (symmetric) PD, in descending order, as T (temptation), R (reward), P (punishment), and S (sucker). My gain when another chooses cooperation over defection is either R-S (if I cooperate) or T-P (if I defect). My loss from choosing cooperation over defection is either T-R (if the other cooperates) or P-S (if the other defects). So, the Gauthier condition requires that R-S and T-P each exceeds T-R and P-S. The definition of the PD ensures that two of these four inequalities are met: R-S>P-S and T-P>T-R. Two inequalities remain: R-S >T-R and T-P > P-S. Let us call these ‘the Gauthier conditions’ and rewrite them as follows:

• G1. R > (T+S)/2

• G2. P < (T+S)/2

Thus, G1 and G2 state that reward and punishment lie above and below the temptation/sucker average. A PD is a ‘dilemma’ regardless of whether these conditions are met. Condition G1 is sometimes, but not always, added as part of the definition of the game. G2, as far as I know, never is.

II. Prisoner’s Dilemma as Problem

In Morals by Agreement, the PD is regarded, not as a model for moral behaviour that makes clear how it can have the paradoxical properties that it does, but rather as a problem that must be overcome for the grand project of reducing morality to rationality. What is missing or obscured in Morals by Agreement is the very commonsensical two-part idea to which I have just alluded:

1. 1) Adhering to moral principles may require real sacrifice (the “disadvantageous acts” mentioned above).

2. 2) The benefit we get from moral institutions is a result of the participation of others.

Of course, Gauthier still understands that cooperation in a PD is not individually utility maximizing. My payoff is increased when the other player cooperates, but it is reduced when I cooperate. That is why the PD poses a problem. If moral behaviour corresponds to cooperation in a PD, it is not individually rational. Gauthier’s foils—Hume’s knave, Hobbes’ fool, and Plato’s older brothers—are correct after all. The problem is overcome by observing that we benefit by acquiring a disposition to cooperate in PD situations, given the plausible assumption that such a disposition can be detected with some reliability by others. This is enough to show that it is rational (in the utility maximizing sense) to be moral and so, if the costs of doing so were not excessive, it would be rational to acquire and maintain the disposition to be a keeper of rational agreements with those similarly disposed. But the Gauthier of Morals by Agreement often seems to want more. He suggests that it also shows that it is rational to act morally, perhaps on the grounds that it is always rational to act on a disposition that it is rational (i.e., expected utility maximizing) to have. That suggestion has engendered critical reflection. In particular some have argued that there are circumstances under which the disposition to carry out one’s threats is rational to acquire and to possess but, as things turn out, disastrously irrational to act upon. ^{Footnote 5}

Although discussion in the literature has emphasized puzzles about the rationality of threat fulfillment, examples of threat resistance seem at least as troubling. Here is one from Derek Parfit, ^{Footnote 6} which is itself adapted from an older example of Thomas Schelling. Derek lives on a small tropical island. Knowing that he is likely to face villainous threats from depraved bombers, he has successfully acquired a strong disposition to resist threats and made his possession of this disposition highly visible. In so doing, he has maximized his expected utility, where that notion is defined as in Chapter 1 of Morals by Agreement. Alas, one depraved bomber unexpectedly fails to recognize the situation and credibly threatens to blow both Derek and himself to smithereens if Derek does not give him a coconut. Can we really imagine that it is more rational for Derek to act in accord with his disposition than to give up the coconut? Anticipating possible responses, we may stipulate, in addition, that the disposition to resist has already prevented threats of other bombers, so that Derek’s life would not have gone better without it, and that it will maximize future expected utility, should Derek somehow survive the present threat.

Of course, threat-fulfilling and threat-resisting dispositions are not moral dispositions. Perhaps there is some as yet unimagined refinement of the principle that acts conforming to rational dispositions are rational, which might allow it to apply to the moral dispositions, but not the threat-fulfilling and threat-resisting ones. There is little reason for optimism, however. The categories are closely related. The promise-keeping disposition, for example, is a moral disposition—exactly the one that allows us to confidently participate in mutually advantageous agreements. But I can use that same disposition to make threats by saying, for example, “I promise to break your knee-caps if you don’t give me that coconut.” ^{Footnote 7}

According to Gauthier Reference Gauthier and Danielson1998 and Gauthier Reference Gauthier1994, it is a matter of definition that acts conforming to intentions or deliberative procedures that are rational in an appropriate sense are themselves rational. If we replace the disposition talk of Morals by Agreement with intention talk or deliberative procedure talk, this understanding of the matter does open the possibility that such acts do not themselves maximize utility, and thereby it perhaps preserves the insight that morality sometimes requires sacrifice. ^{Footnote 8} It also, however, represents a break with common understandings of rationality and with the account Gauthier himself so carefully and lucidly lays out for the case of “parametric choice” in Chapter I of Morals by Agreement.

The insight that morality may require behaviour that is not individually utility maximizing does survive in some form in Morals by Agreement. In Chapter VII, Gauthier considers agreements that, though fully rational in the circumstances in which they are made, would never have been made under fair and equal circumstances. The disadvantaged party, Gauthier says, ought morally to “acquiesce in” but not “comply with” such agreements ^{Footnote 9} and (though Gauthier does not emphasize this point) the advantaged person ought to forgo making them. There is some suggestion that the instability of such agreements makes it disadvantageous to make and comply with them; Gauthier concludes the chapter, however, with an admission that this is not always the case:

In the real world, where people are neither equal nor equally rational and where agreements are made under conditions where some have taken advantage of others, morality does sometimes require sacrifice. But this situation is certainly not characteristic of morality and it is not the situation modeled by the one-shot PD.

III. Game Theorists’ Critique

Philosophically minded students of game theory are understandably sceptical of the idea that the PD is an appropriate device to guide thinking about morality. If there is one lesson that game theory has imparted, it is that it is hopeless to recommend any outcome in a game that it is not a Nash equilibrium. Mutual cooperation in a PD is most certainly not a Nash equilibrium. Let me briefly cite three examples.

Here is Kenneth Binmore:

As Binmore sees it, we are engaged in a game of life. This game has multiple equilibria. The game of morals is just a coordination game, in which we aim to select one of the equilibria in the game of life.

In a similar spirit, Brian Skyrms has urged moral philosophers to turn their attention from the PD, where mutual defection is the only equilibrium, to the Stag Hunt game, where there are two equilibria, one of which is unanimously preferred to the other. The problem that vexes Skyrms is how we can move from the inferior equilibrium to the superior one:

And, a few pages later:

A third PD-sceptic is economist Robert Sugden. Sugden’s focus is on what we have come to regard as moral. He is very cautious about drawing genuinely normative conclusions. What we consider moral are widely followed conventions (i.e., stable equilibria in games with many stable equilibria) with the property that each party benefits from others following that equilibrium strategy. ^{Footnote 14}

Sugden’s full, ‘official’ formulation in the following chapter does leave some room for moral-seeming rules that are not conventions and he allows for those that are conventions to call for disadvantageous acts in certain “atypical” ^{Footnote 15} applications. I think he has in mind the same one-shot situations over which Binmore thinks moral philosophers have long wasted their time. Binmore calls them “sore thumbs.” ^{Footnote 16} Despite Sugden’s more nuanced and charitable treatment of moral philosophy, I think he would also find the emphasis on the PD by Gauthier and others misplaced.

IV. Preference Change and Indirect Contractarian Theory

My suggestion for saving the insights of “Morality and Advantage” is a very simple one. We should not see morality as a matter of acquiring a ‘disposition’ that may or may not call for irrational behaviour. Rather, we should see it in terms of changes to our preferences and the concomitant changes in the payoff structures of games representing situations in which we often find ourselves. Games that would have the hopeless structure of a PD to those without moral sensibilities are transformed, for those with such sensibilities, into games with a more tractable structure. Two psychological traits make this possible. The first is the same trait that makes it useful for corporations and political candidates to spend millions of dollars in advertising. Our preferences can be changed by the urging of others. ^{Footnote 17} These changes often result in considered and stable preferences that are revealed in both choices and thoughts and words—just the kind of preferences whose maximal satisfaction, according to Gauthier, characterizes rationality.

The second trait is that, on an individual basis, this advertising costs us nothing. Indeed, there seems to be little that we humans enjoy more than telling each other what to do and what to refrain from, what to admire and what to disdain, what to applaud and what to condemn. A change in preferences that has profound effects on what a person does may be brought about by simple and virtually costless actions on the part of others. Let me offer a single simple example. At the conference where this paper was first presented, one of my dinner companions was a student from the heartland of Canada who was attending the University of Prince Edward Island. I was curious about how he came to be going to such a small university so far from his home and inquired about it. The answer was that he hadn’t really considered going to that university at all. But his parents owned a small dairy bar. One day he noticed a customer who was wearing a UPEI sweatshirt. It seems very likely that the customer, in deciding to wear the sweatshirt that had such a profound effect on the life of my dinner acquaintance, was satisfying his own preferences. It also seems likely that the choice of buying that particular shirt, and the choice of wearing it on that day cost the dairy bar customer little or nothing more than the alternatives he might have considered.

What we, as philosophers, might try to show to be rational are neither particular dispositions nor the actions that accord with them, but rather our advocacy of those actions in light of the effect of this advocacy on ourselves and others.

Although it does not do the major work in his theory, there is good evidence that Gauthier himself recognizes the power of the preference-change phenomenon and the pertinence of this ‘indirect’ approach to the social contract. Some of this evidence comes from Chapter VII, where Gauthier tries to justify his theory of morality from the perspective of the ideal actor at what he calls the “Archimedean point”:

Other evidence comes from what I think of as the more ‘poetic’ chapters at the end of the book. In response to Glaucon’s challenge to refute the thesis that we behave justly only because we are too weak to do otherwise, Gauthier waxes eloquently about the joys that moral agents take from their participation in cooperative activities. The best way to make this discussion cohere with what I think of as the more ‘mathematical’ early chapters is to understand moral dispositions, not as tendencies to curtail or constrain utility maximizing behaviour, but rather as tendencies to prefer the outcomes engendered by behaviour that would otherwise fail to be utility maximizing.

In the remainder of this paper, I discuss the idea that much of moral discourse is advocacy for certain changes in attitudes and preferences and that the morality of behaviour has to be explained in terms of conformity with what is rational to advocate. I have been thinking about this idea for a long time. Originally, I had conceived of it as being part of a normative theory—a rational contractarian theory of morality like Gauthier’s. Gauthier’s theory is direct. It tells us that actions are right if they, or the plans comprising them, or the principles of deliberation by which they were determined, would have been rationally agreed to by equals bargaining from a fair initial position. Mine was to be indirect—it would tell us that the degree of ‘rightness’ of actions is determined by the moral curriculum that it would be rational to agree to teach to each other. ^{Footnote 19}

More recently, I started to think about this idea as a descriptive theory of moral evolution—a theory about how we might have come to have the moral beliefs and attitudes that we, in fact, have. In fact, I think that these projects must be related. The aspect of rational contractarian theories that has most gripped its proponents has to do with justification. They aim to show that morality must have a claim, at least on rational individuals. But I think an equally important virtue is epistemological. It is a great mystery on most accounts, when and why my moral ‘intuitions’ or ‘considered moral judgments’ are guides to the truth. On an indirect account, at least the general form of the answer becomes obvious. My moral beliefs are correct just to the extent that the moral curriculum that produced them was rationally agreed to.

My evidence for the superiority of the indirect form over the direct form came mainly from some somewhat complicated observations about conditions when it is permissible to break agreements. ^{Footnote 20} I now think the idea can be supported by a much simpler observation. Absent special circumstances, a person who has secured some particular benefit for another has performed an act of greater moral worth than a person who has secured that same benefit for himself. It seems much easier and more natural to explain this by an indirect theory—either utilitarian or contractarian—than a direct theory. The reason that it does not normally make sense to urge people to acquire goods for themselves is that they are already hard-wired to do so. Biology takes care of self-benefit directly. We need morality to take care of other-benefit.

Christopher Morris has discussed what seems to me to be an interesting case in point. ^{Footnote 21} Morris asked about the directive once common at video rental stores, “Please rewind,” which encapsulates a rule that one should rewind a tape after viewing it rather than leaving the job for the next customer (or an employee). Morris thought that the rule made sense because the psychological burden of rewinding after viewing was much lighter than the burden of doing so before. But the rule requiring rewinding before viewing has a notable advantage. If we require rewinding after viewing then, inevitably, some people will be required to rewind twice (because of the lapses of others), whereas if we require rewinding before, then all viewers will only have to rewind once. “Rewind before” is more equitable in this sense than “rewind after.” What gives “rewind after” its moral resonance, I think, is that it benefits somebody else, whereas “rewind before” benefits only me. It seems unlikely that anybody would misinterpret the “Please rewind” slogan as urging us to rewind before viewing. (No special urging needed for that!) The same considerations apply to a similar rule that may have survived longer than the video rental rule. In apartment buildings or families where clothes driers are shared, the rule ‘clean lint trap after use’ is more common than the rule ‘clean lint trap before use.’ Again, equity might seem to favour the second rule. And in this case it is much harder to maintain that the burden of following the second rule is greater. Moral considerations, however, push us towards benefitting others rather than benefitting ourselves. ^{Footnote 22}

For both the prescriptive and the descriptive projects, I thought, like Gauthier—especially the Gauthier of “Morality and Advantage”—that the one-shot PD was exactly the right conceptual tool, and that morality had to make it possible for us to rationally choose cooperation in a situation that, in the absence of morality, would have a PD structure. Whether, in the presence of morality, this represents a sacrifice to me depends on the effectiveness of the moral education I have received. We might hope that the immoral lies, for example, are exactly the lies that I prefer not to tell. It is likely, however, that there are some situations in which it is rational to advocate truth-telling, while my own preference structure remains stubbornly skewed in favour of lying.

V. Advocacy Games

In the last section of this paper, I shall say more about the descriptive project. A framework that seems particularly useful for developing the idea is that of evolutionary game theory. Pairs from a population play a certain simple game, like the PD. Although they may play this game many times, each play is viewed as a one-shot game—it is assumed that players have no knowledge of the history of the previous play of their opponents, either with themselves or with third parties. (Repeated games, of the kind made famous by Robert Axelrod, ^{Footnote 23} may have much to tell us about how selfish people can sometimes cooperate, but they have much less to tell us about why that cooperation might be regarded as moral behaviour.) The population evolves according to an appropriate evolutionary dynamic—more successful strategies become more widely adopted and less successful ones fall out of fashion. To model the phenomenon that interests me, I consider something I call ‘advocacy games.’ In addition to pairing randomly to play a simple game like the PD, players occasionally make an ‘advocacy’ move, i.e., they advocate a move in the underlying game. Advocacy itself does not result in any particular payoffs, but if enough players advocate the same move, the payoffs in the underlying game are revised to favour the move advocated. Moves in the underlying game (play moves) are assumed to be determined by mixed strategies. Advocacy strategies are pure. The idea is that an agent views a pure move like cooperate as a rule, and her mixed strategy indicates the seriousness with which she regards the rule, or at least the degree to which she follows it. Both play mix and advocacy moves are adjusted according to learning algorithms. If a player’s recent returns from cooperation exceed those from defection, her play mix skews more towards cooperation. She advocates cooperation as long as recent payoffs while doing so exceed those while advocating defection. Advocacy is supposed to model something like ‘social pressure’ or the “moral or popular sanction” famously described by Jeremy Bentham.

In the versions of the game I have considered so far, I take social pressure to be exerted population-wide, and to have a strength proportional to the fraction of the population exerting it. Here is a picture:

Figure 1 Advocacy and Social Pressure

The horizontal axis represents the number of advocates for one move, say ‘cooperate,’ in a two move game. Everyone not advocating that move is advocating the other. The vertical axis represents social pressures. Positive pressure is the amount by which payoff to the advocated move is increased. Negative pressure is the amount by which the payoff to the move not advocated is decreased. Pressures are zero when each move is advocated by half the population and they reach a maximum value when everybody advocates the same move. Positive and negative pressures kept distinct to model the possibility that there are different psychological limits to such pressures and to allow for comparison of carrot and stick approaches to behaviour modification. ^{Footnote 25} (The plausible hypothesis that guilt is a stronger force than pride is undermined by the recent phenomenon of suicide bombers who claim to be motivated by positive incentives.)

To get some clue about what might happen in an advocacy game based on the PD, we might look at how the game would change under this social pressure. First consider pressure to cooperate:

Figure 2 The PD under Pressure to Cooperate

The axes in each of the four graphs represent utilities of the two players. The southwestern vertex indicates the punishment payoffs that the x and y players get when they each defect. The northeastern indicates the reward payoffs. The northwestern and southeastern vertices indicate where x gets sucker and y gets temptation and where y gets sucker and x gets temptation. Lines connect pairs of points representing outcomes that differ in the move of only one player. The quadrilateral formed is stretched in successive frames. Positive pressure pulls the reward point further to the northeast and negative pressure pulls the punishment point further southwest, while positive and negative pressures together push the sucker-temptation points inwards towards each other. By the second frame in our little movie, we can see by the marks on the axes, that S exceeds P. Now the game has become a game of Chicken. Each player gains by cooperating if his opponent defects and defecting if his opponent cooperates. By the third frame, in addition, reward exceeds temptation. Now cooperation dominates defection and mutual cooperation is the game’s unique Nash equilibrium. If the original PD had different payoffs, it might happen that reward would surpass temptation before punishment surpassed sucker. In that case, the intermediate game is Stag Hunt rather than Chicken. No matter what the payoffs in the original PD, however, eventually the game will become a completely tractable one in which cooperation dominates defection and mutual cooperation is the most advantageous outcome for both players.

On the other hand, we must also consider what happens to a PD under pressure to defect. The unfortunate story is told in the following picture:

Figure 3 The PD under Pressure to Defect

Now the two sucker-temptation points are pulled apart, while the reward and punishment points are pushed towards each other. By frame three, they have crossed paths. The game becomes a ‘prisoner’s delight,’ where defection dominates cooperation for each player, and mutual defection is unanimously preferred to mutual cooperation.

It is not obvious what will happen in an advocacy game in which the underlying structure is a PD. From the transformations in the previous two figures, one might suspect that, if social pressure is sufficiently strong, both universal defection with advocacy of defection and universal cooperation with advocacy of cooperation will be equilibria.

This idea gets further support by considering a simplified version of the game with a population of two:

Here the positive and negative pressures are p and n when both players advocate the same move and zero when they advocate different moves. The resulting game has one equilibrium where both players defect and advocate defection. If the positive social pressure p exceeds T-R, then there is another equilibrium where both players cooperate and advocate cooperation. No matter what the pressures, there are no other equilibria. The cooperative equilibrium, if it exists, is always preferred by both players to the first.

In larger, evolutionary versions of the game, one might expect a ‘neutral’ population to move to the first equilibrium, where defection is both practiced and advocated. Since each player gets higher returns from defection than cooperation no matter what the others do, players should begin defecting. Advocating cooperation would then seem to be generally costly, lowering the payoffs of exactly the strategies that are widely employed and raising those that are not.

Simulations only partially confirm these expectations. In a wide variety of advocacy games based on the PD, players do tend to reach a state in which everyone always (or as often as the mutation rate permits) defects. Sometimes, however, they reach a state in which everyone is maximally cooperative. Advocacy seems to be somewhat less stable than behaviour. It is almost always true that most players advocate cooperation in the ‘settled-cooperation’ states and defection in the ‘settled-defection’ states. There are often some players, however, who we might call ‘hypocritical’ or, more charitably, ‘reform-minded.’ These agents play one move while advocating another. The number of such agents often remains at some fixed value greater than zero for long periods. It may also spike so that for short periods those advocating against the settled status quo may exceed half the population. Changes in behaviour generally are immediately preceded by changes in advocacy of that behaviour, but sometimes changes in behaviour seem to occur spontaneously, bringing changes in advocacy in their wake. Changes in advocacy do not always lead to long-lasting changes in behaviour. Sometimes the changes are reversed before any changes in behaviour take place (as if a reform movement was launched only to be quickly abandoned when it proved unsuccessful). A few examples of these games are shown in Figure 4 below.

Figure 4 Payoffs and Cooperation in the Advocacy PD

The parameters for all three graphs are the same. The underlying game is a ‘traditional’ PD with payoff values of 5, 3, 1, and 0. The population size is eight. The horizontal axis represents the rounds of play from 0 to 40,000. The thicker plot traces the average play mix (i.e., probability of cooperation) of the eight players in the population. There is a built-in ‘mutation rate’ of .03, which prevents that line from going above .97 or below .03. The thinner plot traces the proportion of the population that advocates cooperation, so that line can assume nine values from 0 to 8. Since two out of the eight members play in each round, each player will have played about 10,000 games, 5000 as Player 1, from where he can consider changes in play and advocacy. With parameters set as they are here, each player will have considered advocacy changes about 100 times. The initial population is ‘balanced’: Players 1-4 advocate cooperation and have high play mixes themselves, while Players 5-8 advocate defection and have low play mixes. As might be expected, the play quickly (i.e., in the first 1,000 moves) moves towards defection. Advocacy remains relatively neutral for a while, however. In the first and third graphs, the proportion advocating cooperation begins to rise, with cooperative behaviour following closely as described above. In the middle graph, it falls somewhat and the population remains in the minimally cooperative state. While the population’s average play mix is in the maximally or minimally cooperative state for most of the time, the population spends relatively little time in a state of uniform advocacy.

A little thought suggests a possible explanation for why settled cooperative states are sometimes reached. An advocacy move does have a relatively quick and direct effect on my payoff. It also has a more delayed and indirect effect, however, by changing the probabilities that I and others will continue to move in the way that we do. By advocating cooperation, I slightly increase the odds that others will cooperate and this benefits me. However, I also increase the odds that I myself cooperate, and this hurts me. If the benefit that I get from others’ cooperation exceeds the cost of my own cooperation, it might be expected that it would behoove me to advocate cooperation. This is exactly the condition that Gauthier invoked to explain the seeming paradox of Baier’s Thesis.

Indeed, simulations seem to confirm the importance of the Gauthier conditions for achieving cooperation in the advocacy PD:

When the payoffs in the underlying PD were 10, 9, 1 and 0 (satisfying both G1 and G2), the population was in a state of maximal cooperation at move 4,000 in 38 trials out of 0 and in a state of maximal defection in only 6. When the payoffs were 10, 2, 1, and 0 (violating G1) the defection state was reached in 25 trials and the cooperation state in none. When the payoffs were 10, 9, 8 and 0 (violating G2), the defection state was reached in all 50 trials.

These simulations are suggestive, but they have a serious limitation. They all employ very small population sizes. I have come to realize that, for larger populations, reaching cooperative states in this kind of model may require unrealistically high numbers of interactions. Perhaps the only way that advocacy of the kind I envisage can evolve and change behaviour is if social pressure is modeled as a ‘local’ effect. Each agent is more likely to interact with some than with others. As the Bentham quotation indicates, the source of the moral or popular sanction is really not the entire population, but rather those individuals with whom one happens to interact. On a local version of the advocacy game, each agent would have her own payoff matrix, shaped in part by the advocacy moves of those with whom she has interacted, and especially by those with whom she interacts frequently. Frequency of interaction among pairs of agents in a population is not entirely random. As social networks like Facebook remind us, if John interacts frequently with Jane and Jane interacts frequently with Jill, then John is likely to interact more frequently with Jill than with a random stranger. One way to model this idea might be to think of the underlying game as a spatial game, where agents are arranged in a suitable geometry and the probability of interaction between a pair of agents is proportional to the distance between them. One might expect the results of local advocacy to be similar to those of global advocacy with small populations. In any event, investigations of this sort would seem to be an appropriate way to understand how morality can lead us to act contrary to our pre-moral preferences in much the way Gauthier’s “Morality and Advantage” suggested it should do.