Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-02-10T21:29:49.713Z Has data issue: false hasContentIssue false

How gender affects the efficacy of discussion as an information shortcut

Published online by Cambridge University Press:  18 June 2019

Yanna Krupnikov*
Affiliation:
Department of Political Science, Stony Brook University, Stony Brook, New York, United States
Kerri Milita
Affiliation:
Department of Political Science, Illinois State University, Normal, Illinois, United States
John Barry Ryan
Affiliation:
Department of Political Science, Stony Brook University, Stony Brook, New York, United States
Elizabeth C. Connors
Affiliation:
Department of Political Science, Stony Brook University, Stony Brook, New York, United States
*
*Corresponding author. E-mail: yanna.krupnikov@stonybrook.edu
Rights & Permissions [Opens in a new window]

Abstract

There are a number of observed gender differences in the frequency of political discussion, perceived levels of expertise, and importantly, openness to persuasion. This article explores the consequences of these differences for political choices. Given the difficulty in separating influence from homophily with observational data, this paper relies on a group-based experiment. Results suggest that when selecting between candidates, women are more likely to accept information from others, even if the information in the signals is not helpful. Men, on the other hand, often ignore outside signals in favor of sticking with their own choices even when outside signals would be helpful to their decision-making. A reanalysis of a previously published experiment on social communication leads to similar gender differences.

Type
Original Articles
Copyright
Copyright © The European Political Science Association 2019

Social influence varies by context. Whether an individual accepts a message from a friend, family member, or co-worker is contingent on the strength of the relationship between the individual and their associate (Kenny, Reference Kenny1998), previously-accepted competing messages (Huckfeldt et al., Reference Huckfeldt, Johnson and Sprague2004), as well as a host of individual traits. When citizens communicate with one another, these individual-level differences can have profound aggregate effects, as some individuals persuade and others are persuaded. For example, Ahn, Huckfeldt and Ryan (Reference Ahn, Huckfeldt and Ryan2014) write that political “experts” play an outsized role in electorates not just because experts are influential “opinion leaders,” but also because their opinion is “more committed, less malleable and thus tends to survive and dominate the democratic conversation” (254).

Gender differences in political discussion can also have profound aggregate effects, where discussion can actually result in larger biases in aggregate public opinion than would exist if the same individuals lived in isolation (Ahn et al., Reference Ahn, Huckfeldt and Ryan2014). For example, men's opinions are overrepresented in politics because they are more likely to offer their opinions (Atkeson and Rapoport, Reference Atkeson and Rapoport2003) and to talk in mixed company (Karpowitz and Mendelberg, Reference Karpowitz and Mendelberg2014). On the other end, women are less likely to be named as key political discussion partners than men are (Huckfeldt and Sprague, Reference Huckfeldt and Sprague1995; Djupe and Sokhey, Reference Djupe and Sokhey2014); and, even after controlling for their actual levels of political expertise, people tend to assume men know more about politics than women (Mendez and Osborn, Reference Mendez and Osborn2010).

In this paper, we argue that men's opinions are overrepresented for another reason: men are less likely to be socially influenced than women which can benefit or hurt men relative to women depending on the context. Early studies of political behavior showed that women tended to consider themselves outside of politics as it was a “man's domain” (Campbell et al., Reference Campbell, Converse, Miller and Stokes1960), an attitude that may still exist today, as many women feel unqualified to run for political office (Kanthak and Woon, Reference Kanthak and Woon2015). Further, while men are typically encouraged to embrace individualism, women are typically socialized to seek consensus (Eagly and Wood, Reference Eagly, Wood, Lange, Kruglanski and Higgins2011; Koenig et al., Reference Koenig, Eagly, Mitchell and Ristikari2011), suggesting that women pay more attention—and give more credence—to outside opinions.

The benefits and drawbacks of these gender differences depend greatly on context. For example, men's tendency to stubbornly stick to their own beliefs prevents them from being misled by biased messengers (Cialdini, Reference Cialdini1984), but also limits opportunities for learning. On the other hand, women's likelihood of seeking and building consensus in legislative settings is beneficial. Yet, consensus is only possible when people are willing to be responsive and thoughtful about the preferences and needs of other individuals (Barnes, Reference Barnes2016).

Keeping the inherent limitations of previous observational research in mind, we utilize a group-based experiment in which subjects are asked to choose between two computer-generated candidates. Subjects are randomly assigned discussion partners with the goal of determining which candidate offers the larger benefit—without knowing the candidates' positions. This design allows us to both observe the process of influence as well as to evaluate the quality of the subjects' vote decisions. The design also rules out the possibility that perceived influence is really just selection effects or differences in how men and women form networks, allowing for stronger causal inference.

In our experimental context, subjects have a candidate they should typically favor in the election—like a voter with a partisan preference. Therefore, subjects often know how they should vote before any social communication. In these situations, women are less likely to vote for the candidate who offers a larger benefit because they accept biased social communication. In situations where prior beliefs are incorrect, however, men are disadvantaged because they often ignore messages that would have corrected their mistaken priors.

Hence, the efficacy of social communication as an information shortcut depends on the probability that the individual knows how to vote prior to social communication, conditioned on the individual's gender. When an individual is more often right than wrong, like in a partisan election in an established democracy (Sniderman, Reference Sniderman, Lupia, McCubbins and Popkin2000), men's stubbornness prevents them from mistakenly changing their mind. On the other hand, when the decision context is complex, as it is with some ballot initiatives (Milita, Reference Milita2015), women's openness to social influence helps guide them to the correct decision.

1. Gender and receptiveness to socially supplied information

People's characteristics affect the extent to which they rely on social information. One such characteristic is gender. A large body of work in psychology suggests that women are generally more persuadable than men, though how much more depends on the conditions in which people receive new information (Guadagno and Cialdini, Reference Guadagno and Cialdini2002; Eagly and Wood, Reference Eagly and Wood2013). A key condition underlying this persuadability is the extent to which a message is perceived as “social”—whether the messages appear to come from another person or group. Women, this research suggests, are more persuaded by social messages, while men are likely to dismiss these messages (Guadagno and Cialdini, Reference Guadagno and Cialdini2002).

Psychological research attributes these gender differences to an underlying theory of social norms and roles often called “social role theory.” Developed in large part by Alice Eagly, the theory suggests that gender differences in behavior are a function of “contrasting distributions of men and women into social roles” (Eagly et al., Reference Eagly, Wood, Diekman, Eckes and Trautner2000, 125), which subconsciously affect individual behaviors. Women's greater persuadability, then, may come from their overall higher likelihood of being sorted into social roles that emphasize interpersonal interactions and communal behavior. Men, on the other hand, are often more likely to be sorted into roles that emphasize independence (Tannen, Reference Tannen1990; Eagly and Wood, Reference Eagly, Wood, Lange, Kruglanski and Higgins2011). Indeed, it is this type of sorting that may underlie the gender differences in social network behavior (Djupe et al., Reference Djupe, McClurg and Sokhey2016), and importantly in our case, in responses to social information: men often deliberately attempt to differentiate themselves and assert their independence, while women strive for greater consensus (Tannen, Reference Tannen1990). Again, these psychological theories point to a dual set of implications: the same forces that make women more persuadable also lead women to seek cooperation and consensus in groups.

Although previous work has shown that under many contexts women are more likely to be influenced by social information, it is unclear how these patterns directly translate to politics. It is possible that the social influence gender gap is not as deleterious as it seems in many psychological experiments, as people enter actual political discussions with preferences and biases (Lodge and Taber, Reference Lodge and Taber2013), encouraging women—as well as men—to cling to their prior beliefs. Furthermore, the nature of political preferences may lead women to ignore signals from discussion partners with different preferences (Lupia and McCubbins, Reference Lupia and McCubbins1998), also helping to shrink this social influence gender gap.

However, if in this context women still remain more persuadable than men, it is important to consider how this will affect the quality of their vote decisions. Does women's willingness to accept new information increase the probability that they will vote for the candidate who offers the larger benefit (Lau and Redlawsk, Reference Lau and Redlawsk1997)? And does men's reluctance to incorporate social information into their judgments harm their decision making? As we explain in the next section, the benefits of openness to persuasion are conditional.

2. The risks and rewards of taking a shortcut

In evaluating the consequences of gender differences in openness to persuasion, we must consider how often discussion will lead to political decisions that can be considered “better.” The quality of a political decision is an elusive concept. Indeed, the political decisions of largely uninformed people may be considered to be of high quality, so long as the individuals believe they have made good decisions (Lau et al., Reference Lau, Andersen and Redlawsk2008). Nonetheless, with full acknowledgment that a “correct” political choice is a difficult construct, scholars have suggested ways of addressing whether a political choice aligns with the political interests of a given individual or fails to do so. Specifically, Lau and Redlawsk (Reference Lau and Redlawsk1997) propose a concept termed “correct voting.” This approach suggests that voters vote “correctly” when they choose the candidate that matches their subjective preferences. This could mean, for example, that voting for a candidate who pledges to raise the voter's taxes can actually be “correct”—as long as that is what the citizen desires.

Keeping this definition of correct voting in mind, we can begin to examine how political discussion can be helpful in increasing the likelihood of voting correctly. First, political discussion can be a useful shortcut, clarifying a candidate's positions and characteristics for those less attentive to politics. Those who find politics interesting for its own sake can subsidize information costs for people who lack the time, ability, or interest to follow politics (Downs, Reference Downs1957). This discussion shortcut is not equally valuable to all people in all contexts (Sokhey and McClurg, Reference Sokhey and McClurg2012). If an individual has no information about how to vote, then discussion can be quite helpful (Ryan, Reference Ryan2011)—but, on the other hand, if an individual already has correct prior beliefs, additional information can either make no difference or harm the quality of the decision (Lupia and McCubbins, Reference Lupia and McCubbins1998). Of particular concern is the possibility that people will wrongly follow the advice of their discussion partner and “incorrectly” vote for the candidate the discussion partner prefers. For this reason, Cialdini (Reference Cialdini1984) describes social influence as a shortcut that leaves people “vulnerable to the attacks of profiteers who lie in wait along its path” (117).

Aware of this risk, Downs (Reference Downs1957), and many others since (e.g., Crawford and Sobel, Reference Crawford and Sobel1982; Lupia and McCubbins, Reference Lupia and McCubbins1998), argue that people should only believe information from peers with similar political preferences. For example, one should only trust a Republican's recommendation when they advocate voting for a Democratic candidate, as this is against their typical preferences (Calvert, Reference Calvert1985). This signal is only useful, however, to non-Democrats because Democrats would have chosen their party's candidate anyway. Still, while many will ignore a self-serving recommendation, some boundedly-rational voters will mistakenly follow a discussion partner's advice and choose the “wrong” candidate (Crawford, Reference Crawford2003; Ryan, Reference Ryan2013). If women are more open to persuasion, then, it is possible that they are also more open to making this voting error.

We next consider which is more common: discussion helping or harming decision making. Established democracies, with stable political parties and predictably-timed elections, provide individuals who have some information—even a little—with the means of making sensible vote choices (Sniderman, Reference Sniderman, Lupia, McCubbins and Popkin2000). This means that most of the time people can use established information—like party cues—to make correct decisions without any information particular to the current election. For this reason, previous research has found little aggregate benefit from social communication (Jackman and Sniderman, Reference Jackman and Sniderman2006; Ahn and Ryan, Reference Ahn and Ryan2015)—while some people may profit from it, others are instead hurt by it. And, in general, most people do not even need it to make a correct decision.

There are situations, however, when one should not support the candidate who they would typically back. For example, while a candidate may match an individual's issue positions, if the candidate is involved in corruption, they may actually be a poor choice. And, when it is the case that the candidate one typically supports is a poor choice, men's unwillingness to use social communication will hurt their ability to make correct decisions. If women in the same situation, on the other hand, are able to receive social information from sources with similar preferences, they will instead benefit from social communication.

In this discussion our focus is on individual choice, without making the assumption that every individual who selects between two candidates will ultimately take a costly vote (Krupnikov and Ryan, Reference Krupnikov and Ryan2017). Certainly, the process that leads an individual to make a choice is likely linked to the process that may ultimately lead them to either pay the costs of voting or abstain entirely. The extent to which people are persuadable can influence not only how they come to make a choice, but how they perceive the effectiveness of that choice and ultimately, whether they are confident enough in the choice to pay the costs of voting. Nonetheless, although social information may have these downstream consequences for turnout as an act in itself, the costs of the vote serve as a backdrop: our key interest is in adjudicating the role social information can play in an individual's decision making process.

3. Experimental design

To examine how gender affects social communication and voting, we utilize an incentivized experimental framework as used in previous studies (Ahn et al., Reference Ahn, Huckfeldt and Ryan2014; Kanthak and Woon, Reference Kanthak and Woon2015). This design is advantageous for the types of questions we are asking, because it allows us to know the information voters possess prior to social communication, observe the signals voters send to each other, and evaluate the quality of the final choice in light of the information they have received.Footnote 1

Another advantage of this experiment is that discussion partners are randomly assigned overcoming two key issues in studies of social influence. First, in observational studies of influence it is difficult to separate out influence from homophily (Achen and Shively, Reference Achen and Shively1995). If two friends agree, it could be that one convinced the other or it could be that they are friends because they agree. Second, men and women form their interpersonal networks differently because of the gender dynamics present in larger society (Huckfeldt and Sprague, Reference Huckfeldt and Sprague1995). Random assignment of discussion partners overcomes both of these problems.

The limitation of these types of experiments is that they do not allow subjects to change their opinions regarding their ideological placements—the subjects' preferences are chosen for them and remain fixed throughout the experiment. In contrast, elections outside the lab allow for social communication to persuade individuals that they were wrongheaded in their previous beliefs, but this particular experiment cannot account for that. The assumption of fixed preferences is necessary, however, because, as Ryan (Reference Ryan2011) points out, “Without this assumption, it is impossible to tell when a voter was duped into voting in favor of someone else's interests instead of her own and when she rationally updated her opinions” (756).

3.1. Subjects

Subjects are 63 students at a German university and 35 students at a US university.Footnote 2 Subjects received their earnings from the experiment as well as a € 5 or $10 show up fee. In all analyses, we include a dummy variable coded 1 if the subject is from Germany and 0 if the subject is from the United States.

Exactly half of all the subjects were women (53 percent of the German subjects and 46 percent of the American subjects). All the interactions were anonymous, allowing us to isolate whether men and women respond differently to social signals regardless of the gender of the person sending the signal. Note that we also ask subjects for their gender after the experiment reducing the probability that we observe gender differences because of “stereotype threat” (Spencer et al., Reference Spencer, Steele and Quinn1999).

Subjects participate in the experiment in groups of seven and are voters deciding between two computer-generated candidates. Voters and “candidates” are placed on the same seven-point preference scale, which is abstract and lacks any issue content. In each period, voters are paid based on their proximity to the winning candidate.

3.2. The voters

Each of the seven subjects has a unique integer position on a seven-point policy scale. The policy scale is completely abstract and has no actual issue content, allowing us to randomly assign subjects to any point on the scale.Footnote 3 Voter positions remain fixed throughout the experiment, and all subjects know their own position, the position of anyone with whom they are communicating, and the fact that there is one subject at each point of the scale.

Voters are randomly assigned to either be uninformed or informed. Informed voters have more information about the candidate's positions on the scale. They send signals about how to vote to uninformed voters. We will explain this in greater detail in a later section.

3.3. The candidates

In the experiment, subjects are placed on the same scale as two computer-generated candidates: Candidate A and Candidate B, who have randomly chosen positions. Candidate A's position is somewhere between 1 and 5 while Candidate B's position is somewhere between 3 and 7. Thus, on average, voters with positions less than 4 should prefer A; voters with positions greater than 4 should prefer B; voters at position 4 should be indifferent. This is public knowledge to all voters.Footnote 4 Candidate positions are randomly redrawn at the beginning of each period.

3.4. Payoffs

Subjects' payoffs for a period are based on the absolute distance between their position and the position of the winning candidate (note: how the winning candidate is chosen will be discussed below). Subjects are paid in tokens with 200 tokens equaling € 1 or $2. Subjects are paid for two randomly chosen periods, and they are unaware of which periods are chosen until the end of the experiment.

If the subject and the winning candidate hold the same position, the subject earns 400 tokens (€ 2 or $4). For every unit difference between the subject and the winner, the subject earns 50 tokens less. The smallest payoff a subject can receive is when the subject is at one extreme and the winner is at the other. In this case, the subject and winner are six units apart and the subject earns 100 tokens (400-6*50).

3.5. Signals from candidates

At the beginning of each period, the computer makes an independently random choice regarding the clarity of the signal each candidate sends. Figure 1 shows each of the three possible configurations: (A) both candidates are precise; (B) one candidate is precise and one candidate is ambiguous; (C) both candidates are ambiguous.

Figure 1. Signals from candidates for different levels of ambiguity. (a) Both candidates precise. (b) One candidate ambiguous. (c) Both candidates ambiguous.

If the candidates' positions are precise, then informed subjects are told what the candidate's position is. If the candidates are ambiguous, then informed subjects are shown a range of three positions and told that the candidate's position is equally likely to be any of those positions. This is shown in Figure 1B where Candidate A's position is precisely known and Candidate B's position is ambiguous.

3.6. Voter information

At the beginning of each period, subjects are randomly assigned to be either informed or uninformed. “Informed voters” see the signals shown in Figure 1. In each period, either 3 or 4 subjects are assigned to be informed.

The other subjects, the “uninformed voters,” are simply reminded of the ex-ante probabilities of candidate positions. Although we use the term “uninformed” for ease of writing, these individuals do have the type of information that is similar to a voter who is informed only of a candidate's partisan position. Note that at the beginning of each period, subjects are again randomly assigned, and so subject information levels may change between periods.

Each uninformed voter (the message receiver) is randomly assigned an advisor (the message sender) from among the informed subjects.Footnote 5 Senders and receivers know each other's positions.

Informed senders then recommend one of the two candidates to the uninformed receiver. They also state how confident they are in that recommendation. This is the only signal that uninformed voters will receive regarding the positions of the candidates—the receivers are not informed about the clarity of the candidates' signals that the sender observed. Hence, they do not know how certain the sender should be about which candidate they each should support.

It is important to note that because information levels and senders are reassigned each period, there is little possibility for senders to build a reputation for being either truthful or misleading. And, as we stated previously, all interactions are anonymous.

3.7. The election

All subjects choose which candidate they prefer. After making that choice, they can pay 25 tokens to vote. We distinguish between the choice and the vote as in an election outside the lab it is costless to decide between two candidates, but it is more costly to cast a vote (Krupnikov and Ryan, Reference Krupnikov and Ryan2017). The winner of the election is the candidate that receives the most votes cast, and if the election ends in a tie the winner is chosen randomly. Further, whether or not they voted, all subjects are paid based on the position of the winning candidate, with the reward increasing as proximity increases.

3.8. Stages of the experiment

Subjects participate in 15 periods. At the start of each period, candidate positions, clarity levels, voter information levels, and senders are randomly chosen. Each period proceeds as follows.

  • Stage 1. Informed voters are shown the candidate signals. Uninformed voters are reminded of the ex-ante probabilities of candidate positions. All voters are asked, “At this point, which candidate do you prefer?” and they must choose between A and B. They are also asked, “How confident are you in this choice?” Voters can say, “Very confident,” “Confident,” “Somewhat confident,” or “Not at all confident.”Footnote 6

  • Stage 2. Informed voters are told they are to provide information to an uninformed voter. They are told of that voter's position. They provide a vote recommendation (A or B) and also say how confident they are in that recommendation with the same options as before.

  • Stage 3. Uninformed voters receive the vote recommendation from the informed voter and they are told how confident the informed voter said they were in that recommendation. They are also told the informed voter's position. All voters are asked, “if you pay to vote, for which candidate do you vote?” and must choose Candidate A or Candidate B.

  • Stage 4. All voters are asked if they wish to pay 25 tokens to vote for the candidate they chose in Stage 3.

  • Stage 5. The candidate that receives the majority of votes actually cast wins the election. If there is a tie, a winning candidate is randomly chosen. Subjects then learn payoffs for the round.

Subject positions do not change between periods, but candidate positions, information levels, and senders are all randomly reassigned at the beginning of each period. Following 15 periods, subjects participate in a short survey before they are paid their earnings.Footnote 7

Prior to the experiment described above, subjects participate in ten periods in which they are all informed and there is no information sharing between subjects. The goal is for subjects to understand the incentives central to the main experiment. In these periods, men and women made the “correct” candidate choice—i.e. chose the candidate most closely aligned with their assigned positions—with exactly the same likelihood (87.96 percent). Thus, we have no reason to believe that any differences in the main experiment are the result of men and women having different levels of understanding or ability.

3.9. Hypotheses

The design has an underlying theoretical framework, but the goal of this experiment is not to test whether subjects behave rationally. In the language of Smith (Reference Smith1982), this is a “heuristic” experiment examining the consequences of unexplored experimental treatments. We will use the underlying theoretical framework in explicating our hypotheses about how subjects will behave.

  • H1. Women receivers are more likely to use social information than male receivers.

In the theoretical discussion, we note the literature suggesting women are more open to influence due to socialization, but there are other possible mechanisms. For one, women may accept more social information because women are significantly less likely to believe they are political experts and more likely to overreport the political expertise of others in their network (Djupe and Sokhey, Reference Djupe and Sokhey2014). Also, it is possible that women accept social information because they are more trusting, as they are more likely than men to trust discussion and information sharing partners (Maddux and Brewer, Reference Maddux and Brewer2005).

  • H2a. When the likelihood that sender and receiver support different candidates is low, then women are more likely to vote correctly than men.

  • H2b. When the likelihood that sender and receiver support different candidates is high, then men are more likely to vote correctly than women.

Without any social information, receivers should be able to vote correctly 87 percent of the time based solely on the distribution of possible candidate positions.Footnote 8 This means that in most situations the receiver's prior candidate choice will be the correct one. Social information can aid a receiver if it correctly informs him or her that the candidate he or she typically should support offers a lower benefit. Social information can harm a voter if it suggests a receiver should change his or her vote when the initial belief is correct.

We assume that some senders will choose to send signals that are self-serving—that is, the sender will suggest the receiver vote for the candidate that will provide the sender with a larger benefit regardless of whether that is the candidate who will provide the larger benefit to the receiver. When the probability that sender and receiver support different candidates is low—a variable we will call Ex-Ante Conflict—the signal should more frequently recommend the correct vote since even a purely self-interested sender will want the receiver to vote correctly. When the probability that sender and receiver support different candidates is high, then a self-interested sender will often want the receiver to vote incorrectly. Hence, they might send a misleading signal to the receiver.Footnote 9

This implies that when a sender tells a receiver that their initial beliefs are incorrect, then this is more likely to be true if the likelihood the sender and receiver support different candidates is low. Hence, when the sender and receiver probably do not support different candidates, then women can use social information to update incorrect beliefs. However, when sender and receiver likely do support different candidates, then men's tendency to ignore social information protects them from making a mistake by believing misleading information.

Note that we use the terms “correct” and “incorrect” vote as shorthand for selecting a candidate that best aligns with a participant's assigned preferences of that round, following Lau and Redlawsk (Reference Lau and Redlawsk1997). The goal is to ease the discussion of our results, rather than offer a stringent classification of votes.

4 The social signals

Before evaluating the hypotheses, we first examine an assumption central to the argument: senders will send more accurate social signals when they typically support the same candidate as the receiver. The key variable, therefore, is Ex-Ante Conflict—the probability the sender and receiver support different candidates.Footnote 10 Across all sender–receiver pairs, receivers would vote for the candidate offering the larger benefit 67.6 percent of the time if they followed the sender's signal.Footnote 11 If the sender and receiver support the same candidate a majority of the time, then the sender's signal is correct 81.6 percent of the time. If they support different candidates a majority of the time, the probability of an accurate signal drops to 58.4 percent. Importantly, this means that a sender's signal is accurate the majority of the time even when the sender would prefer the receiver votes for a different candidate.

However, this makes social information appear more valuable than it actually is. Remember, in expectation, receivers can vote correctly 87 percent of the time if they only consider the distribution of candidate positions. Subjects do a good job of using the distribution to come to decisions. Receivers initial belief's about which candidate to support are correct 84 percent of the time. Hence, if social information is correct only 67.6 percent of the time, then that means subjects are typically better off ignoring the social information.

Social information is going to be most useful when it corrects a subject's initial beliefs. For this reason, Figure 2 analyzes four possible scenarios: (1) both initial belief and social signal are correct; (2) the initial belief is correct, but the social signal is incorrect; (3) the initial belief is incorrect, but the social signal is correct; (4) both the initial belief and the social signal are incorrect. In scenarios 1 and 4, the social signal neither benefits nor harms the decision making of the receiver. In scenario 2, the receiver is worse off using social information, while in scenario 3 the social signal is helpful to the receiver.

Figure 2. Comparing initial choices and social signals by level of ex-ante conflict. Error bars represent 95 percent confidence intervals.

Figure 2 provides support for the key assumption. As Ex-Ante Conflict increases, the probability that the signal is potentially harmful increases and the probability that it is helpful decreases. Receivers are three times more likely to be in scenario 2, in which they would be worse off if they followed the social information, when Ex-Ante Conflict is greater than 0.5. They are least likely to be in scenario 3, when they should follow the social information, if Ex-Ante Conflict is greater than 0.5.Footnote 12

Another finding from Figure 2 is that, when Ex-Ante Conflict is low, the social signals are more accurate, but they do not necessarily help the receiver figure out which candidate offers the larger benefit. When Ex-Ante Conflict is low, the signal typically tells the receiver that their initial belief is correct.

If the social signal differs from the initial belief, then the social signal and initial belief are equally likely to be correct if Ex-Ante Conflict is less than 0.5. If Ex-Ante Conflict is greater than 0.5, then the social signal is five times more likely than the initial belief to be wrong. Hence, a receiver who decides to always ignore social signals will be better off than a receiver who always decides to follow social signals. A sophisticated receiver might only follow social signals when Ex-Ante Conflict is low (Lupia and McCubbins, Reference Lupia and McCubbins1998; Ahn et al., Reference Ahn, Huckfeldt and Ryan2010), but that heuristic will not result in a more accurate decision than the heuristic of always ignoring the signals in this experiment.

5. Vote choices

In H1, we state: receivers who are women are more likely to use social information than receivers who are men. We test this hypothesis with the first model in Table 1. The dependent variable is coded 1 if the subject chose the sender's recommended candidate and 0 if the subject chose the other candidate. The key independent variables are Woman Receiver (a dummy variable coded 1 if the subject is a woman and 0 if the subject is a man), Ex-Ante Conflict, Same Signal (a dummy variable coded 1 if the signal suggests the same candidate as the subject's initial beliefs and 0 if the signal suggests a different candidate). We also control for whether the subject was in Germany to account for possible differences between the German and American subjects and the period of the experiment in which the decision took place.

Table 1. Differences between how men and women use social information to choose which candidate to support

Logit models with standard errors adjusted to account for clustering on subjects. Dependent variable coded 1 if subject chose the sender's recommended candidate and 0 if subject chose the other candidate.

The results in Model 1 confirm H1. Holding all other variables at their means, the model predicts women will vote for the candidate suggested by the social signal 86 percent of the time while men will vote for the candidate suggested by the social signal 74 percent of the time.

While this confirms H1, it does not take into account factors that might condition the difference between men and women. When the sender's signal confirms the receiver's initial belief, we would not expect anyone to change their vote choice, meaning gender should not matter. As Ex-Ante Conflict increases, we would expect that everyone would be less likely to vote as the social signal suggests, but the slope might be different for men and women. Hence, the difference between men and women would probably be greatest when Ex-Ante Conflict is high and the sender's signal is different from the receiver's initial belief.

For this reason, we include a three-way interaction among our three key independent variables in Model 2. We plot the marginal effect of the receiver's gender in Figure 3.Footnote 13 The results confirm that the difference between men and women is significant only when the signal differs from the receiver's initial belief and if Ex-Ante Conflict is at least 0.3.

Figure 3. Are women more likely to use social information? Error bars represent 95 percent confidence intervals. Estimates taken from Model 2 in Table 1.

When we say women are more likely to use social information, this implies women are more likely to change their votes following social information. Men choose a different candidate following social information 13 percent of the time while women change their votes following social information 24 percent of the time.Footnote 14 In the next section, we determine to what extent this difference affects the probability that women choose the candidate that offers them the larger benefit.

6. “Correct” choices

Hypotheses H2a and H2b suggest that the relationship between gender and the likelihood of casting a “correct” vote is conditioned by the ex-ante preferences of the sender–receiver pair. H2a states that if Ex Ante Conflict is low, then we would expect women to vote correctly more frequently then men. On the other hand, H2b states that the relationship reverses when Ex Ante Conflict is high. Because of this conditional relationship, the logit model includes an interaction between Ex Ante Conflict and Woman Receiver.

The dependent variable is coded 1 if the receiver chose a candidate who offers a benefit as large or larger than the other candidate and 0 if the receiver chose a candidate who offers a smaller benefit. This definition of a “correct” vote in this case means the decision the receiver would have made with all possible information (Lau and Redlawsk, Reference Lau and Redlawsk1997). It does not mean, however, that the receiver necessarily made correct decisions from a game theoretic standpoint. For example, because of the uncertainty in the candidate positions and the crudeness of the signals, it is possible that a receiver believes a sender with similar preferences who attempted to send a correct signal and the receiver still votes for the candidate that offers a smaller payoff; we will consider this possibility in the next section.

We include the full model estimate in Appendix 5, but given our hypothesis we examine the marginal effect of Woman Receiver (Figure 4). We see some support for H2a as women vote correctly with greater frequency when Ex-Ante Conflict is 0 (p = 0.097). At the other end of Ex-Ante Conflict, we see stronger support for H2b. Women are 20 percentage points less likely than men to vote correctly when Ex-Ante Conflict is at its largest possible value (p = 0.001). This decrease is mostly the result of the decline in the probability a woman votes correctly as Ex Ante Conflict increases. When Ex-Ante Conflict is at 0, the model predicts women will vote correctly 89 percent of the time. When Ex-Ante Conflict is at 0.854, then the model predicts women will vote correctly only 67 percent of the time.

Figure 4. The conditional effect of gender on the probability of correct choices. Error bars represent 95 percent confidence intervals.

This is not the result of differences in the initial beliefs of men and women. The initial beliefs of men and women are almost identical (85 percent correct for men and 83 percent for women). The difference between men and women is the result of differences in the willingness of men and women to believe senders.

7. Additional considerations

We consider two additional perspectives. First, we stated earlier that the goal of this paper was not to determine whether subjects behave rationally. For this reason, we only looked at whether the receivers' candidate choices led to larger payoffs which could happen even if they made decisions that were different from the decisions made by someone who “solved” the game. We do not expect our subjects to actually figure out the game because there are too many moving parts to derive an analytical solution (Austen-Smith and Feddersen, Reference Austen-Smith and Feddersen2009). We can, however, follow Ahn et al. (Reference Ahn, Huckfeldt and Ryan2014) and compare subjects' decisions to decisions made using heuristics derived from simple cheap-talk communication models as well as pre-social communication benchmarks, taking into account that subjects do not have to pay to vote.

These analyses are in Appendix 6; we find that models that account for the “credible messenger” heuristic (Ahn et al., Reference Ahn, Huckfeldt and Ryan2014) (p. 104) lead to outcomes that are similar to those we observe with the “correct voting” measure. We find the same result if we consider what would happen if subjects ignored all social signals. Hence, gender affects the efficacy of social communication as an aid for uninformed voters in our experiment.

Second, there is a possible alternative explanation for women's incorrect choices: women receivers may follow their sender's signals more frequently because women are more pro-social (for a review, see Croson and Gneezy, Reference Croson and Gneezy2009). In Appendix 7 we explore whether women behaved more pro-socially by looking at the decisions women make when they are informed. We find that women are not engaging in altruistic decision-making.

8. Discussion

In Appendix 9, we re-analyze Ryan (Reference Ryan2011) which concludes, “within-subjects results cast doubt on the efficacy of political discussion…. Informed subjects…were hurt by social information” (p. 762-763), to consider gender differences. Among men, there is almost no difference between the likelihood of a correct vote with or without social communication. Among women, there is a 13 percentage point drop in the likelihood of a correct vote following social communication.

Hence, this paper's original experiment and a reanalysis of an already-published study lead to two conclusions regarding how gender conditions the efficacy of discussion as an information shortcut. First, women tend to incorporate information from social communication more frequently than men. Second, in many cases, this reliance on social information causes women to be more likely than men to vote for candidates who are less beneficial to them. We also, though, have weaker evidence that there are situations in which the behavior of women is beneficial. With a different design in which individuals needed social information to improve their decision making, the aggregate results may have reversed such that men would be less likely to vote correctly than women. One potential implication of this is that the strategy typically employed by men is beneficial in national elections in established democracies (Sniderman, Reference Sniderman, Lupia, McCubbins and Popkin2000), but women's behavior is preferable in new democracies or very low information elections—e.g., local elections (Boudreau et al., Reference Boudreau, Elmendorf and MacKenzie2015).

Like all studies, we reach these conclusions after making a series of implicit assumptions. The most important assumption is that individuals have fixed positions. As we note, this assumption is necessary to both determine what is a correct vote and to understand when senders and receivers hold similar preferences (Baldassarri and Bearman, Reference Baldassarri and Bearman2007). Even if the assumption is necessary, though, it is important to consider whether or not the assumption is reasonable. It is certainly not reasonable if we are arguing that an individual's preferences are fixed on any particular issue, as scholars would characterize individuals' preferences on particular issues as ambivalent and unstable (Zaller and Feldman, Reference Zaller and Feldman1992). People do have, however, underlying core principles and partisan preferences which remain fairly stable even if voters will not admit to themselves what that partisan attachment is (Klar and Krupnikov, Reference Klar and Krupnikov2016). In fact, one reason that individuals are ambivalent on particular issues is because some issues can put a core principle in conflict with other core principles or their partisanship (Lavine et al., Reference Lavine, Johnston and Steenbergen2012). Our model of social communication with fixed preferences is more about underlying principles and partisanship than preferences on any particular issue.

Both experiments also assume that individuals can identify the political preferences of the people with whom they are speaking. The existing survey research suggests this assumption is warranted, but not ironclad. For example, individuals will incorrectly identify their spouse's preference for president, perceiving disagreement when there is none (Mendez and Osborn, Reference Mendez and Osborn2011). People also have an easier time determining the preferences of both strong partisans (Huckfeldt et al., Reference Huckfeldt, Levine, Morgan and Sprague1998) and individuals with whom they talk about politics frequently (Eveland and Hutchens, Reference Eveland and Hutchens2013).

Further, our results speak most directly to situations in which a person is considering how to cast his or her vote. In election contexts, when party heuristics and media coverage provide information to voters, men's stubborn refusal to listen to others may serve them well in selecting a candidate who best aligns with their interests. Political contexts that include more deliberation—for example, legislative debates—are often much more ambiguous. In these contexts, the discussants may agree on broad goals and outcomes, but disagree on the means with which to achieve these outcomes. In these more ambiguous contexts, women's greater responsiveness to social information may prove highly beneficial—both in terms of matching their preferences to a decision and reaching compromise. Indeed, the same forces that lead women to be more influenced by social information also lead women to seek consensus during group debate and rely on more communal techniques when part of a group (Tannen, Reference Tannen1990).

Supplementary Material

The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2019.26

Acknowledgments

We thank Tiffany Barnes, Nichole Bauer, Olaf Bock, Chris Karpowitz, Michele Margolis and Ken Shotts, as well as Vera Troeger for her guidance as editor. We also thank participants at the International Society of Political Psychology, Behavioral Models of Politics and the Yale CSAP American Politics Conference for very helpful feedback.

Footnotes

1 While there are other conditions that may affect choice, here we control for these conditions to focus on establishing whether gender differences exist.

2 We recruited the German subjects using the HRoot recruitment system (Bock et al., Reference Bock, Baetge and Nicklisch2014)while the ORSEE recruitment system was used at the American university (Greiner, Reference Greiner, Kremer and Macho2004). The experiment is programmed using zTree—software for running economic experiments (Fischbacher, Reference Fischbacher2007). On both campuses, the experiment was conducted in English. See Appendix 1, for instructions. We note that the subjects recruited in Germany knew the study would be in English (see Appendix 8).

3 The mean absolute distance from the median position of 4 was 1.73 for women and 1.69 for men.

4 The process by which candidates are drawn and the probabilities associated with the candidates is included in Appendix 1

5 In periods with more uninformed subjects than informed subjects, one sender is randomly assigned to two receivers. The sender can send different signals to each receiver.

6 Among informed subjects, men say they are “very confident” 62 percent of the time while women say the same 58 percent of the time. Among uninformed subjects, the percentages are 43 percent for men and 36 percent for women.

7 Appendix 2 has information on how long each period took on average.

8 In US presidential elections, voters could vote correctly based on party alone 85 percent of the time (Lau et al., Reference Lau, Andersen and Redlawsk2008).

9 A sophisticated receiver should ignore signals that appear self-serving (Lupia and McCubbins, Reference Lupia and McCubbins1998). Not all receivers are sophisticated; some will accept information they should not believe and some will ignore information they should believe (Crawford, Reference Crawford2003). Hence, a sender could convince an unsophisticated receiver to believe a misleading signal.

10 When either the sender or receiver is indifferent between the candidates' realized payoffs in a period, then the sender and receiver are not in conflict. Values of Ex-Ante Conflict for all possible sender–receiver pairs are in Appendix 3.

11 Gender differences in the probability of truthful signals are in Appendix 4.

12 Though, the difference is not statistically significant with the lowest level of Ex-Ante Conflict since the probability that the signal will ever be useful is so low.

13 While the three-way interaction term is not statistically significant, the lower AIC value indicates Model 2 fits the data better than Model 1. This could mean that a conditional relationship exists in regards to the predicted probabilities even with an interaction term that is not significant (Berry et al., Reference Berry, DeMeritt and Esarey2010). For this reason, it is imperative that we plot the marginal effects to interpret the model's results.

14 P-value = p = 0.02 in an adjusted Wald test.

References

Achen, CH and Shively, WP (1995) Cross-Level Inference. Chicago: University of Chicago Press.Google Scholar
Ahn, T and Ryan, JB (2015) The overvaluing of expertise in discussion partner choice. Journal of Theoretical Politics 27, 380400.10.1177/0951629814531672CrossRefGoogle Scholar
Ahn, T, Huckfeldt, R and Ryan, JB (2010) Communication, influence, and informational asymmetries among voters. Political Psychology 31, 763787.10.1111/j.1467-9221.2010.00783.xCrossRefGoogle Scholar
Ahn, T, Huckfeldt, R and Ryan, JB (2014) Experts, Activists, and Interdependent Citizens: Are Electorates Self-Educating? New York: Cambridge University Press.10.1017/CBO9781107706156CrossRefGoogle Scholar
Atkeson, LR and Rapoport, RB (2003) The more things change the more they stay the same: gender differences in political attitude expression, 1952–2000. Public Opinion Quarterly 67, 495521.10.1086/378961CrossRefGoogle Scholar
Austen-Smith, D and Feddersen, TJ (2009) Information aggregation and communication in committees. Philosophical Transactions of the Royal Society of London—Series B: Biological Sciences 364, 763769.10.1098/rstb.2008.0256CrossRefGoogle ScholarPubMed
Baldassarri, D and Bearman, P (2007) Dynamics of political polarization. American Sociological Review 72, 784811.10.1177/000312240707200507CrossRefGoogle Scholar
Barnes, TD (2016) Gendering Legislative Behavior: Institutional Constraints and Collaboration. New York: Cambridge University Press.10.1017/CBO9781316534281CrossRefGoogle Scholar
Berry, WD, DeMeritt, JHR and Esarey, J (2010) Testing for interaction in binary logit and probit models: is a product term essential? American Journal of Political Science 54, 248266.10.1111/j.1540-5907.2009.00429.xCrossRefGoogle Scholar
Bock, O, Baetge, I and Nicklisch, A (2014) hroot: Hamburg registration and organization online tool. European Economic Review 71, 117120.10.1016/j.euroecorev.2014.07.003CrossRefGoogle Scholar
Boudreau, C, Elmendorf, CS and MacKenzie, SA (2015) Lost in space? Information shortcuts, spatial voting, and local government representation. Political Research Quarterly 68, 843855.10.1177/1065912915609437CrossRefGoogle Scholar
Calvert, RL (1985) The value of biased information: a rational choice model of political advice. The Journal of Politics 47, 530555.10.2307/2130895CrossRefGoogle Scholar
Campbell, A, Converse, PE, Miller, WE and Stokes, DE (1960) The American Voter. Chicago: The University of Chicago Press.Google Scholar
Cialdini, RB (1984) Influence: How and Why People Agree to Things. New York: William Morrow and Company.Google Scholar
Crawford, VP (2003) Lying for strategic advantage: rational and boundedly rational misrepresentation of intentions. American Economic Review 93, 133149.10.1257/000282803321455197CrossRefGoogle Scholar
Crawford, VP and Sobel, J (1982) Strategic information transmission strategic information transmission. Econometrica 50, 14311451.10.2307/1913390CrossRefGoogle Scholar
Croson, R and Gneezy, U (2009) Gender differences in preferences. Journal of Economic Literature 47, 448474.10.1257/jel.47.2.448CrossRefGoogle Scholar
Djupe, PA, McClurg, SD and Sokhey, AE (2016) The political consequences of gender in social networks. doi: 10.1017/S0007123416000156.CrossRefGoogle Scholar
Djupe, PA and Sokhey, AE (2014) The distribution and determinants of socially-supplied political expertise. American Politics Research 42, 199225.10.1177/1532673X13486924CrossRefGoogle Scholar
Downs, A (1957) An Economic Theory of Democracy. New York: Harper Row.Google Scholar
Eagly, A and Wood, W (2011) Social role theory. In Lange, PAV, Kruglanski, AW and Higgins, ET (eds), Handbook of Theories of Social Psychology, vol. 2. Los Angeles: Sage, pp. 458476.Google Scholar
Eagly, A and Wood, W (2013) The nature-nurture debates: 25 years of challenges in understanding the psychology of gender. Perspectives on Psychological Science 8, 340357.10.1177/1745691613484767CrossRefGoogle ScholarPubMed
Eagly, A, Wood, W and Diekman, AB (2000) Social role theory of sex differences and similarities: a current appraisal. In Eckes, T and Trautner, HM (eds), The Developmental Social Psychology of Gender. New York: Taylor and Francis Group, pp. 123174.Google Scholar
Eveland, WP Jr. and Hutchens, MJ (2013) The role of conversation in developing accurate political perceptions: a multilevel social network approach. Human Communication Research 39, 422444.10.1111/hcre.12011CrossRefGoogle Scholar
Fischbacher, U (2007) Z-tree: Zurich toolbox for ready-made economic experiments. Experimental Economics 10, 171178.10.1007/s10683-006-9159-4CrossRefGoogle Scholar
Greiner, B (2004) An online recruitment system for economic experiments. In Kremer, K and Macho, V (eds), Forschung und wissenschaftliches Rechnen. Gottingen, Germany: Datenverarbeitung, pp. 7993.Google Scholar
Guadagno, RE and Cialdini, RB (2002) Online persuasion: an examination of gender differences in computer-mediated interpersonal influence. Group Dynamics: Theory, Research, and Practice 6, 3851.10.1037/1089-2699.6.1.38CrossRefGoogle Scholar
Huckfeldt, R and Sprague, J (1995) Citizens, Politics, and Social Communication: Information and Influence in an Election Campaign. New York: Cambridge University Press.10.1017/CBO9780511664113CrossRefGoogle Scholar
Huckfeldt, R, Levine, J, Morgan, W and Sprague, J (1998) Election campaigns, social communication, and the accessibility of perceived discussant preference. Political Behavior 20, 263294.10.1023/A:1024866630373CrossRefGoogle Scholar
Huckfeldt, R, Johnson, PE and Sprague, J (2004) Political Disagreement: The Survival of Diverse Opinions within Communication Networks. New York: Cambridge University Press.10.1017/CBO9780511617102CrossRefGoogle Scholar
Jackman, S and Sniderman, PM (2006) The limits of deliberative discussion: a model of everyday political arguments. The Journal of Politics 68, 272283.10.1111/j.1468-2508.2006.00405.xCrossRefGoogle Scholar
Kanthak, K and Woon, J (2015) Women don't run? Election aversion and candidate entry. American Journal of Political Science 59, 595612.10.1111/ajps.12158CrossRefGoogle Scholar
Karpowitz, CF and Mendelberg, T (2014) The Silent Sex: Gender, Deliberation, and Institutions. Princeton, NJ: Princeton University Press.Google Scholar
Kenny, C (1998) The behavioral consequences of political discussion: another look at discussant effects on vote choice. The Journal of Politics 60, 231244.10.2307/2648009CrossRefGoogle Scholar
Klar, S and Krupnikov, Y (2016) Independent Politics: How American Disdain for Parties Leads to Political Inaction. New York: Cambridge University Press.10.1017/CBO9781316471050CrossRefGoogle Scholar
Koenig, AM, Eagly, AH, Mitchell, AA and Ristikari, T (2011) Are leader stereotypes masculine? A meta-analysis of three research paradigms. Psychological Bulletin 137, 616642.10.1037/a0023557CrossRefGoogle ScholarPubMed
Krupnikov, Y and Ryan, JB (2017) Choice versus action: candidate ambiguity and voter decision making. Quarterly Journal of Political Science 12, 479505.10.1561/100.00016051CrossRefGoogle Scholar
Lau, RR and Redlawsk, DP (1997) Voting correctly. American Political Science Review 91, 585598.10.2307/2952076CrossRefGoogle Scholar
Lau, RR, Andersen, DJ and Redlawsk, DP (2008) An exploration of correct voting in recent U.S. presidential elections. American Journal of Political Science 52, 395411.10.1111/j.1540-5907.2008.00319.xCrossRefGoogle Scholar
Lavine, HG, Johnston, CD and Steenbergen, MR (2012) The Ambivalent Partisan: How Critical Loyalty Promotes Democracy. New York: Oxford University Press.10.1093/acprof:oso/9780199772759.001.0001CrossRefGoogle Scholar
Lodge, M and Taber, C (2013) The Rationalizing Voter. New York: Cambridge University Press.10.1017/CBO9781139032490CrossRefGoogle Scholar
Lupia, A and McCubbins, MD (1998) The Democratic Dilemma: Can Citizens Learn What They Need To Know? New York: Cambridge University Press.Google Scholar
Maddux, WW and Brewer, MB (2005) Gender differences in the relational and collective bases for trust. Group Processes and Intergroup Relations 8, 159171.10.1177/1368430205051065CrossRefGoogle Scholar
Mendez, JM and Osborn, T (2010) Gender the perception of knowledge in political discussion. Political Research Quarterly 63, 269279.10.1177/1065912908328860CrossRefGoogle Scholar
Mendez, JM and Osborn, T (2011) Two become one? Spouses and agreement in political opinions. American Politics Research 39, 783803.Google Scholar
Milita, K (2015) Election laws and agenda-setting: how election law restrictiveness shapes the complexity of state ballot measures. State Politics and Policy Quarterly 15, 119146.10.1177/1532440015575901CrossRefGoogle Scholar
Ryan, JB (2011) Social networks as a shortcut to correct voting. American Journal of Political Science 55, 753766.10.1111/j.1540-5907.2011.00528.xCrossRefGoogle Scholar
Ryan, JB (2013) An experimental study of persuasive social communication. Political Communication 30, 100116.10.1080/10584609.2012.737416CrossRefGoogle Scholar
Smith, VL (1982) Microeconomic systems as an experimental science. The American Economic Review 72, 923955.Google Scholar
Sniderman, PM (2000) Taking sides: a fixed choice theory of political reasoning. In Lupia, A, McCubbins, MD and Popkin, SL (eds), Elements of Reason: Cognition, Choice, and the Bounds of Rationality. New York: Cambridge University Press, pp. 6784.10.1017/CBO9780511805813.004CrossRefGoogle Scholar
Sokhey, AE and McClurg, SD (2012) Social networks and correct voting. Journal of Politics 74, 754761.10.1017/S0022381612000461CrossRefGoogle Scholar
Spencer, SJ, Steele, CM and Quinn, DM (1999) Stereotype threat and women's math performance. Journal of Experimental Social Psychology 35, 428.10.1006/jesp.1998.1373CrossRefGoogle Scholar
Tannen, D (1990) Gender differences in topical coherence: creating involvement in best friends' talk. Discourse Processes 13, 7390.10.1080/01638539009544747CrossRefGoogle Scholar
Zaller, J and Feldman, S (1992) A simple theory of the survey response: answering questions versus revealing preferences. American Journal of Political Science 36, 579616.10.2307/2111583CrossRefGoogle Scholar
Figure 0

Figure 1. Signals from candidates for different levels of ambiguity. (a) Both candidates precise. (b) One candidate ambiguous. (c) Both candidates ambiguous.

Figure 1

Figure 2. Comparing initial choices and social signals by level of ex-ante conflict. Error bars represent 95 percent confidence intervals.

Figure 2

Table 1. Differences between how men and women use social information to choose which candidate to support

Figure 3

Figure 3. Are women more likely to use social information? Error bars represent 95 percent confidence intervals. Estimates taken from Model 2 in Table 1.

Figure 4

Figure 4. The conditional effect of gender on the probability of correct choices. Error bars represent 95 percent confidence intervals.

Supplementary material: PDF

Krupnikov et al. supplementary material

Online Appendix

Download Krupnikov et al. supplementary material(PDF)
PDF 570.4 KB
Supplementary material: Link

Krupnikov et al. Dataset

Link