Despite the use of the secret ballot in the United States since the late 1800s (Wigmore Reference Wigmore1889; Bishop Reference Bishop1893; Evans Reference Evans1917; Fredman Reference Fredman1968), misperceptions about ballot secrecy among the American electorate, and among nonvoters in particular, remain prevalent (Gerber et al. Reference Gerber, Huber, Doherty, Dowling and Hill2013b, 541, figure 1). Doubts about ballot secrecy appear to depress political participation (Gerber et al. Reference Gerber, Huber, Doherty and Dowling2013a; Gerber et al. Reference Gerber, Huber, Doherty, Dowling and Hill2013b) and may also undercut the perceived legitimacy and fairness of democratic elections in the United States (Claassen et al. Reference Claassen, Magleby, Monson and Patterson2008). Consequently, researchers have sought to assess whether and under what conditions providing information about ballot secrecy protections increases turnout among recently registered nonvoters, which prior research has defined as individuals who have never voted in any prior election, who had registered to vote since the general election six years prior, and had not ever voted, including not voting in at least one high-salience presidential election.Footnote 1 Recent debates have focused in particular on whether message effectiveness depends on the credibility of the actor delivering a message (see e.g., Hovland and Weiss Reference Hovland and Weiss1951–1952; Lupia Reference Lupia1994; Druckman Reference Druckman2001; Malhotra, Michelson and Valenzuela Reference Malhotra, Michelson and Valenzuela2012). Malhotra, Michelson and Valenzuela (Reference Malhotra, Michelson and Valenzuela2012) explain the importance of source credibility in mobilization campaigns, arguing that governmental appeals to vote are more effective than nongovernmental appeals because citizens perceive governmental sources to be more credible and authoritative than nongovernmental sources.Footnote 2
As a first stage effort to understand whether message source matters, existing research has tested whether mobilization campaigns addressing ballot secrecy concerns are effective when delivered by nongovernmental groups. Prior research provides strong evidence about the effectiveness of governmental communications to increase participation by addressing misperceptions about ballot secrecy, but offers inadequate evidence about whether similar information provided by nongovernmental sources is effective at increasing participation. Analyzing data from a field experiment conducted in Connecticut in 2010, Gerber et al. (Reference Gerber, Huber, Doherty, Dowling and Hill2013b) show that there is statistically significant evidence that governmental communication (a letter from the Secretary of State) providing information about ballot secrecy protections boosts participation rates among recently registered nonvoters by about 3.8 percentage points (p<0.01, two-tailed; n=69,488) when compared with recently registered nonvoters in a placebo condition who received election-related mail from the Secretary of State that excluded any text addressing ballot secrecy. In a second field experiment conducted during the 2012 Wisconsin recall election, similar messages were sent by a private nonpartisan and nonprofit voter mobilization organization. Analyzing data from this second experiment, Gerber et al. (Reference Gerber, Huber, Biggers and Hendry2014) estimate that nongovernmental messages sent approximately four to seven days before Election Day increase turnout among recently registered nonvoters by 1.1 points (p=0.51, one-tailed; n=10,200) when compared with an uncontacted control group. The magnitude of this effect is substantively large relative to the effects of comparable nonpartisan and nonadvocacy GOTV mail treatments,Footnote 3 but this effect is sensitive to model specification and the experiment’s sample is modest in size and underpowered to reliably detect a 1 point effect. These findings suggest that nongovernmental ballot secrecy messages appear to increase turnout, but it is unclear if the estimate is representative. Additional replication is therefore warranted to understand whether this result is reliable or would occur in other electoral contexts.
To address this need, this essay reports results from two new field experiments. One involves over 280,000 subjects across six states, which is a sample at least 27 times larger than those in previous comparable field experiments. The other, smaller field experiment is paired with the first and involves over 30,000 subjects across six states. In both experiments, a private nonpartisan and nonprofit voter mobilization organization implemented a randomized direct mail voter mobilization campaign targeting recent registrants who had not previously voted with assurances about ballot secrecy protections. We evaluate whether these efforts increased turnout in the 2014 midterm general election. In doing so, we replicate prior research, contribute evidence about the effectiveness of nongovernmental voter mobilization campaigns that address potential ballot secrecy misperceptions, and address concerns about the reliability of statistical estimates from previous field experiments that arise because they are underpowered. The large sample size and multistate design also provide adequate statistical power to conduct exploratory analyses assessing whether the mobilizing effect of ballot secrecy messages varies across state political contexts with varying contest intensitiesFootnote 4 and electoral institutions.Footnote 5
We find that a voter mobilization mailer communicating assurances about ballot secrecy protections that is sent by a nongovernmental and nonpartisan source increases turnout by 1 percentage point in our primary experimental sample containing subjects under 55. This effect is very precisely estimated. In a separate second smaller experiment, we do not observe an effect on turnout among recently registered nonvoters over 55. We conjecture this is because among older registrants, those identified as recently registered nonvoters using administrative voter files are more likely to be misclassified ever-voters (i.e., individuals who have ever voted but are labeled as never having voted at their current registration address) than among younger registrants. We also find that ballot secrecy communications are effective at increasing turnout in the 2014 midterm general election among younger nonvoting recent registrants across states with varying election institutions and intensities. Our finding bolsters the claim previously made in the literature that mobilization efforts are more effective and less sensitive to variation in political context when the content of the mobilization appeal communicates information about the integrity of electoral institutions that is orthogonal to standard GOTV communications that simply provides information seeking to invoke instrumental or pro-social norms to induce participation.
We note, however, that the designs are limited in several important ways: they are unable to isolate the effects of providing information that correct misperceptions about ballot secrecy from the bundle of appeals contained in the treatment mailer and they do not provide leverage to causally assess differences in effects by governmental versus nongovernmental source. Moreover, the designs are underpowered to detect a statistically significant difference between the effect of the ballot secrecy treatment and the effect of a standard GOTV appeal on turnout. Accordingly, we conclude by discussing these limitations and directions for future research to address them. The remainder of this essay describes the design of these experiments, presents our main findings, and discusses the implications and limitations of our findings and directions for future research.
EXPERIMENTAL DESIGN
We analyze two field experiments that were designed and concurrently implemented during the 2014 midterm election by the Voter Participation Center (VPC), a nonprofit, nonpartisan organization dedicated to increasing the participation of historically underrepresented groups.Footnote 6 The VPC targeted a set of citizens believed unlikely to participate in the 2014 midterm general election in the absence of additional outreach. Specifically, they focused on unmarried women or minority (i.e., nonwhite) registrants for whom over 99 percent had no prior record of voting.Footnote 7 Both experiments were conducted in six states with varying election intensities and varying sets of electoral institutions at baseline: Arkansas, Georgia, Louisiana, Michigan, North Carolina, and Texas.Footnote 8 The two experiments tested similar voter mobilization mailers, providing recently registered nonvoters with information about ballot secrecy protections, but targeted distinct age subgroups. The first experiment focused on householdsFootnote 9 containing at least one person under age 55, whereas the second included only households where all members were age 55 or older.Footnote 10 Consistent with prior research that defines recently registered nonvoters as the population of interest, our analysis sample is restricted to include only recently registered nonvoters;Footnote 11 this yields 281,929 subjects (in 270,345 households) in the Under 55 experiment and 32,978 subjects (in 32,077 households) in the Over 55 experiment.
For both experiments, the VPC conducted restricted randomizationsFootnote 12 clustered at the household level and blocked by state.Footnote 13 In the Under 55 experiment, subjects are randomly assigned to a pure control condition or to one of five other treatment conditions. Of the five treatment conditions, three contained similar language that prominently addressed ballot secrecy concerns with slight variations in additional minor appeals to vote, and the other two encouraged the subject to visit a personalized URL with information about how to vote and contained minor language addressing ballot secrecy fears. In the Over 55 experiment, subjects are assigned to either a pure control condition or to a ballot secrecy treatment mailer similar to those in the Under 55 experiment.Footnote 14 We describe the treatment messages in greater detail in the following section. For both experiments, the outcome of interest is participation in the November 4, 2014 midterm general election as recorded in state-specific voter files; this variable is coded 1 if the registrant voted in the November 4, 2014 general election and 0 if not (i.e., did not vote or was no longer in the voter file).Footnote 15
Description of Treatment Mailings
The VPC mailed informational letters to subjects assigned to the treatment conditions. The letters were printed on official letterhead displaying the organization’s logo and name and emphasizing its status as a “non-government, nonprofit, and nonpartisan 501(c)(3) organization.” There are two classes of treatment letters mailed: (1) a letter providing information about ballot secrecy protections; and (2) a personalized URL treatment that directed subjects to visit a personalized website with information about the name and location of their polling place, what they will need in order to vote, and the candidates who will be on the subject’s ballot.Footnote 16 Both classes of treatment mailers remind the subject about the upcoming election and applies social pressure to vote by informing subjects that “You may be called after the election to discuss your experience at the polls.” The bulk and remainder of the treatment mailers differ in what they communicate.
Ballot secrecy treatments
The majority of the ballot secrecy letter text provides information to allay potential concerns about ballot secrecy and voter privacy at the polling place. The language used in this ballot secrecy treatment letter is very similar to the treatment language used in previous ballot secrecy studies,Footnote 17 and in fact, the bolded text is identical to that used in the previous field experiment conducted in Wisconsin. The contents of this letter are designed to mitigate three types of fears citizens might have about ballot secrecy. The first paragraph addresses concerns about an individual’s name being matched to their actual vote. The second paragraph addresses a concern about privacy at the polling station. The final paragraph attempts to relieve fears about intimidation from polling workers or campaigns when voting.
Personalized URL treatments
Two additional treatment conditions that only provide a limited amount of information about ballot secrecy were tested in the Under 55 experiment. The text of these conditions are identical, but one was delivered in the form of a letter and the other was a postcard. Instead of focusing on ballot secrecy, the letter invites subjects to visit a personalized link on the VPC’s website for information about their polling place and the candidates running for office. A subtle combination of ballot secrecy and social pressure language appears at the end of the letter. As such, any difference in the relative effectiveness of this treatment, compared with the other secrecy message, could be due to it having less secrecy content, applying more social pressure, and being delivered in a different way (e.g., by requiring the subject to use a computer to visit a website).
Causal Quantities of Interest and Estimation Strategy
We are principally interested in the causal effect on turnout of nongovernmental mobilization campaigns informing voters about ballot secrecy protections. Thus, we collapse the original six-category treatment variable into a three-category variable in our analysis of the Under 55 experiment such that subjects are assigned to either to any ballot secrecy message,Footnote 18 to any personalized URL treatment,Footnote 19 or to the control condition. For the Over 55 experiment, the treatment variable remains unchanged as a binary indicator of being assigned to a ballot secrecy message or to the control condition.Footnote 20
For each experiment, we specify the sample intent-to-treat (ITT) effect among households where all subjects are recently registered nonvoters and conditional ITT effects by state as our causal quantities of interest. We estimate these quantities using ordinary least squares with inverse probability weights (IPW), regressing turnout in 2014 on treatment indicators and baseline covariates that are prognostic of turnout.Footnote 21 The weights are the inverse of the subject’s probability of assignment to their observed treatment group. We estimate and report robust standard errors clustered at the household level. For conditional ITTs by state, we partition the sample by subgroup and re-estimate ordinary least squares with IPW excluding state fixed effects.
RESULTS
The Effect of Nongovernmental Information About Ballot Secrecy on Turnout
Table 1 presents regression estimates of the effect of sending a nongovernmental voter mobilization mailer addressing concerns about ballot secrecy protections on voting rates among recently registered nonvoters in the 2014 midterm general election for the Under 55 (left panel) and Over 55 (right panel) experiments. In both panels, we focus on the left-most column presenting estimates from the (preferred) base specification.Footnote 22
Table 1 Effect of Ballot Secrecy Interventions on Voting in 2014 Among Recently Registered Nonvoters
Note: Sample restricted to households where all subjects did not previously vote in 2010 or 2012 elections.
Standard errors in brackets.
GA=Georgia; HH = Household; LA=Louisiana; MI=Michigan; NC=North Carolina; TX=Texas.
*, **, *** Significant at 10, 5, 1%, respectively.
First, we report the baseline turnout rate, defined as the control group mean turnout level. In both experiments, baseline turnout rates are fairly low at 13 percent for the Under 55 experiment and 26 percent for the Over 55 experiment, which are consistent with expectations about low turnout in midterm election years and with expectations that turnout rates and age are positively correlated.
Next, we examine the ITT estimates. In the Under 55 experiment, sending a mailing from a nongovernmental, nonpartisan source on ballot secrecy protections increases turnout among recently registered nonvoters by 1 percentage point, from 13 percent in the control group to 14 percent in the ballot secrecy treatment group, or a 7.7 percent increase. This effect is precisely estimated (SE=0.002; p<0.01, two-tailed; n=281,929).Footnote 23 In contrast, sending the ballot secrecy mailing has no significant effect on turnout among recently registered nonvoters in the Over 55 experiment. Compared with the control group mean turnout rate of 26 percent, sending the ballot secrecy mailer decreases turnout by 0.3 percentage points (SE=0.008; p=0.69, two-tailed; n=32,978). However, this difference is not statistically significant.Footnote 24
Effects Across Political Contexts: State-Specific Estimates
Next, we briefly summarize estimates of the effect of nongovernmental communications about ballot secrecy protections on turnout rates by state.Footnote 25 Table 2 presents the estimated effect of the ballot secrecy treatment on turnout among recently registered nonvoters by state in each experiment. Only results from the base specification are presented.
Table 2 Effect of Ballot Secrecy Interventions on Voting in 2014 Among Recently Registered Nonvoters, by State
Note: Sample restricted to households where all subjects did not previously vote in 2010 or 2012 elections.
Robust standard errors in brackets.
*, **, *** Significant at 10, 5, 1%, respectively.
As the left panel shows, in the Under 55 experiment the ballot secrecy treatment significantly increases turnout among recently registered nonvoters in four of the six states. The right panel shows state-specific ITT estimates from the Over 55 experiment. The subsamples by state in the Over 55 experiment are generally too small to provide adequate statistical power to make useful inferences. In all but one state, we fail to reject the null hypothesis that the ballot secrecy treatment has no effect. These results suggest that ballot secrecy messages are generally effective at increasing political participation regardless of state political context.
DISCUSSION
In this essay, we report results from two large-scale field experiments that build on prior research to test the effects on turnout of nongovernmental and nonpartisan voter mobilization campaigns that communicate assurances about ballot secrecy protections. We present credible evidence that such a message sent from a nonofficial and nonpartisan voter mobilization group to recently registered nonvoters increases turnout by 1 percentage point in the 2014 general election among targeted subjects under 55. The estimated 95 percent confidence interval (CI) is (0.005, 0.014); thus the design is adequately powered to detect an effect at least as large as 0.5 points. We find no effect of such messages among recently registered nonvoters over age 55, which we speculate is due to the fact that older registrants identified as recently registered nonvoters (using conventional filtering methods applied to administrative voter records) may in fact be ever-voters who moved but have not yet voted at their new address and who are not prone to misperceptions about ballot secrecy. This finding begins to address recent debates about whether the effect of ballot secrecy treatments varies by source credibility.
We also provide the first exploratory analysis of heterogeneous effects by state political context and thereby provide insight into existing debates about whether effects vary by contest intensity and by the range of alternatives to Election Day voting available to registrants. Using a single field experiment testing the same treatments across six states, we show that there are fairly consistent positive effects of ballot secrecy treatments on participation levels among recently registered nonvoters. These results strengthen our belief that the effects of nongovernmental GOTV mailers containing ballot secrecy assurances are sensitive neither to variation in the intensity of electoral contests nor to variation in the set of voting options available to citizens.
Finally, we briefly discuss limitations with this study and fruitful avenues for future research. First, given practical constraints, these designs are unable to isolate the effects of providing information correcting misperceptions about ballot secrecy protections from the effect of the bundle of appeals contained in the mailer urging the subject to vote and reminding subjects about the upcoming election. (We are also unable to directly compare the effectiveness of governmental and nongovernmental communication.) We briefly discuss two potential strategies to address this concern. One general strategy involves comparing the estimated effect of the ballot secrecy treatment on turnout from the Under 55 experiment to the effects of treatments containing very minor or no language addressing ballot secrecy concerns. We can make two such comparisons. First, in the Under 55 experiment, we can compare the effect of the ballot secrecy treatment (1 percentage point) to the effect of the personalized URL treatment (0.8 percentage points), which contains only a single clause addressing ballot secrecy concerns at the bottom of the mailer but is otherwise designed to emphasize a reminder to vote and to provide information about where and when to vote. We interpret this difference of 0.2 percentage points (SE=0.0014, p=0.24) as the additional effect of providing a strong message addressing ballot secrecy concerns over and above a reminder to vote and very minor language addressing ballot secrecy. While the magnitude of the estimated difference in effects of the ballot secrecy and personalized URL treatments is positive and substantively meaningful, it is not statistically distinguishable from 0. Second, we can compare the effect of the ballot secrecy treatment in the Under 55 experiment to the estimated effect on turnout of a GOTV mobilization mailer that only provides a reminder about the upcoming election and information about how to vote (without any social pressure or ballot secrecy language) among a comparable population in the same election from a separate, companion field experiment. Specifically, we analyze data from a separate field experiment conducted in Florida, Iowa, and Kansas by the same nonofficial and nonpartisan voter mobilization group, where subjects were randomly sent either an “information-only” GOTV mobilization mailer containing only a reminder about the upcoming election or no mailer. Importantly, the information-only GOTV message includes no social pressure language and no ballot secrecy language. Because the companion experiment targeted a broader population of interest than that for the Under 55 experiment, we restrict the definition of subjects to recently registered nonvoters in households containing at least one registrant under 55 in order to improve the comparability of the samples across studies. We find that sending the information-only GOTV mailer increases turnout levels by 0.7 points (SE=0.003; p<0.01, two-tailed; n=95,430) above a control group mean turnout level of 19.3 percent (in contrast, the control group turnout rate in the Under 55 experiment is lower at 13 percent).Footnote 26 Comparing the effect of the ballot secrecy treatment mailer on turnout from the Under 55 experiment (1 point) and the effect of the information-only GOTV mailer without ballot secrecy language from the companion experiment (0.7 points) suggests that there is an additional effect (of about 0.3 points) of the ballot secrecy appeals on turnout above and beyond the effect of the information-only GOTV appeal. However, such an observational comparison across field experiments does not provide strong leverage to interpret this difference in effects causally or to formally test the difference in effects. Taken together, these two comparisons (of the ballot secrecy treatment effect against the personalized URL effect in the Under 55 and against the information-only GOTV treatment effect in the companion experiment) suggest the possibility of a substantively meaningful difference in effects between the ballot secrecy treatment and the effects of GOTV treatments with much weaker or no ballot secrecy language. However, the experimental comparison we make is statistically underpowered and the observational comparison we make cannot be interpreted causally. Thus, we recommend that future field experiments should employ an adequately powered, placebo-controlled design to isolate the effect of providing corrective information about ballot secrecy, over and beyond the effect of providing reminders to vote, to a subpopulation who hold misperceptions about ballot secrecy protections.
Second, this study does not measure subjects’ baseline perceptions about ballot secrecy and does not validate whether the ballot secrecy treatment message corrects these beliefs. Thus, future designs should elicit these beliefs at baseline and post-treatment and compare the effect of the treatment on beliefs about ballot secrecy between subgroups with varying baseline concerns about ballot secrecy. Third and relatedly, we are limited in our ability to interpret the results for the Over 55 experiment because this study does not validate whether older subjects Over 55 possess fears and misperceptions about ballot secrecy at baseline that may be addressed and corrected by the treatment. Lastly, we are unable to formally test differences in message effects by messenger type and assess whether any differences that exist are due to differences in perceived source credibility. Thus, future studies should randomize subjects to receive the same message from either a governmental or nongovernmental source, and assess whether apparent heterogeneous effects by messenger type, if they exist, are attributable to the hypothesized channel of differences in perceived source credibility.
To summarize, we show that nongovernmental voter mobilization efforts that communicate assurances about ballot secrecy protections are effective at increasing participation among recently registered nonvoters, and that this effect holds across a range of state political contexts. Further research is needed to test hypothesized mechanisms explaining observed effects and to formally test heterogeneous effects by message source.
