On May 7, 2013, former South Carolina Governor Mark Sanford, accused of state-funded secret trips to visit a mistress in Argentina and of violating his divorce decree by trespassing at his ex-wife’s home, completed a remarkable political comeback by winning a special election for US Congress in South Carolina’s First District by a 54% to 45% margin. That he could win at all was shocking to many, but perhaps the most surprising aspect of Sanford’s comeback was the ultimate ease of his victory in contrast to poll projections. In the weeks leading up to the election, three polls showed Sanford’s opponent leading by an average of 3%. Footnote 1 Sanford not only won the election, he also outperformed the polls by 12 points. Although there may be several explanations for this unexpected result, it appears than many people who voted for Sanford were unwilling to express this preference in the polls.
Moreover, Sanford’s experience is far from unique among congressional candidates accused of personal or financial scandal during recent election cycles. Several prominent anecdotes suggest that these scandals did not prove as decisive in the minds of voters as public polling anticipated. In 2006, Pennsylvania Representative Don Sherwood attracted national attention when he was accused of strangling his mistress. His opponent, Chris Carney, ran a television ad titled “Father,” in which a former Sherwood supporter holds up a photograph of his daughter and asks, “How can I tell her I support Don Sherwood and feel good about myself?” Footnote 2 Every poll projected that Sherwood would lose by double digits, but he lost by only 6%. Despite this narrow loss, it is clear that almost half of the voters in his district still “felt good about themselves” enough to vote for him in private. Two years later, both Republican Representative Don Young of Alaska and Democratic Representative Paul Kanjorski of Pennsylvania—accused of scandals involving oil-lobbyist bribery and earmarks to relatives, respectively—won reelection despite trailing significantly in all public polls.
These anecdotes suggest that perhaps voters actually are more willing to vote for a politician who is engulfed in scandal than they are willing to express in polls. This article provides evidence for this phenomenon, framing it within existing literature on polling bias in political campaigns.
SURVEY RESPONSE BIAS AND THE INCUMBENT RULE
Two categories of bias are of particular importance in evaluating the polling of scandal-tarred congressional candidates: (1) bias that generally benefits or hinders incumbents in legislative races; and (2) bias that may be specific to low-valence candidates, such as those accused of personal misconduct. This section reviews recent literature about the first category (i.e., “the incumbent rule”) because it has a potentially opposite effect than the hypothesized scandal effect. However, other well-known forms of polling bias, such as social-desirability bias, also may be implicated as potential causes of a scandal effect. Possible causal explanations for a bias against scandal-tarred candidates, although not tested in this article, are described in the discussion section.
The incumbent rule that undecided voters tend to break in favor of the challenger is deeply ingrained in the conventional wisdom of the media and political professionals. Among academics, the widespread popular acceptance of the rule is frequently mentioned but its empirical truth rarely is tested. Van de Ven, Gilovich, and Zeelennberg (2010, 568) refer to the incumbent rule as “a regularity in U.S. elections” and Kaufman, Petrocik, and Shaw (2008, 71) call it “the received wisdom of political consultants.” However, both references cite a single article from a practitioner’s newsletter (Panagakis Reference Panagakis1989) as the only published empirical support for the rule. If there is truth to this rule, most incumbents would find their support overstated in polls relative to their challenger.
The incumbent rule was named in a Polling Report article, in which Panagakis (Reference Panagakis1989) compiled a dataset of 155 polls showing that undecided voters broke predominantly for the challenger more than 80% of the time. Practitioners subsequently confirmed this rule and estimated its effect. Mellman (Reference Mellman2006) analyzed the 1994–2004 Senate races and determined that challengers overperformed their last public poll by an average of 3.6% (see also Bowers Reference Bowers2004). Yet, there is considerable disagreement in academic literature about the application of the incumbent rule to presidential elections, in which both candidates are well known and high valence. Some scholars found that early leads by both sides tend to fade and that polls more often are biased against the incumbent party (Campbell Reference Campbell2000; Erikson and Wlezien Reference Erikson and Wlezien2008; Kaufmann, Petrocik, and Shaw Reference Kaufmann, Petrocik and Shaw2008).
Most of these studies cannot determine exactly which voters are causing the incumbents to underperform their polling. Panagakis hypothesized that many voters actually are inclined to vote against the incumbent but do not know the challenger well enough to commit in the poll. Van de Ven, Gilovich, and Zeelennberg (Reference Van de Ven, Gilovich and Zeelennberg2010) explored a micro-foundational explanation for the incumbent rule in a laboratory setting and found that delays in making a choice lead to increased doubts about a focal option. This article does not attempt to evaluate any particular explanation for the phenomenon; however, it is important to acknowledge this effect as a possible interaction with the hypothesized bias toward scandal-tarred incumbents.
From the anecdotal evidence, we would expect to observe bias in comparing public polling with actual election results for races that involve incumbent candidates accused of personal or financial scandal; I refer to these candidates as “scandal-tarred.”
HYPOTHESES
From the anecdotal evidence, we would expect to observe bias in comparing public polling with actual election results for races that involve incumbent candidates accused of personal or financial scandal; I refer to these candidates as “scandal-tarred.” In these cases, we might expect to see an interaction between this bias and the incumbent rule. Thus, whereas most incumbents will do worse than anticipated by public polling, this trend will be reversed in the case of incumbents involved in scandal. The following two hypotheses summarize these anticipated effects:
-
• Scandal-Effect Prediction: Polls involving an incumbent candidate accused of scandal will underestimate the incumbent’s support compared to polls of races involving incumbents not so accused. That is, the effect of scandal on Incumbent Party Underperformance (IPU) is negative.
-
• Incumbent-Rule Prediction: Polls involving an incumbent candidate not accused of scandal will overestimate the incumbent’s support. That is, the effect of incumbency on IPU is positive. Footnote 3
To test these hypotheses, this article relies on polling and election data from six recent congressional election cycles.
ELIGIBLE RACES AND POLLS
The dataset draws from general-election races for the US Congress from 2002 through 2012—six cycles that present abundant available polling data and a cross section of partisan and scandal environments (scandal accusations were more frequent in 2006 and 2008; see Table A of the online appendix available on the PS website at http://dx.doi.org/10.1017/S1049096514001097). The dataset also includes only “competitive elections,” defined as those in which the challenging-party candidate received at least 40% of the two-party vote; this encompassed approximately 20% of House elections. Footnote 4 Races involving both incumbents and open seats were included as a baseline to compare the incumbent rule to bias due to scandal. However, races involving two incumbents running against one another in a general election, newly created seats, and those not held on a national election day were excluded.
I compiled publicly released, live telephone-interview polls that were completed within six weeks of the general election and that met certain methodological requirements (see the online appendix). The resulting dataset included 551 polls from a total of 256 races, which are summarized and described in detail in online appendix tables B and F, respectively. Polls for the 2002 and 2004 election cycles were gleaned from the “Poll Track” archive of the National Journal website; polls from 2006 through 2012 were gleaned from the archives of the Real Clear Politics website.
To determine whether an incumbent should be designated as “scandal-tarred,” I examined the summary of the campaign presented in the Almanac of American Politics. Footnote 5 If the race summary mentioned that the incumbent was accused of personal or financial scandal by his or her opponent or in prominent media portrayals, the incumbent was designated as scandal-tarred. However, this designation was applied only when the incumbent candidate was accused of personally having engaged in wrongful or scandalous activity. If the challenger merely charged that the incumbent’s party promoted a “culture of corruption,” this did not warrant the scandal designation.
The method produced a total of 29 competitive races with available polling involving scandal-tarred incumbents (see the online appendix), which accounted for 78 eligible polls. Of the 29 races, 22 occurred in 2006 or 2008 and 23 involved a Republican incumbent. To address this asymmetry, the online appendix includes an analysis for individual incumbent parties and years.
MEASURE OF BIAS
For each of the 551 polls, I evaluated how well the incumbent party's electoral performance predicts their performance in the polls, measuring how much the poll overestimated (or underestimated) an incumbent’s support by calculating an “incumbent poll margin” (i.e., the incumbent’s lead in the poll as a percentage of the two-party vote; it is a negative number if the challenger was leading) and an “incumbent vote margin” (i.e., the incumbent’s final election margin as a percentage of the two-party vote). I then subtracted the vote margin from the poll margin to calculate the IPU, which is the amount by which the incumbent party did worse than expected by the poll.
The following results were generated by examining the raw IPU scores and by regressing the IPU on a set of controls including incumbency and scandal dummies. The regression equation that predicts the IPU was modeled as follows for each poll p and congressional race r:
$${\rm{Incumbent Party Underperformanc}}{{\rm{e}}_{pr}} = {{\rm{\beta }}_{\rm{1}}}{\rm{(Incumbent Dumm}}{{\rm{y}}_r}{\rm{)}} + {{\rm{\beta }}_{\rm{2}}}{\rm{(Scandal Dummy Margi}}{{\rm{n}}_r}{\rm{)}} + {u_{pr}} + {v_r}$$
In this model, the u disturbance term represents the sampling error of the poll. The v r term represents the change in the race r between the time of the poll and election day, when this change was chaotic and not predictably biased toward either party. Footnote 6 To accommodate these two distinct error terms, the data were clustered by congressional race. For certain specifications, additional controls were added for the date on which the poll was taken and then interacted with incumbency and scandal for insight about the following two questions. First, when interacted with incumbency, this addresses the extent to which the “incumbent rule” is the result of the challenger simply gaining name recognition over the course of the last weeks of the campaign. If this were true, we would anticipate earlier polls to overestimate the margin for the incumbent more than later polls. Second, when interacted with scandal, this addresses one theory for the cause of bias: if bias were the result of scandals being prominent in the media in earlier points of the election, but fading from memory as the election approached, we would expect bias to decrease in polls close to the election.
For robustness tests, I also used specifications with interacted fixed effects for year and incumbent party and another specification to match polls involving scandal-tarred and non-scandal incumbents. Matching is a statistical technique used to create balance in observational data between treated and control groups. The method used in this study, Coarsened Exact Matching (CEM), assigns all units to discrete categories before matching the treated and control units. CEM has been recently advanced as providing improvements over other matching methods with respect to balancing the desire to reduce bias while retaining as much data as possible (Iacus, King, and Porro Reference Iacus, King and Porro2009). In this case, matching was used to achieve balance between the sample of polls of scandal races (i.e., the treated units) and those of non-scandal incumbents (i.e., the control units). This was done with respect to year, incumbent party, and competitiveness of electoral outcome, matching 73 of the 78 scandal polls to 104 non-scandal polls in categories constructed using these variables. Footnote 7 This analysis tests only for the scandal effect and does not seek to provide a direct test of possible explanations for the effect.
RESULTS
Preliminary evidence of a scandal effect can be observed first by examining the raw subgroup means for IPU. As shown in table 1, non-scandal incumbents underperformed by a mean of 3.5% over the sample of 295 polls, whereas scandal-tarred incumbents overperformed by an average of 3.2%—a difference of 6.7 points. Figure 1 displays a kernel-density function of IPU for the polls of scandal-tarred and non-scandal incumbents.
Table 1 Incumbent Party Underperformance: Scandal versus Non-Scandal
Figure 1 Kernel Density of Incumbent-Party Underperformance
Table 2 shows the results of regressing IPU on dummies for incumbency and scandal under several specifications. Table 3 limits the results to the matched sample, as described previously. For comparison purposes, column 1 in table 3 shows the unmatched sample with open seats excluded because polls of these races are all unmatched to scandal polls. Columns 2 and 3 replicate the results from the same column in table 2, using weights returned by the CEM algorithm. Returning to the two hypotheses, the incumbent rule predicts a positive coefficient for the incumbent dummy and the scandal effect predicts a negative coefficient for the scandal dummy.
Table 2 Regression Results of Model, Including Incumbency and Scandal
Notes: Errors are clustered by congressional race. ***p < 0.01, **p < 0.05, *p < 0.10 (two-tailed test). In Specifications (4) and (5), the excluded year and party fixed-effects categories are for 2012 and Democratic seats.
Table 3 Regression Results of Model, Including Scandal (Matched Sample with Open Seats Excluded)
Notes: Errors are clustered by congressional race. ***p < 0.01, **p < 0.05, *p < 0.10 (two-tailed test). Data in the Matched Sample model are matched on party, year, and incumbent vote margin, with scandal as a treatment variable using CEM.
In the first specification, when races are undifferentiated with respect to scandal, we observe that incumbents systematically underperform their polls by an average of 2.7%. Additionally, the constant is near 0, which suggests no bias with respect to the incumbent party in open seats. However, specification 2, which adds the scandal dummy, reveals an even stronger opposing scandal effect. The coefficients in this column suggest that polls of incumbents not accused of scandal overestimate incumbent support by 4.1%, whereas polls of scandal-tarred incumbents underestimate incumbent support by 2.6%; the total scandal effect is 6.7%. In this case, there is strong evidence of bias understating incumbent support in scandal-tarred races, as well as evidence of the incumbent-rule hypothesis (i.e., both effects are significant at p < 0.01, two-tailed).
Column 3 includes a polling-date variable, operationalized as days before the election that the poll was completed and ranging between 1 and 40. Here, there is little support for the theory that name recognition explains the incumbent rule; the coefficients for both date and date interacted with incumbency are close to 0, and no support for the causal theory that scandals fade in importance as the election draws closer. The coefficient for the date interacted with scandal is not significant and is in the opposite direction than would be posited by this causal theory.
These results are unchanged when fixed effects are included (columns 4 and 5) or open seats are excluded (i.e., column 1 in table 3); all show a scandal effect of 6% to 7%. Finally, these results are replicated in the smaller matched samples (i.e., columns 2 and 3 in table 3), showing a scandal effect of approximately 5%.
I tested the robustness of these findings in several ways. Online appendix table C lists the results of specification 2 for subsamples by incumbent party and individual year. The table shows that scandal has a significant negative effect (i.e., p < 0.01) in each subsample, which indicates understated support for scandal-tarred incumbents. I also examined specifications that included random effects for individual races, as well as those that incorporate controls and/or weighting for the sample size of individual polls and percentage of undecided voters in the polls, or limit the dataset by sample size or polling-firm quality. The effect of incumbency and scandal was substantively unchanged in all of these robustness checks (see the online appendix for complete results).
Concerns that the observed scandal effects are the product of collinearity between scandal and race competitiveness were addressed in three ways. First, the correlation between scandal and vote outcome was -0.10, which is not close to the levels that are statistically problematic. Second, all specifications were replicated with the poll result as the dependent variable, including the vote result as an independent variable with substantively identical results (see the online appendix). Third, the matching method used in these results matched polls on vote outcome; the correlation between scandal and vote outcome among the matched sample was 0.01. Footnote 8
DISCUSSION
This article exposes a bias in the polling of scandal-tarred candidates that is substantively large in the context of other observed polling biases as well as the actual campaign effects of scandal, while also finding robust evidence of the incumbent rule. Various studies have found that scandal accusations cost congressional incumbents between 5% and 10% of the general-election vote (Basinger Reference Basinger2013; Hendry, Jackson, and Mondak Reference Hendry, Jackson, Mondak and Mondak2009; Hirano and Snyder Reference Hirano and Snyder2012). My results suggest that about as many voters were induced by scandal accusations to incorrectly tell pollsters that they would oppose a candidate as were actually induced to vote against the same candidate.
My results suggest that about as many voters were induced by scandal accusations to incorrectly tell pollsters that they would oppose a candidate as were actually induced to vote against the same candidate.
Although the evidence that scandal-tarred incumbents overperform their polls is strong, aggregate-level data from public polling do not explain this phenomenon at the individual level. One possible explanation is social-desirability bias, which manifests as a bias in polling because people over-report socially desirable views and under-report undesirable views and behaviors in a conscious or subconscious effort to be viewed favorably by the pollster (e.g., Gibson, Hudes, and Donovan Reference Gibson, Hudes and Donovan1999; Tourangeau and Yan Reference Tourangeau and Yan2007; Wyner Reference Wyner1980). A well-known example in political polling is the Bradley Effect, wherein poll respondents express a preference for an African American candidate or an undecided preference when, in fact, they intend to vote for a white candidate, to conceal what could be perceived as a racist choice (see, e.g., Berinsky Reference Berinsky1999; Finkel, Guterbock, and Borg Reference Finkel, Guterbock and Borg1991; Hopkins Reference Hopkins2009). In this case, however, social-desirability bias might lead some voters—who genuinely support the policies or ideology of a scandal-tarred candidate—to suppress the public reporting of their support for fear that it will signal enthusiasm for socially undesirable behavior. Carney’s “Father” ad against Sherwood certainly played exactly to this sentiment by suggesting that voters should feel bad about telling others they voted for the scandal-tarred candidate. Also of particular relevance, Winters and Weitz-Shapiro (Reference Winters and Weitz-Shapiro2011) found that respondents in Brazil expressed less tolerance for political corruption in a survey than was indicated by their voting patterns; however, the researchers attributed this gap to a lack of voter information rather than bias in their own survey. Thus, whether electoral approval of corrupt or scandal-tarred candidates might be subject to social-desirability bias remains a largely open question. The literature suggests that automated voice polling may reduce the social-desirability bias (Tourangeau, Steiger, and Wilson Reference Tourangeau, Steiger and Wilson2001); the recent increase in Interactive Voice Response (IVR) polling in campaigns may provide a future avenue to test this theory. Footnote 9
Moreover, it is likely that the effect is not confined to US House races. Congressional elections have the advantage of being sufficiently numerous to observe a significant response bias in scandal races; however, the same patterns emerge in anecdotal evidence from statewide contests. For example, in 2006 and 2008, respectively, Senators Conrad Burns and Ted Stevens substantially overperformed their polls in narrow defeats despite multiple serious scandal allegations. Apparently, the basic ideological disposition of the electorate—not personal misconduct—again may have been more influential in the result than the polls anticipated.
The findings in this article have implications for both the future polling of elections involving scandal accusations and the evaluation of strategic entry and exit decisions of candidates involved in such elections. It is possible that some incumbents who chose to retire or resign based on poll evidence that a race was unwinnable due to scandal would have been reelected if the polls were calibrated appropriately. The results also suggest a possible expansion to the contexts in which we might observe social-desirability bias or priming, but additional exploration of these issues on an individual level is necessary. Thus, the research described in this article only hints at the several avenues that remain available to explore this phenomenon.
