In 2004, the PollyVote project (www.pollyvote.com) was launched with the goal of applying evidence-based principles to election forecasting, as well as to track different forecasting methods over time and in various settings to learn about their relative accuracy. The original idea of the project was to demonstrate the power of combining forecasts in reducing error, which is a well-established finding in forecasting research (Graefe et al. Reference Graefe, Armstrong, Jones and Cuzán2014).
Combining forecasts has two major advantages. First, the relative accuracy of different forecasting methods often varies across time. That is, methods that have worked well in one election might not work well in another, and vice versa. Combining forecasts thus reduces bias by protecting forecast users from choosing a poor forecast and instead enabling them to incorporate more information. Second, combining reduces error by canceling out systematic and random errors of individual forecasts, an effect that is particularly strong if the individual forecasts rely on different methods and data.
To date, the PollyVote has demonstrated the benefits of combining forecasts by providing highly accurate forecasts for the four US presidential elections from 2004 to 2016 (Graefe et al. Reference Graefe, Armstrong, Jones, Cuzán, Cavari, Powell and Mayer2017) and the two German federal elections in 2013 and 2017 (Graefe Reference Graefe2019). An analysis across the last 100 days before the seven US elections from 1992 to 2016 found that the PollyVote’s popular-vote forecast missed the final vote shares by 1.1 percentage points on average, which was lower than the error of any individual-component forecast (Graefe et al. Reference Graefe, Armstrong, Jones, Cuzán, Cavari, Powell and Mayer2017).Footnote 1
THE POLLYVOTE METHOD OF COMBINING FORECASTS
For its 2020 popular-vote prediction, the PollyVote uses a hierarchical approach to combining forecasts. The basic idea is to combine forecasts within and across four categories of methods: (I) vote intentions (polls), (II) expectations, (III) models, and (IV) naïve forecasts. Forecasts are grouped in these categories by the type of underlying data. The basic idea of this approach is to create conditions under which combining forecasts is most useful (Graefe et al. Reference Graefe, Armstrong, Jones and Cuzán2014). Whenever forecasts are combined, the PollyVote calculates simple unweighted averages. The forecasts from the PollyVote and its components are shown in table 1 (Graefe Reference Graefe2020b).
Table 1 Forecasts of the PollyVote and Its Components: Trump Two-Party Vote
Vote Intentions: Polls
When people speak of polling results, they commonly refer to respondents’ answers to the vote-intention question, which asks respondents for whom they would vote if the election were held today. Strictly speaking, polls do not provide forecasts of what will happen, particularly if the election is still far ahead. Rather, polls merely provide snapshots of public opinion at a certain point in time. The PollyVote incorporates these data by averaging daily estimates of three poll aggregators (i.e., FiveThirtyEight, RealClearPolitics, and The Economist).
Expectations
Methods that rely on people’s expectations elicit and aggregate their judgmental forecasts of what will happen on Election Day. The 2020 PollyVote combines forecasts from three different expectations-based methods: betting markets, expert judgment, and citizen forecasts.
Betting markets, also known as prediction markets, allow people to bet money on the election outcome, whereby the betting odds reflect a forecast of a given candidate’s vote share or probability of winning (Gruca and Rietz Reference Gruca and Rietz2020). A review of forecast accuracy of betting markets in different countries found that these markets tend to outperform forecasts made by polls, models, and experts, whereas relative performance compared to simply asking voters who will win (i.e., citizen forecasts) was mixed (Graefe Reference Graefe, Arzheimer, Evans and Lewis-Beck2017a). This review, however, did not include the 2016 US presidential election, when betting markets performed poorly (Graefe Reference Graefe2017b). The 2020 PollyVote combines forecasts from two markets that provide estimates of the popular vote: the Iowa Electronic Markets and Predictit.
Expert judgment is probably one of the oldest methods to predict election outcomes. Since 2004, the PollyVote has periodically asked a panel of experts to predict the popular two-party vote in US presidential elections. An analysis of the past four elections from 2004 to 2016 found that the forecasts of the majority of 452 individual experts correctly predicted the directional error of polls. That said, the typical expert’s forecast error was 7% higher than estimates of a polling average. The results also showed that the experts did not sufficiently incorporate information from so-called election fundamentals (Graefe Reference Graefe2018).
Citizen forecasts are based on simply asking survey respondents who they think is going to win (Murr and Lewis-Beck Reference Murr and Lewis-Beck2020). Although often overlooked and—unfortunately—rarely asked by pollsters, responses to this question are highly predictive of election outcomes, particularly when converted to vote-share forecasts. When analyzing data for the seven US presidential elections from 1988 to 2012, Graefe (Reference Graefe, Armstrong, Jones and Cuzán2014) found that citizen forecasts were more accurate than forecasts from polls, betting markets, and models. For example, compared to a typical poll, citizen forecasts decreased forecast error approximately by half (Graefe Reference Graefe2014).
Models
The 2020 PollyVote combines forecasts from various models, which are assigned to different categories depending on the underlying data. At the top level, the PollyVote distinguishes models that are purely retrospective, purely prospective, and those classified as mixed because they incorporate both types of information.
Retrospective models view an election solely as a referendum on the incumbent government’s performance by assuming that voters reward incumbents for good performance and punish them otherwise. Some models (e.g., Fair Reference Fair2009) measure performance by focusing exclusively on structural variables—for example, by using one or more macroeconomic variables (e.g., GDP and unemployment).Footnote 2 The PollyVote classifies these models, which completely ignore public-opinion variables, as fundamentals-only models. Other models (e.g., Lewis-Beck and Tien Reference Lewis-Beck and Tien2020) additionally incorporate poll-based variables (e.g., presidential job approval) as proxies for voters’ satisfaction with the incumbent’s handling of both economic and noneconomic issues. These models are classified as fundamentals-plus models. The PollyVote’s retrospective models’ component is calculated as the simple average of fundamentals-only and fundamentals-plus models.
Prospective models build on the idea that voters have different expectations of their personal—or the nation’s—future, depending on who will win. Therefore, voters are expected to vote for the candidate with whom they expect to be better off. The PollyVote combines forecasts from two models that fall into that category: the Big-Issue model (Graefe and Armstrong Reference Graefe and Armstrong2012) and the Issues and Leaders model (Graefe Reference Graefe2020a).
Mixed models are those that incorporate both retrospective and prospective variables. For example, any model that uses economic fundamentals as well as trial-heat polls would be classified as a mixed model. Examples include the Erikson and Wlezien (Reference Erikson and Wlezien2020) and Lichtman (Reference Lichtman2020) models and those published by media outlets (e.g., Economist and FiveThirtyEight).
Naïve Forecasts
Complexity tends to harm forecast accuracy. That is, the accuracy of simple models, such as a naïve no-change model, is often difficult to beat by more complex models. One reason for using a no-change model may be that a forecaster concludes from prior knowledge that the situation is stable. Another reason may be that, in many situations, prior knowledge is insufficient to improve on a no-change forecast (Green and Armstrong Reference Green and Armstrong2015). The 2020 PollyVote averages forecasts from two naïve models: Norpoth’s (Reference Norpoth2014) electoral-cycle model and a 50/50 model. Norpoth’s electoral cycle predicts the election outcome based on electoral history by using the vote of the two most recent elections as predictor variables in a multiple-regression model estimated on data from 1828 to 2016. The 50/50 model assumes that both major-party candidates will gain 50% of the popular vote and thus represents the age of political polarization. By including a naïve component, the PollyVote adheres to the principle of conservatism in forecasting by acknowledging the situation’s underlying uncertainty (Armstrong, Green, and Graefe Reference Armstrong, Green and Graefe2015).
THE POLLYVOTE 2020 POPULAR-VOTE FORECAST
As of August 21, 2020, the combined PollyVote predicts that Joe Biden will gain 52.1% of the popular two-party vote compared to 47.9% for President Trump. Given the PollyVote’s historical error, the estimated chance for Biden to win the most votes is 96%. Three of the four component methods are in agreement about Biden’s advantage, with poll aggregators being the most bullish for Biden (i.e., an almost 9-point lead). Only the naïve forecasts component predicts a narrow advantage for Trump, which in effect dampens Biden’s lead in the combined PollyVote forecast, thereby making the forecast more conservative.
DATA AVAILABILITY STATEMENT
Replication materials are available on Dataverse at https://doi.org/10.7910/DVN/RLECFV.
