In the United States, election forecasting has expanded from First Generation to Second Generation approaches. Footnote 1 The First Generation took hold in the early 1980s, and was dominated by a battle between structural modelers and pollsters (and to a lesser extent, the markets). The Second Generation, which had sunk deep roots by the 2012 presidential election, was dominated by Structuralists, Aggregators, Synthesizers, and Experts (as labeled by Lewis-Beck and Stegmaier 2014). The Structuralists continued in the tradition of issuing forecasts from static, single-equation explanatory models (e.g., Abramowitz Reference Abramowitz2014, Campbell 2014). The Aggregators departed from reliance on the polls of individual leading houses, instead freely combining many polls to come up with averaged, and dynamic, forecasts (e.g., Real Clear Politics). The Synthesizers joined structural models and poll aggregates to provide changing forecasts as Election Day approached (e.g., Erikson and Wlezien Reference Erikson and Wlezien2014; Linzer Reference Linzer2014). In contrast to this, The Experts, or Judges looked at whatever information they considered relevant, quantitative or not, arriving at forecasts shaped by current data and intuition (e.g., Cook and Wasserman Reference Cook and Wasserman2014; Rothenberg Reference Rothenberg2014).
The Second Generation work, at least some applications, produced successes without precedent when it came to presidential election forecasting. These successes have set the stage for breakthrough performances vis-à-vis congressional forecasting, an area long dominated by First Generation work (Abramowitz Reference Abramowitz2010; Campbell 2010). Of particular interest here is the Second Generation idea of combining methods (e.g., Erikson and Wlezien Reference Erikson and Wlezien2014, Graefe et al. 2014). In terms of the four methods described for the 2012 presidential election, the idea perhaps was most fully represented by joining structural modeling and poll aggregation. We propose to combine methods to generate House forecasts, uniquely bringing together structural modeling and expert judgment to create what we dub Structure-X models.
The notion behind Structure-X models is simple. First, we forecast with a sound structural model and then “adjust” that forecast by considering expert predictions. These adjusted forecasts, it can be shown, dramatically reduce the error incurred from a structural model used alone. In the next section, we lay out our preferred structural model, based on the tried-and-true theory of congressional elections as referenda on the performance of the presidential party. We diagnose the strengths and weaknesses of the referenda model. Then, we test it, estimating step-ahead forecasts of net presidential party seat change. Finally, we add forecast estimates derived from an expert, namely Stuart Rothenberg and his Rothenberg Political Report. Footnote 2 We compare the accuracy of our forecasts with and without the expert, demonstrating that the addition of expert judgments easily cuts error in half. We conclude that combining sound theoretical structure with good, expert opinion can dramatically improve forecasting power. As an ex ante test, we apply this Structure-X method to the forecast of the 2014 congressional races. That forecast calls for a net seat loss of 15 Democratic House seats.
A STRUCTURAL EXPLANATION OF HOUSE SEAT CHANGE: THE REFERENDUM MODEL
The development of election forecasting for US presidential elections has always carried a tension between explanation and prediction. This tension exists, too, within the work on congressional election forecasting. Our efforts on the House, from the beginning, have stressed the paramount importance of explanation, applying strong theories of congressional vote choice to forecast outcomes (Lewis-Beck and Rice Reference Lewis-Beck and Rice1984). Looking at our most extended treatment, we contended that “future improvements in election forecasting will come increasingly from explorations and testing of voting theory” (Lewis-Beck and Rice 1992, 141). The theory we draw on relies heavily on the pioneering work of Tufte (1978, 106), who saw these elections, especially at midterm, as “a referendum on the incumbent administration’s handling of the economy and of other issues.” Conceptually, we can state our model as follows
$$House Seat Change = Popularity + Economy + Midterm.$$
This simple model, which we call “structural” because of its theoretical foundation, grew out of the political economy literature that focuses on the sanctioning of governments at the ballot box. As we have stated elsewhere, “the variables in the models must measure, at least by proxy, what we know for sure about voter decision-making” (Lewis-Beck and Tien Reference Lewis-Beck and Tien2008, 230). The literature on the electoral impact of issues frequently cites the critical role of economy and popularity, although the measurement of these variables lacks consensus (Lewis-Beck and Stegmaier Reference Lewis-Beck and Stegmaier2013; Stegmaier and Lewis-Beck Reference Stegmaier, Lewis-Beck and Valelly2013). We posit the following operationalization of Equation (1):
$$House Seat Change = PresidentialApproval_{t - 1} + Disposable Income_{t - 1} + Midterm_t .$$
The estimates (ordinary least squares), with the precise measures and accompanying statistics, appear here,
$$\eqalign{ & S = - 45.25 + 0.83*P + 4.85*I - 29.81*M \cr & ( - 3.38) (3.29) (2.80) ( - 4.51) \cr & \cr} $$
R 2 = .60, adj. R 2 = .56, SEE = 18.5, D-W = 1.90, N = 33,
where S = presidential party seat change in the House of Representatives, I = change in real disposable income for initial six months of the election year (from the Bureau of Economic Analysis’s National Income and Product Account Table 2.6: Personal Income and Its Disposition), P = June Gallup poll presidential popularity rating from Gallup (2014), M = midterm dummy (0 = presidential election, 1 = midterm election), figures in parentheses are t-scores, * = statistical significance beyond .05, R 2 = coefficient of multiple determination, adj. R 2 = adjusted coefficient of multiple determination, SEE = standard error of estimate, D-W = Durbin Watson statistic, and N = the elections from 1948 to 2012.
To forecast ex ante seat change for 2014, we insert the appropriate independent variable values: I = 2.11, P = 44, and M = 1.
$$S_{2014} = S = - 45.25 + 0.83(44) + 4.85(2.11) - 29.81(1) = - 31 Democratic seats.$$
This referendum model, which we have tested ex ante over a series of contests, meets an important forecasting objective offered by Campbell (2004, 735), in his call for “model stability (the constancy of model specification from one election to the next).” However, other robustness diagnostics also should be consulted. For one, the coefficients are in the expected directions and readily achieve statistical significance. For another, the R-squared and the Standard Error of Estimate values indicate the patterns under study are far from random. However, the magnitudes of these fit statistics suggest that the model leaves the explanation of House seat change incomplete. We have earlier contended that the political economy core of this model is theoretically inviolate (Lewis-Beck and Tien Reference Lewis-Beck and Tien2008, 230). Although the theory embodied may be sound, it only goes so far. The variance left unexplained, for example, suggests there might be an omitted variables problem. Perhaps the experts, whom we now turn to, somehow tap into these omitted variables.
The question we pose comes down to this: Can our forecasts be improved by the addition of expert assessments?
THE REFERENDUM MODEL: ENLISTING EXPERT HELP
In the competition to determine congressional election outcomes, the experts have long been key participants. Careful observers such as Charlie Cook, Stuart Rothenberg, and Larry Sabato regularly examine House races district by district and make judgments about who has taken the lead. These experts often shun the word “forecast,” and they make it clear that their evaluations go well beyond the confines of modeling and statistics. Listen to Rothenberg (Reference Rothenberg2014), worth quoting at some length, about his method:
…we use qualitative judgments and general rules of thumb to base our analysis. In other words, our process can’t really be replicated…. My approach is decidedly qualitative, and therefore I use an ordinal scale of nine categories—Safe Democrat, Safe Republican, Democrat Favored, Republican Favored, Lean Democrat, Lean Republican, Toss-Up/Tilt Democrat, Toss-Up/Tilt Republican, Pure Toss-Up—to reflect my assessment of the relative vulnerability of seats…. [A]ssessments made 18 (or 12 or even 6) months before an election are not meant to be “predictions.” They simply reflect my judgment, at various points throughout an election cycle, of where races [are] headed.
The question we pose comes down to this: Can our forecasts be improved by the addition of expert assessments? As an heuristic exercise, we deploy Rothenberg’s “seats in play” variable, reported in May, June, or July of the election year. (If there were multiple reports, we aimed to take the June number, to parallel our own June measure. However, for 2006 we used May, and for 2010 it was July). Within these competitive seats, Rothenberg further breaks it down by calling seats either a “pure toss-up” or a “tilt” toward one party, or “favored” by one party. We simply used his total “seats in play” number, in making the “expert” (X) calculation. That is, we took the difference between the Democratic seats in play and the Republican seats in play (i.e., subtracted the president’s party number from the out-party number). For example, in June 2014 Rothenberg reports 26 Republican seats in play, versus 24 Democratic Party seats in play. Thus, the X-number = (26−24)=2. In other words, the expert forecast we use amounts to a pick-up of two House seats for the Democratic Party this fall.
Taken alone, this expert forecast (+ 2 ) differs sharply from our structural forecast (−31). This difference has a double concern because the expert forecast calls for a presidential party seat pick-up at midterm. Such a midterm bump for the president would be rare, historically, having occurred in recent times only in 1998 and 2002 (when the president was unusually popular). How can these different forecasts be reconciled? We propose a Structure-X model, which combines the two forecasts into one, simply by averaging the two estimates. In the case at hand, that means a June-based forecast of (−31 + 2)/2 = −14.5 net Democratic seat loss. If that forecast holds, then the White House party would sustain an overall loss, and at a level just slightly less than the historic norm for a midterm.
Does such a Structure-X model, in general, reduce forecasting error when compared to the Structural model alone? An answer comes from table 1, where step-ahead forecasts are offered for the four previous House elections, in addition to the upcoming 2014 contest. Take, as an example, the first contest in the series, the midterm election of 2006. (Earlier “seats at play” data are available from the Rothenberg Political Report, at least back to 1994, and we are in the process of assembling these data). The Structural model forecast an incumbent loss of −33, while the parallel Rothenberg number calculated an incumbent loss of −31. When these are averaged, it gives a Structure-X forecast incumbent seat loss of −32, yielding a prediction error of one seat.
This small number of experimental trials cautions us against bold claims. However, by implication, in the long run the combination of structural models with expert judgments could clearly lower forecasting error.
Table 1 Structure-X Model Forecast Performance
Where the Referendum model uses June Gallup approval, change in disposable income over the first six months of the election year, and a midterm dummy.
Rothenberg’s seats in play differential is calculated by subtracting the number of seats in play of the president’s party (as reported in The Rothenberg Political Report: May 2006, July 2010, and June all other years) from the number of seats in play for the out-party.
Making such a calculation for the subsequent elections, we observe (in the last column of table 1), the Structure-X forecasting error for each. Overall, that error, in absolute terms, averages about nine seats per contest [i.e., (1 + 17 + 19 + 0)/4 = 9.25.] By contrast, the average absolute error from the Structural model alone equals about 20 seats [i.e., (2 + 29 + 40 + 10)/4 = 20.25). Thus, observe that Structure-X reduces forecasting error by about one-half when compared to the Structural model alone (i.e., 9.25/20.25 ≈ ½). This small number of experimental trials cautions us against bold claims. However, by implication, in the long run the combination of structural models with expert judgments could clearly lower forecasting error.
CONCLUSIONS
Our investigations of congressional election forecasting has been inspired by reliance on strong voting theory as expressed in a parsimonious structural model viewing elections as referenda on the government. This referenda idea is strong, but not long. That is, its accounting does not reach far enough to render the prediction error negligible. It appears that more explanatory variables could be added to good effect. However, we are not sure about what those omitted variables might be. Therefore, to cope with this omitted variables problem, we turn to expert judgment to supplement our forecasts. In our experiment, we combine this expert forecast—X—with the structural model forecast, arriving at a unique six-month ahead prediction of the net incumbent seat change. We find, for the 2014 midterm election in particular, that the in-party Democrats stand to lose approximately 15 seats in the House. We conclude that, in general, the Structure-X forecasting approach, with its unique combination of methods, holds promise in the forecasting of future congressional contests.
