Regression discontinuity design (RDD) has provided political science with an important tool to identify causal relationships between wining an election and a series of outcomes. For example, a vast literature has looked at the electoral effects of incumbency (e.g., Lee, Reference Lee2008 and De Magalhães, Reference De Magalhães2015). The conventional analysis of RDDs in these applications uses vote share as the running variable and relies on the assumption that the running variable is continuous, i.e., takes a continuum instead of a finite number of values. This assumption allows us to get arbitrarily close to the cut-off in order to estimate the local average treatment effect. This assumption does not hold, however, if the running variable is the number of seats in a legislature, as this variable has clear mass points at the integers. Thus, the results from the “continuity-based” canonical RDD setup are not applicable, nor do the standard smoothing methods apply without additional assumptions (Lee and Card, Reference Lee and Card2008; Kolesár and Rothe, Reference Kolesár and Rothe2018).
The use of the number of seats as a running variable is further complicated as there is no clear mapping between the underlying vote share—the preferred running variable for an RDD—and the number of seats allocated to each party. Folke (Reference Folke2014) is the first to discuss this issue in the context of proportional elections and their seat allocation rules. In our case, the underlying vote share is determined by district-level first-past-the-post races where the cut-off is known, but also where these district-level races can be aggregated in various ways and result in the same seat share (Feigenbaum et al., Reference Feigenbaum, Fouirnaies and Hall2017).
The contribution of this paper is to draw on recent theoretical work regarding the implementation and interpretation of RDD as a local randomized experiment (Cattaneo et al., Reference Cattaneo, Frandsen and Titiunik2015; Sekhon and Titiunik, Reference Sekhon and Titiunik2017) and use that work to study whether the number (or percentage) of seats can be directly employed as a running variable for RDDs. We apply the local randomization RDD framework to determine a specific window for our discrete running variables (the number and the percentage of seats). Furthermore, we use the underlying vote share to establish whether there were enough district-level races that were themselves unpredictable/unanticipated, so that we may think of the slim-majority versus slim-minority treatment status also as unpredictable/unanticipated.
Note that the method presented here can be directly applied to proportional elections. The discrete running variable remains the number of seats and the same window selection method applies. The equivalent to the test proposed here would be to check how close the vote share of each party was to the threshold that determines the allocation of an extra seat.
There are two necessary assumptions in order for the number (or percentage) of seats to be an appropriate running variable in the local randomization RDD framework. First, for a specific window around the cut-off, the value taken by the running variable (i.e., the number (or percentage) of seats) is considered to be randomly allocated. Second, we must impose an exclusion restriction: the size of the majority within this window (e.g., whether a majority of one or two seats) has no direct effect on the outcome of interest.Footnote 1 All that matters for the outcome is assignment of majority status, i.e., treatment. We see this is as a justifiable assumption as the posts of House Speaker and Senate President are assigned as soon as there is a one-seat majority.Footnote 2 A few more seats do not increase formal power over procedures. The exclusion restriction also requires that differences in the winning margins for each seat have no direct effect on the outcome of interest. We see this as justifiable as we find that almost all slim-majorities have some seats that were themselves the result of close district-level races.
Implementing the method suggested by Cattaneo et al. (Reference Cattaneo, Frandsen and Titiunik2015), we find that windows around the cut-off of two seats or 1 percent of seats in state Houses have enough observations and achieve balance of predetermined covariates. This indicates that RDD estimates within these windows can be interpreted as local randomized experiments. However, slim-majorities in state Senates cannot be interpreted as local randomized experiments, as the balance of predetermined variables is not achieved for any window.
We go beyond Cattaneo et al. (Reference Cattaneo, Frandsen and Titiunik2015) and propose a further test. We verify whether slim-majorities had enough state-district races that were themselves close and won by the majority party. For example, we determine that within our sample all state Houses with slim-majorities of one seat resulted from elections with at least one close state-district race won by the majority party. In other words, there is no evidence to suggest that one-seat slim-majorities could have been predetermined or manipulated.
The closest paper to ours is Feigenbaum et al. (Reference Feigenbaum, Fouirnaies and Hall2017). They propose a continuity-based multidimensional RDD using district-level voting in order to address the lack of continuity in the number of seats in the US state Houses (they do not apply their method to the Senates). Their method requires the choice of a metric to build a one-dimensional running variable from a series of state-district race results. With infinite metrics to validate, their method would—at the limit—be a matching estimator and no longer a design-based approach (Feigenbaum et al., Reference Feigenbaum, Fouirnaies and Hall2017, p. 18). Here, instead, we employ the number of seats directly as the running variable. Thus, the method presented here—if valid—is more straightforward to apply. Moreover, our approach does not require the availability of lower-level electoral data. If lower-level data are available or partially available, as in our case, the test proposed here can be implemented.
Another stance would be to define an aggregate measure of the results in district-level races as an alternative running variable that is measured with error. In this alternative setting, an election could yield a slim-majority without this running variable being close to the cut-off or vice versa.Footnote 3 Our approach is different. We assume that the running variable is the number (or percentage) of seats and that it is measured without error. We identify our effect by comparing slim-majorities within a precise window. Any observation that does not fall within the determined widow is excluded from the analysis, whether or not it could have been classified as “close to the cut-off” with an alternative running variable based on the aggregation of district-level races.
As an illustration, we use the method proposed here to estimate the effects of partisan control of the state Houses on electoral outcomes. Formally, our RDD parameter of interest is E[Y i(1)|X i = 1] − E[Y i(0)|X i = −1], where Y i(1) is the outcome in t+1 given that party i won a slim-majority in t of one seat, X i = 1; and Y i(0) is the outcome in t+1 given that party i won a slim-minority in t of one seat, X i = −1. This parameter is identifiable under the assumptions described above (see also Cattaneo et al., Reference Cattaneo, Idrobo and Titiunik2019). In practice, we determine in the following whether we can compare slim-majorities and minorities of only one seat or whether we must increase the interval to more seats.
Our estimates suggest there is no causal effect of winning a slim-majority in the state House on:
(i) winning another majority in the subsequent election, (ii) influencing the election for
Governor, or (iii) influencing the majority in the state Senate.Footnote 4
1. Validity of RDD with slim-majorities
The available data are comprised of that from the 48 continental American states from 1960 to 2006 and is the same as Besley and Case (Reference Besley and Case2003). We have updated their sample from 1960 to 1998 with data from 1999 to 2006.Footnote 5 To keep the sample comparable, we focus the analysis on the states with the most common set of institutions: a partisan two-chamber legislature and a governor with line-item veto power. The working sample has 41 states from 1960 to 2006. Each observation is a state-election-year.
We also have data on state electoral returns at the district-level spanning 1967–2003 (Carsey et al., Reference Carsey, Berry, Niemi, Powell and Snyder2008). There are an approximate 18 percent missing values for the variable that we are interested in: the margin of victory, defined as the difference between the percentage of votes that the winner received and the percentage of votes that the second-place candidate received in each state district.
For clarity, we reserve the word “election” for the state-wide poll that determines how many seats each party will hold in the state legislature. We reserve the word “race” for a state-district-specific poll that determines the partisan identity of each state-district's representative.
1.1. Slim-majorities and close state-district races
Each US state is divided into state-districts. During an election, each state-district chooses a representative to the state legislature by a first-past-the-post system.Footnote 6 If each of these state-district races was the result of large and predictable winning margins, then there would be no uncertainty regarding which party would win the majority of seats in each election. A slim-majority of one seat would be as predictable and predetermined as a landslide majority of 20 seats. Therefore, the first test we propose in order to establish that slim-majorities can be interpreted as local randomized experiments is to establish that enough state-district seats were themselves the result of close races that were won by the same party that won the majority of seats.
In Table 1, we present the data in the form of different windows around the cut-off of our running variable, Democratic majority. The variable takes a value of 1 if the Democrats hold a majority of one seat, 2 if there is a two-seat majority, − 1 if the Republicans hold a majority of one seat, − 2 if the Republican majority is of two seats, and so on. The cut-off is 0. Democratic majority also takes the value 0 if there is a tie (some legislatures have an even number of seats). In row 1, column 2, we can see that there are ten elections for state Houses that have ties (i.e., both Republicans and Democrats hold the same number of seats) or slim-majorities of one seat. In row 1, column 3, we can see that all (i.e., 100 percent) of these 10 elections had at least one state-district race won by the majority party with a winning margin of less than 5 percent of the district vote, our chosen threshold for a close election.Footnote 7 This suggests that all majorities of one seat were neither predetermined nor predictable. Had that one seat gone to the other party, the partisan identity of the majority in the House would have changed.
Table 1. Percentage of slim-majorities that could not be predicted
Note: The first column indicates the size of a window around the cut-off Democratic majority = 0. The second column reports the number of observations in each widow for which we have state-district electoral data. The third column reports the percentage of state-district races in each window that were close wins for the majority party. The data on election results by state district has been provided by Carsey et al. (Reference Carsey, Berry, Niemi, Powell and Snyder2008).
As we descend through Table 1, we increase the size of the window around the cut-off. Row 2 shows that out of 20 observations (i.e., state-years) with slim-majorities of two seats or less, 19 (i.e., 95 percent) had at least two state-district races won by the majority party that were close (the remaining observation only had one district race won by the majority party that was close). Ill-health, scandal, or the weather, for example, could also change the result of a particular seat in unpredictable ways and still deliver a winning margin above 5 percent of the vote-share. For that reason, we see the rate of 95 percent in row 2, column 3, as indicating that the − 2, 2 window should not be rejected as a potential window within which we can interpret the results as a local randomized experiment. As the window size increases to the − 5, 5 window, the percentage of close state-district races won by the majority party approaches 80 percent.
We now pursue the same exercise, but instead look at the percentage of seats as the running variable (rows 6–10 in Table 1). There are 18 slim-majorities of 1 percent of seats or less; 83 percent of them have at least 1 percent of close state-district races that were won by the majority party. As mentioned in the previous paragraph, the result in individual seats can be unpredictable and still deliver winning margins above 5 percent. For this reason, we see a rate above 80 percent as an indication that these slim-majorities were neither predetermined nor predictable. As the window size increases in rows 7–10, the percentage of slim-majorities that had enough state-district close races won by the majority party decreases to 62 percent for the -2,2 window and to approximately 50 percent for larger windows. Overall this indicates that windows other than the -1,1 interval are not valid as a local randomized experiment.
From rows 11 onwards in Table 1, we follow the same procedure as before for the state Senates. In all five windows in which the number of seats is the running variable, approximately half of slim-majorities in the state Senates are the result of elections in which none of the state-district races won by the majority party were close. This suggests that even the smallest window of − 1, 1 seats cannot be interpreted as a local randomized experiment. Utilizing the percentage of seats, both the -1, 1 and the -2, 2 intervals have more than 80 percent of slim-majorities with enough close state-district races that were won by the majority party. Therefore, we do not reject that these two windows could be interpreted as local randomized experiments.
1.2. Determining the windows for local randomization
In this section, we follow the practical steps suggested by Cattaneo et al. (Reference Cattaneo, Frandsen and Titiunik2015) to establish whether local randomization is plausible in small windows around the cut-off and determine the size of such a window. The procedure involves a simple difference-in-means test for the predetermined covariates comparing their values on each side of the cut-off. This test is carried out for each candidate window. If the p-value regarding the null that a covariate has the same value for both sides of the cut-off is below 0.15 (as suggested by Cattaneo et al., Reference Cattaneo, Frandsen and Titiunik2015), then that window is rejected and we attempt the procedure with a smaller window. A window is selected if one cannot reject the null for any of the predetermined covariates using a threshold p-value of 0.15. As suggested by Cattaneo et al. (Reference Cattaneo, Frandsen and Titiunik2015), we only consider windows with at least ten observations on each side of the cut-off.
The implementation of this method for the US legislatures can be seen in Table 2. In column 1, we list different windows around the cut-off, Democratic Majority = 0.Footnote 8 In column 2, we list the number of observations (i.e., of state-years) on either side of the cut-off. In column 3, we list the lowest p-value from the balance test for a series of predetermined covariates. In column 4, we list the covariate with the lowest p-value.Footnote 9
Table 2. Minimum p-value of predetermined covariates for different intervals
Note: The first column indicates the size of the window around the cut-off Democratic majority = 0. The second column reports the number of observations on each side of the cut-off within a widow. The third column reports the lowest p-value of a balance test for a series of covariates using large-sample inference. Column four reports the covariate with the lowest p-value.
The electoral covariates that we test for balance are the following: Dem Majority Senate, an indicator variable that takes a value of 1 if the Democrats gain control of the state Senate in the current election; Dem Majority Senate in t−1; Dem Majority House, an indicator variable that takes a value of 1 if the Democrats gain control of the state House at the current election; Dem Majority House in t−1; Dem Governor, an indicator variable that takes a value of 1 if the governor is a Democrat during the current election or wins the current election; Dem Governor in t−1, i.e., the election result two or four years previous depending on the electoral cycle. The economic covariates that we test for balance are: State real income per capita in 1982 US$; unemployment rate; local property taxes (%GDP); and state tax level (%GDP). We also treat the year of the election as a covariate.
The − 2, 2 window does lend itself to be interpreted as a local randomized experiment (Table 2, row 1). There are more than ten observations on each side of the cut-off and the predetermined covariate with the highest p-value is Dem Majority Senate with a p-value of 0.27, above the 0.15 rejection threshold. Note that the number of observations is similar on either side of the cut-off, also a condition for the validity of RDD. Finally, from Table 1, we know that more than 90 percent of the observations in this interval are slim-majorities with at least two close state-district races won by the majority. Windows − 3, 3 and − 4, 4 seem valid as local randomized experiments (Table 2, rows 2–4) as all covariates are balanced. The − 5, 5 window, however, is rejected as a local randomized experiment. The reason we do not have a row entry for the -1,1 window is because the number of observations on either side of the cut-off, 4 and 7, is too low for inference.
The same exercise is repeated for the percentage of seats in the House as the running variables (rows 5–9 in Table 2). Only the -1, 1 window cannot be rejected as a local randomized experiment.
Slim-majorities in the state Senate cannot be interpreted as local randomized experiments. This can be seen in the remaining rows of Table 2. For every window, there is at least one covariate for which the balance test is rejected with a p-value below 0.15.Footnote 10
Continuity-based RDD
The number of seats is a discrete running variable with mass points at the integers as can be seen in Figure 1(a) and (c). If the running variable is defined as the percentage of seats, however, it is not obvious a priori that such a variable would have mass points.
Figure 1. Number of observations at each value of Democratic majority. (a) Senate—number of seats. (b) Senate—percentage of seats. (c) House—number of seats. (d) House—percentage of seats. Note: Democratic majority is the running variable presented on the horizontal axis. Positive values imply a Democratic majority and negative values a Republican majority. In each figure, the caption indicates an alternative definition of the running variable: Senate, House, number of seats, and percentage of seats. The number of observations for each value of the running variable is shown on the horizontal axis.
In Figure 1(b) we plot the number of observations at each value of the variable Democratic majority in the Senate measured as the percentage of seats. There is a clear mass point in the proximity of − 1 percent (five observations) and of 1 percent (four observations); an empty interval in the proximity of 0 and a mass point at 0 (16 observations). Hence, the assumption of continuity of the running variable does not hold and the “continuity-based” RDD setup should not be implemented for the state Senates.
In Figure 1(d), it is less clear whether we should reject the assumption of continuity for the variable, Democratic Majority, measured as the percentage of seats in the House—as there are fewer mass points. In particular, owning to some of the Houses having high numbers of representatives, the variable reaches the value − 0.2 percent from the left and 0.2 percent from the right.
Suppose we were to assume continuity for the running variables in the Senate and House, number of seats, and percentages. Usual practice would dictate we decide on the validity of the regression discontinuity design by analyzing Table 3. In this table, we test the balance of covariates via a local linear regression with a triangular kernel, the optimal bandwidth suggested in Imbens and Kalyanaraman (Reference Imbens and Kalyanaraman2012), and p-values as suggested in Calonico et al. (Reference Calonico, Cattaneo and Titiunik2014). The lack of any statistically significant jump at the cut-off would suggest we maintain the assumption of continuity of the potential outcome functions at the cut-off. These results, in addition to the lack of a jump in the density as measured by the McCrary (Reference McCrary2008) test in the last two rows, would suggest we maintain the assumption of continuity of all variants of the running variable.
Table 3. Balance test for covariates and Democratic majority—CCT robust
Note: The first column reports the predetermined covariates. Columns 2–5 present the p-value for the balance test for different running variables. Local linear estimates and bandwidth tests are implemented as suggested by Imbens and Kalyanaraman (Reference Imbens and Kalyanaraman2012) and p-values as suggested in Calonico et al. (Reference Calonico, Cattaneo and Titiunik2014). Density test as suggested in McCrary (Reference McCrary2008).
We would then go on naively to implement the canonical “continuity-based” RDD for running variables that we have shown to be discrete and with clear mass points. Therefore, any result obtained would be hard to interpret or wrong. The potential exception is the results for the Democratic Majority in the House measured as a percentage of seats - as it is not clear from Figure 1(d) that we should reject the assumption of continuity regarding this running variable.
Electoral outcomes in the state Houses under local randomization
We present results for the smallest window deemed valid as a local randomized experiment for both the number of seats and percentage of seats, respectively − 2, 2 and -1, 1. Sample sizes are low, but we show in the online Appendix that the results are robust to larger windows with larger sample sizes. We also estimate the canonical continuity-based RDD with the running variable Democratic Majority in the House measured as the percentage of seats.
First, we check whether winning a slim-majority today increases the chance that the party wins a majority in the next election (Table 4, rows 1, 2, and 3). The outcome variable Dem House in t+1 takes a value of 1 if Democrats gain control of the state House in the next election. None of the estimated jumps is statistically different from zero. The point estimates are themselves near zero. This result is in line with what Feigenbaum et al. (Reference Feigenbaum, Fouirnaies and Hall2017) have estimated using a multidimensional RDD.
Table 4. Randomization-based estimation of slim-majority effects in state Houses
Note: Local linear estimates and bandwidth tests are implemented as suggested by Imbens and Kalyanaraman (Reference Imbens and Kalyanaraman2012) and p-values as suggested in Calonico et al. (Reference Calonico, Cattaneo and Titiunik2014). Local randomization windows chosen according to Cattaneo et al. (Reference Cattaneo, Frandsen and Titiunik2015) and p-values calculated using large-sample inference.
We depart from Feigenbaum et al. (Reference Feigenbaum, Fouirnaies and Hall2017) as the null effect we estimate is persistent over time (see the online Appendix). One potential reason for our results differing may be that their method makes use of more information, i.e., the race result of each individual district. Another potential reason may be that their downstream result is dependent on the particular metric chosen to construct their aggregating running variable.
Another individual electoral outcome that has been studied through RDD and close elections is coat-tail effects. Broockman (Reference Broockman2009) finds that congressional incumbency has no impact on presidential electoral results. The parallel slim-majority effect would be for a slim democratic majority in the House to affect the odds of the Democrats becoming the majority in the Senate or winning the governorship in the following election. We also find a null effect in both cases (Table 4, rows 4–9).
Further, in the online Appendix we explore the effect of partisan control on the tax level and unemployment and find null effects as well.
3. Concluding remarks
Our results indicating that slim-majorities in the state Senates are not valid for RDD may have been expected. The average Senate has 39 seats and one-third of the seats are contested in each election. The probability of at least one of them being a close race is small compared to the average House that has 100 seats contested in each election. We contribute by establishing that indeed there are not enough close races in Senate elections to warrant an RDD. Even if we were to ignore the lack of close races (they could have been unpredictable even if not close), we show that slim-majorities in the state Senates do not satisfy the conditions to be interpreted as local-randomized experiments. These results are important because under the erroneous assumption that the running variable is continuous, the usual balance tests would fail to reject the validity of Senate slim-majorities as RDDs.
Slim-majorities in the state Houses, on the other hand, seem to be valid under the local randomization assumption. If the running variable is the number of seats, the clear presence of mass points implies that we should not use RDD methods that are based on the continuity of the running variable. However, if the running variable is the percentage of seats, it is not clear that we should reject the assumption of a continuous running variable. In practice, we show that results using the “continuity-based” canonical RD setup with percentage of seats as the running variable yields similar results to the local-randomization methods. Therefore, following Cattaneo et al. (Reference Cattaneo, Titiunik and Vazquez-Bare2017) we suggest that results under both assumptions be shown in order to check for robustness. Further, if available or partially available, information on the underlying vote share should be used as a falsification analysis in the form of the test depicted in Table 1. The application of these methods contributes to the literature by allowing a series of questions on slim-majority effects to be addressed with an easily-implementable design-based method.
A remaining concern is that such a design may be unable to capture partisan effects even if there was one. This could be based on power concerns, for example. In the online Appendix, we show that the RDD method described in this paper does capture a statistically significant effect: a unified government—defined as the state House and Governor belonging to the same party—has a higher tax level than a divided government. This result is a re-estimation of the effect described in De Magalhães and Ferrero (Reference De Magalhães and Ferrero2015) using the methods discussed herein.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/10.1017/psrm.2020.5
