2. Research design and methods

To examine how satisfied citizens are with different election results, our module was implemented at the very beginning of the 2019 Canadian federal election campaign as part of the Canadian Election Study, an internet survey that is nationally representative according to province, gender, age, and education with weights. The sample size for our module is 2900 respondents.

Canada is an interesting case for the study of citizens' views about different electoral outcomes. First-past-the-post (FPTP) prevails in the country for all elections, national, provincial and local. There have been, however, a few experiments with some forms of PR at the provincial level (Massicotte, Reference Massicotte and Blais2008). More importantly, electoral reform is on the political agenda. No less than five referenda on some form of PR have been held in three provinces (British Columbia, Ontario and Prince Edward Island) in the last 15 years, and in 2019 the Québec provincial government presented a proposal for a mixed system. At the federal level, two parties (the New Democratic Party and the Green Party) openly advocate for PR. The fact that a majority of citizens rejected the PR proposal in four of the five referendaFootnote ¹ suggests that many Canadians are attached to FPTP but the fact that electoral reform proposals keep emerging also indicates that the status quo is questioned by many.

In our survey experiment respondents were randomly exposed to manipulated election outcomes. There were two treatments: the first was the electoral performance of the preferred party and the second the proportionality of the seat/vote relationship. To identify the preferred party, respondents were asked to rate the parties on a 0 to 10 scale. We focus on the four main parties that had candidates in all the constituencies in Canada: Liberals, Conservatives, New Democratic Party (NDP) and the Greens. The most preferred party is the party receiving the highest score among these four (ties were randomly untied).

For the treatment on the electoral performance of the preferred party (see Appendix A for a detailed exposition of the design), we had three conditions: plus/reference/minus. The reference condition is one in which the preferred party's vote share corresponds to the vote share in the polls at the time of the survey (according to the CBC poll tracker). At that time, the Liberals and the Conservatives were tied at 34 percent, the NDP had 13 percent and the Green party 11 percent. The “plus” outcome is the one in which the preferred party gets 6 percentage points more votes than in the reference condition and the “minus” outcome corresponds to the party obtaining six fewer percentage points.Footnote ² The plus (minus) outcomes corresponded to gains (losses) that could take place during the campaign and that were both realisticFootnote ³ and substantial enough to affect citizens' evaluations.

The second treatment is the degree of disproportionality of the seat/vote relationship. There are two conditions: proportional and disproportional. For a disproportional outcome, we take the situation that presently exists in Canada with FPTP. The CBC poll tracker that we use for the “reference” condition indicated the likely distribution of the seats in Parliament given the distribution of the votes. The Liberals were predicted to win 49 percent of the seats, the Conservatives 42 percent, the NDP 4 percent and the Greens 1 percent. We use these numbers. Unsurprisingly, the two larger parties, the Liberals and the Conservatives, win more seats than their vote share while the opposite holds for the NDP and the Greens. The Liberals do better than the Conservatives with the same vote share, mostly because the latter waste many votes in the province of Alberta, where they have huge leads in many constituencies. In the plus (minus) condition, we assume that the party's 6 percentage point vote share gain (loss) leads to a 9 percentage point seat share gain (loss). We respect the logic of FPTP under which the gain (loss) in seat share is greater than the gain (loss) in vote share.Footnote ⁴

For a proportional outcome, we assume a “typical” system under which for all the parties except the very small seat shares correspond to vote shares. In the case at hand, the small parties are in the “other” category, with a combined total of 4 percent of the votes. In all the scenarios, these small parties are assumed to win no seat. This produces a 4 percentage point bonus seat, which we allocate to the two larger parties, which are both slightly overrepresented, by 2 percentage points, in seat share.

In short, we have four groups namely Liberal, Conservative, NDP and Green supporters, defined on the basis of their preferred party. For each of these groups, there are two treatments and six scenarios: disproportional with more votes (plus), disproportional with “reference” votes, disproportional with fewer votes (minus), proportional with more votes (plus), proportional with “reference” votes and proportional with fewer votes (minus).

Each respondent was invited to evaluate four of these six scenarios (Appendix A displays an example for a Conservative party supporter). The scenarios and their order were randomly selected. Respondents were asked to indicate on a 0 to 10 scale how satisfied they would be with such electoral outcomes. This is the dependent variable.

3. Findings

Table 1 shows summary results for the scenarios corresponding to the two treatments, combining all party supporters (Appendix B presents the mean satisfaction score for each of the 24 scenarios, six for each of the four supporter groups). We see, without much surprise, that party supporters are most satisfied when their party has more votes and least satisfied when it has fewer votes. Mean satisfaction is 4.3 in the reference scenario; it increases to 5.8 when the preferred party has more votes and decreases to 3.4 when it has fewer votes. We can also see that, all in all, voters slightly prefer proportional outcomes; mean satisfaction is 4.3 when the outcome is disproportional, and it increases to 4.7 when it is proportional, a relative increase of 5 percent.

Table 1. Mean satisfaction by scenario (all supporter groups combined)

All differences are statistically significant at p < 0.05.

Table 2 presents a more systematic analysis of the results. The dependent variable is the satisfaction score. The three main independent variables are the electoral performance of the preferred party, that is the percentage of the vote obtained by the party in that scenario,Footnote ⁵ proportionality (1 for proportional outcomes), and dummies for each party (the Liberals are the reference category). The ordinary least squares (OLS) estimations allow us to ascertain the independent impact of each variable, everything else being equal.

Table 2. Impact of the scenarios: OLS estimations

Note: Clustered standard errors by respondents in parentheses. *p < 0.05, **p < 0.01, ***p < 0.001.

Column 1 shows that people are more satisfied with an outcome in which their party has more votes. Recall that our treatments correspond to increases (decreases) of 6 percentage points in vote share. These increases (decreases) produced increases (decreases) of 1.2 in satisfaction on the 0 to 10 scale (6 × 0.2). We can also see that satisfaction is slightly higher, overall, with a proportional result. Finally, column 1 indicates that, everything else being equal, NDP and Green supporters express greater satisfaction compared to the Liberals. This is only because we control for vote shares. Without that control, satisfaction is lower among NDP and Green supporters (see Appendix B). What this tells us is that NDP and Green supporters would be happier than Liberal supporters if their party had as many votes, which makes sense since this would entail many more votes than expected at the beginning of the campaign. Conservative supporters, for their part, are less satisfied. This reflects the fact that at the time of the survey the Conservatives were tied with the Liberals in vote intentions but lagging behind in projected seats.Footnote ⁶

These results suggest that what matters is not only how many votes one's party obtains and the proportionality of the outcome but also whether the party is the plurality winner, that is, it has more seats than all the other parties. There are five scenarios in which one's party wins a plurality of the seats (and plausibly forms the government, as is the tradition in Canada): scenarios 1 and 4 for the Conservatives and 7, 8, and 10 for the Liberals; the mean rating of each of these five scenarios is above 6. The mean rating of each of the 19 other scenarios is below 5 (see Appendix B). One is unlikely to be happy if and when one's preferred party is not in government.

In column 2, we test whether what matters is having more seats than the other parties by adding a plurality seat dummy variable which equals 1 when the preferred party has a plurality of the seats. That variable is strongly significant. Satisfaction is 2.6 points higher when the party is a plurality winner. The introduction of that variable weakens the other coefficients (except for the proportional dummy, which becomes slightly higher), which indicates that votes still matter though they matter much less than having a plurality of the seats.

Column 3 looks at the interaction between proportionality and votes. We could expect party supporters to be more favorable to proportionality when their party has fewer votes since they lose less under a proportional outcome. The interaction term has the correct (negative) sign but is not significant. Respondents do not rate proportional outcomes more negatively when their party has more votes (and would do better under a disproportional scenario).

Column 4 includes interaction terms between proportionality and the parties. The results show that proportional outcomes are preferred by all party supporters. The findings concerning the NDP and the Greens were expected. Both parties are disadvantaged by FPTP and both parties advocate a more proportional electoral system.

The positive coefficient associated with the main proportionality term in column 4 suggests that Liberal supporters (the reference category) prefer proportional outcomes. This result may be misleading. Everything else being equal, Liberal supporters give higher ratings to proportional outcomes, but the bottom line is that it is under a disproportional outcome that they are more likely to win a plurality of the seats, which they like a lot. All in all, Liberal supporters slightly prefer disproportional outcomes (mean rating of 5.58) over proportional ones (mean rating of 5.07). Note that this is only a slight preference (a relative difference of 10 percent), which suggests that they are also sensitive to fairness considerations.

It is with respect to the Conservative party that the results are the most intriguing. Because it is a big party it usually benefits from FPTP. At the time of the survey, the CBC poll tracker was giving them 34 percent of the vote and 42 percent of the seats. But in the context of the 2019 general election, the party was disadvantaged relative to the Liberals, their main rival. With the same vote share, the Liberals were getting more seats, and the Liberals managed at the end to win more seats (and form the government) with fewer votes than the Conservatives. With a proportional system the Conservative party was getting fewer seats but what matters the most is that they were getting as many seats as the Liberals (with the same vote share). These results suggest that Conservative supporters prefer outcomes that do not confer an unfair advantage to their main rival.

We may wonder whether people's reactions depend on how much they like or dislike the various parties. To investigate this possibility, we distinguish voters with strong and weak preferences, the former being defined as those whose top-rated party gets at least 2 points more than the second most-liked party on the 0 to 10 scale. We performed the same analyses reported in Table 2 separately for the two groups (strong and weak preferences). The results are presented in Appendix D. We see that the patterns are the same in both groups, that is Votes, Proportional and Plurality seats have positive and significant coefficients but that, unsurprisingly, strong supporters react more strongly to the electoral performance of their party and attach less weight to proportionality.

4. Summary and discussion

The question that we set up to address is whether views about PR are shaped by partisan preferences. Our findings suggest that although partisan considerations matter, perceptions of fairness matter as well. Yet, people pay attention, first and foremost, to which party will form the government. To address our research question we proposed an innovative research design. We used an experimental design because of its great advantage in terms of causal inference, and thus to elicit citizens' reactions to potential electoral outcomes, some proportional and some disproportional, some where their preferred party performs well and some where it performs poorly. We were keen to enhance external validity by conducting our experiment during a real election campaign, using realistic vote and seat results, and real parties. Furthermore, we included as one of the scenarios the distribution of votes and seats that the CBC poll tracker suggested would occur if the election were to take place at the time of the survey.

We believe this research design is a fruitful one. The seat–vote relationship that the participants were shown corresponded to the reality of FPTP is applied to the Canadian context. The proportional outcomes that we created are what is usually found in proportional systems: a small overrepresentation of the big parties and the underrepresentation of very small ones. As for scenarios corresponding to plus or minus outcomes for each of the parties, we wanted to make sure that our treatments were strong enough to elicit voters' reactions while also being realistic.

The information that was presented concerned only the vote and the seat distribution. We did not mention which party was going to form the government nor did we mention the possibility of a coalition government, an option that was sometime raised during the campaign although it has never occurred at the federal level in Canada. We considered including the government formation dimension, but we were concerned that this would be too much information to digest. We ran a pilot study in the summer of 2019 and the feedback we received suggested that it would indeed be too much information. Our findings further show that the respondents did indeed pay attention to which party won a plurality of the seats (and would probably form the government). Citizens spontaneously think about the composition of government when they are informed about the outcome of an election, and this affects their reactions even when that aspect is omitted from the experimental set up.

Our study was intended to examine how citizens in a disproportional system react to more proportional outcomes. There is little doubt that voters' evaluations of the various outcomes depend in part on their own prior experience with the electoral system. But, we do not see this necessarily as a shortcoming of our experimental design. If context matters (and we have every reason to believe it does) then voters' reactions to the treatments should vary across countries.

We should, however, acknowledge one limitation of our approach. We made sure that the scenarios that the respondents were asked to evaluate were realistic in the Canadian context. Whether these scenarios would appear realistic in other contexts is an open question, which raises difficult questions about how we could compare findings from different countries. There is also the possibility that the effects that we have uncovered in this particular instance would be greater in a more polarized party system.

What is reassuring is that our design has produced findings that make sense. Our findings are consistent with those by Plescia et al. (Reference Plescia, Blais and Högström2020): there is support for PR even in countries with majoritarian electoral systems such as Britain and Canada. Because our study deals with real parties during a real election, we can add an important caveat, that is, support for PR is conditional on partisan considerations. Finally, we find that citizens' evaluations focus on who forms the government, which is consistent with research that shows that perceptions of winning (or losing) an election is based first and foremost on whether one's party is in government or opposition (Stiers et al., Reference Stiers, Daoust and Blais2018).