Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-02-06T03:08:13.211Z Has data issue: false hasContentIssue false

The ABCs of Electoral Reform: The Impact of Reading Levels on Knowledge, Interest and Opinion

Published online by Cambridge University Press:  17 August 2020

Holly Ann Garnett*
Affiliation:
Royal Military College of Canada, P.O. Box 17000, Station Forces Kingston, ONK7K 7B4
André Blais
Affiliation:
Université de Montréal, Pavillon Lionel-Groulx local C4040, 3150, rue Jean-Brillant, Montréal, QCH3T 1N8
*
*Corresponding author. Email: holly.garnett@queensu.ca
Rights & Permissions [Opens in a new window]

Abstract

How should citizens be educated about complicated political issues like electoral reform? Are there basic principles that should be followed? This article tests one potential principle for government bodies, the media and educators to follow when conducting information campaigns: namely, lowering the reading level of information. Educators have long argued that texts can be confusing when written at a literacy level higher than the reader is able to digest. This article tests the impact of reading level on knowledge, interest and opinion on an electoral reform proposal. It employs an experimental design, conducted in person in fall 2018 with college students in Ontario, Canada. The experiment asked the students to read a text on a single transferable vote (STV) electoral system at one of three reading levels (or a control text) and then answer a series of questions gauging their knowledge, interest and opinion on the electoral reform proposal. The results provide an assessment of the impact of different levels of information on these factors and suggest concrete recommendations for election management bodies (EMBs) and other actors seeking to educate the public on complex political issues.

Résumé

Résumé

Comment sensibiliser les citoyens à des questions politiques complexes comme la réforme électoraleY a-t-il des principes de base à suivreCet article teste un principe potentiel que les organismes gouvernementaux, les médias et les pédagogues devraient suivre lorsqu'ils mènent des campagnes d'information : à savoir, abaisser le niveau de lecture des informations. Les pédagogues soutiennent depuis longtemps que les textes peuvent être déroutants lorsqu'ils sont écrits à un niveau d'alphabétisation supérieur à celui que le lecteur est capable d'assimiler. Cet article teste l'impact du niveau de lecture sur les connaissances, l'intérêt et l'opinion autour d'une proposition de réforme électorale. Il utilise un modèle expérimental, mené en personne à l'automne 2018 auprès d'étudiants de l'Ontario, au Canada. L'expérience consistait à demander aux étudiants de lire un texte sur un système électoral à vote unique transférable à l'un des trois niveaux de lecture (ou un texte de contrôle) et de répondre ensuite à une série de questions évaluant leurs connaissances, leur intérêt et leur opinion sur la proposition de réforme électorale. Les résultats fournissent une évaluation de l'impact des différents niveaux d'information sur ces facteurs et suggèrent des recommandations concrètes pour les organes de gestion des élections et les autres acteurs qui cherchent à éduquer le public sur des questions politiques complexes.

Type
Research Article/Étude originale
Copyright
Copyright © Canadian Political Science Association (l'Association canadienne de science politique) and/et la Société québécoise de science politique 2020

Introduction

The issue of electoral reform in Canada has never quite gone away despite frequent failed attempts. Referendums (of various kinds, with a variety of questions) have been held in Prince Edward Island (2005, 2016 and 2019), Ontario (2007) and British Columbia (2005, 2009 and 2018)—not to mention the failed promise for electoral reform following the 2015 federal election, which the Liberal party had pronounced would be the last election conducted with the first-past-the-post (FPTP) electoral system. Similar referendums on the issue have likewise taken place in the United Kingdom and New Zealand (Katz, Reference Katz and Milner2004; LeDuc, Reference LeDuc2011; Nagel, Reference Nagel1994; Vowles, Reference Vowles2013).

After some referendums on electoral reform, commentators and academics have blamed the failure of these referendums, at least in part, on poor education campaigns (Cutler and Fournier, Reference Cutler and Fournier2007; Stephenson and Tanguay, Reference Stephenson and Tanguay2009).Footnote 1 Uninformed citizens, it is argued, cannot make competent decisions on the complicated issue of electoral reform (Cutler and Fournier, Reference Cutler and Fournier2007; Stephenson and Tanguay, Reference Stephenson and Tanguay2009). In some cases, it is the election management body (EMB) that is tasked with the massive project of educating the public on possible new electoral systems. But in the Ontario case, for example, it was criticized for failing to adequately ensure that the public was informed that there would be a referendum or about the possible options on the ballot (Garnett, Reference Garnett2014). These critiques raise the question: How should citizens be educated about complicated political issues like electoral reform?

This article examines one principle to follow when conducting information campaigns: namely, the readability of political information. Educators have long argued that texts written at a reading level outside the reader's abilities can be confusing. However, this principle is rarely applied to political issues because there is a concern that the information may lose its nuance and accuracy if written for audiences with lower literacy levels. Political messaging (even the non-partisan kind) is often so carefully crafted that reducing it to clear and simple language is extremely challenging—and sometimes it is a politically unpalatable option.

This article examines the impact of a text's readability on knowledge, interest and opinions on one electoral reform proposal: single transferable vote (STV). It employs an experimental design, conducted with college students in Ontario, Canada. The researchers asked students to read a text on an STV electoral system at one of three different reading levels or a control text on an unrelated topic. We then surveyed the students, asking a series of questions about their background and the electoral reform proposal. The results allow us to analyze the impact of different reading levels of informational texts on four outcome variables: general information, knowledge, interest, and opinion on electoral reform.

This research has implications not only for the study of referendum education but also for our understanding of how literacy and clear design affect citizens’ ability to make informed choices in a democracy. The results will be of interest to policy makers and educators as they seek to craft political information that is more easily digestible for the public. It also speaks to our scholarly understanding of the merits and limits of public deliberation and the responsibilities of those creating education campaigns when the public's opinions are sought on major policy changes.

Knowledge, Information and Referendum Campaigns

We know that the information voters have on political issues plays an important role in their decision to vote and subsequent vote choice, in both referendums and general elections. In addition to the well-documented (though often debated) relationship between education and voter turnout (Burden, Reference Burden2009; Smets and van Ham, Reference Smets and van Ham2013), some research has delved into how information affects not just the choice of whether or not to vote (Lassen, Reference Lassen2005; Matsusaka, Reference Matsusaka1995) but also the choice of who (or what) to vote for (Bartels, Reference Bartels1996; Roy, Reference Roy2009). In Canada, research has found that levels of political information affect voting decisions (Roy, Reference Roy2009) and that levels of information may even have disparate effects, depending on social group (Bittner, Reference Bittner2007). In sum, information matters in both turnout and the choices the voter makes at the polls.

However, “perfect information” is rarely the case in any type of election, with many voters lacking information about a variety of aspects of politics and the voting process. Acquiring this information, however, costs time and energy, which voters are unable or unwilling to devote to politics amid the other concerns of daily life (Downs, Reference Downs1957; Verba et al., Reference Verba, Schlozman and Brady1995). Scholars also note differences in civic skills, including education and language skills needed to digest information or understand complex concepts (Verba et al., Reference Verba, Schlozman and Brady1995).

To counteract this lack of information, or an inability or unwillingness to acquire it, voters may use information shortcuts, or cues, to limit the information costs of political decision making (Lupia and McCubbins, Reference Lupia and McCubbins1998; Popkin, Reference Popkin1991). Cues might include partisanship, ideology or candidate characteristics (Campbell et al., Reference Campbell, Converse, Miller and Stokes1960; Popkin, Reference Popkin1991; Lau and Redlawsk, Reference Lau and Redlawsk2006). But referendums can present a unique set of challenges to voters seeking to take shortcuts in decision making (Hobolt, Reference Hobolt2007). While some referendums take on a partisan nature, with clear opinion leaders fighting on different sides of the debate or with traditional partisan cues playing a role (Jenssen and Listhaug, Reference Jenssen, Listhaug, Mendelsohn and Parkin2001), other referendum questions are more technical in nature and lack candidates to evaluate or partisan cues to follow (McAllister, Reference McAllister2001). For example, in some Canadian provincial referendums on electoral reform, parties remained silent on the question, deciding to allow the voters to decide free from partisan debate (Garnett, Reference Garnett2014; Stephenson and Tanguay, Reference Stephenson and Tanguay2009).

Furthermore, particularly with complex technical issues such as electoral reform, there is little likelihood that the voter is able to relate the referendum proposal to their daily life (LeDuc, Reference LeDuc2011; Vowles, Reference Vowles2013); nor does the referendum necessarily appeal to emotional debates or issues, as was the case, for example, in the Brexit referendum (Clarke et al., Reference Clarke, Goodwin and Whiteley2017). In these cases, then, information provided by electoral authorities or the media is particularly important, since it is sometimes the only window a voter has into the proposal to make a reasoned decision about the referendum.

Empirical research has also demonstrated that information or knowledge about referendum proposals can have a significant effect on the results of the referendum. For example, Elkink and Sinnott (Reference Elkink and Sinnott2015) find that higher knowledge about the Lisbon Treaty predicts voters’ likelihood to vote yes to the Irish referendum on the question.Footnote 2 Christin et al. (Reference Christin, Hug and Sciarini2002) likewise demonstrate a status quo bias in referendum campaigns, suggesting that voters are risk averse when they have low information.

This phenomenon of higher-information voters tending to be more likely to support new proposals extends to referendums on electoral reform. Considering the 2011 referendum on electoral reform in the United Kingdom, Vowles (Reference Vowles2013) finds that when survey respondents were provided with an explanation of the AV system, they became more likely to support the proposal. This echoes previous findings on electoral reform from British Columbia's 2005 and 2009 referendums and Ontario's 2007 referendum (Cutler and Fournier, Reference Cutler and Fournier2007; LeDuc, Reference LeDuc2011; Stephenson and Tanguay, Reference Stephenson and Tanguay2009).

If information campaigns are so important, how have major actors sought to educate voters during referendums when information is scarce and traditional cues hard to find? Depending on the issue at hand, a variety of actors and strategies may be employed, ranging from public debates by passionate campaigners for and against the proposed reform to neutral information campaigns conducted by government bodies or interest groups. This article specifically considers neutral information campaigns—for example, literacy groups and community centres that want their clients to make knowledgeable decisions but do not have allegiance to a “yes” or “no” campaign. Additionally, official information campaigns may be presented by the electoral authorities. These bodies are often tasked with providing enough information for citizens to make a reasoned decision, while not promoting a specific position.

Canada's provincial governments have tried a variety of approaches to educating the citizenry while remaining neutral. British Columbia's first referendum in 2005 on electoral reform featured media advertisements, an information hotline, a website, and brochures mailed to each household. The government also employed provincial liaison officers for targeted population groups, including Aboriginal voters, Indo-Canadians, Chinese-Canadians and youth (Chief Electoral Officer of British Columbia, 2005). Ontario's referendum two years later used many similar strategies, including advertisements, public meetings, a website and brochures. Ontario also expanded the liaison programme introduced in British Columbia, introducing a referendum resource officer programme, where an individual was hired in each electoral district to give presentations to community groups and the public and to act as a community source of information (Chief Electoral Officer of Ontario, 2008).

A 2014 study on this programme, however, revealed some interesting feedback provided by these referendum resources officers on the materials provided to them for achieving their mission of educating the public. According to many of them, one of the most common complaints heard from voters was difficulty in understanding the texts provided by Elections Ontario (Garnett, Reference Garnett2014). In Elections Ontario's final report, it was noted that voters were presented with information at a grade 9 reading level. The report stated that attempts to rewrite the materials to a lower reading level had resulted in “inaccurate, confused or unclear” concepts (Chief Electoral Officer of Ontario, 2008). This leads to the question: At what literacy level should information about electoral reform be presented?

Hypotheses

This article ascertains the impact of information provided at three reading levels: grades 6, 9 and 12. In general, we hypothesize that the respondents who are provided information about the electoral reform proposal at the lowest reading level will exhibit the greatest retention of information, levels of interest and positive opinions on electoral reform. We expect that those who receive information at higher reading levels will be ill equipped to understand the proposal because they are simply unable to digest the information. While some nuance may need to be left out at the lowest reading level, the respondents that receive this information will actually retain the greatest amount of information and will be more interested in the proposal since they know more about it, reducing a status quo bias.

We measure three main dependent variables. First, this article will test respondents’ knowledge of the electoral reform proposal. Can the respondent identify key facts about the electoral reform proposal and understand how the electoral reform, if implemented, would affect their voting experience? This article hypothesizes that respondents that received the most digestible information (we predict this will be the grade 6 reading level) will respond most correctly to factual questions about the electoral system presented [H1a]. We also test subjective information levels, predicting that there may be some difference between what voters think they know and what they actually know. Using both measures can provide insights into this debate. Thus, we also hypothesize that the respondents who received a grade 6 level of information will have greater subjective information levels [H1b].

Second, this article tests the impact of the information on interest in an electoral reform proposal. When voters can understand a proposal better, they may be more likely to take interest in the proposal. We predict that those provided with grade 6 reading level text will be most interested in the issue of electoral reform after reading the text [H2]. Those respondents who have received the most complicated text (grade 12) may simply lose interest because they do not understand what the text is explaining. Those who received simpler information can digest the information better and are thus more likely to develop an interest in the subject.

Finally, a number of researchers have demonstrated that those who are more knowledgeable about a referendum proposal will be more likely to support it [H3]. We know that there is some status quo bias among low-information voters (Cutler and Fournier, Reference Cutler and Fournier2007; Elkink and Sinnott, Reference Elkink and Sinnott2015; LeDuc, Reference LeDuc2011; Stephenson and Tanguay, Reference Stephenson and Tanguay2009; Vowles, Reference Vowles2013). Those who know little about a proposal are unlikely to support it and instead, being risk averse, fall back to the status quo. We therefore predict that those who receive the text at a grade 6 reading level will also be the most likely to support electoral reform, since they better understand the proposed electoral system.

Method

To test these hypotheses, an experimental design was used in which respondents were provided with texts on an electoral reform proposal at varying reading levels (in addition to a control text). Before proceeding, it is important to briefly define key terms regarding literacy levels and reading levels. The Programme for the International Assessment of Adult Competencies (PIAAC), an organization that conducts international surveys to benchmark adult learning, defines literacy as “understanding, evaluating, using, and engaging with written texts to participate in society, to achieve one's goals, and to develop one's knowledge and potential” (Programme for the International Assessment of Adult Competencies, n.d.). They further identify six literacy levels, ranging from below 1 to level 5. It is suggested that in Canada, level 3 would be needed to complete high school and that level 2 is needed to adequately to function in society (Jamieson, Reference Jamieson2006). Canada's results from a 2012 assessment averaged to a score in the higher end of level 2 (Statistics Canada, 2013).

Since PIAAC requires engagement with the text (it measures ability to follow written instructions, for example), it measures an individual's literacy level, not the reading level of a text. In this study, we instead use texts that are graded according to the common Flesch-Kincaid reading levels (Kincaid et al., Reference Kincaid, Peter, Fishbourne, Rogers and Chissom1975). We recognize that readability is only a first step toward understanding how readers engage with political texts. Future research will consider other factors, such as multiple media or digital texts. However, because so much of the information provided from EMBs comes in the form of flyers or websites (with written text), understanding how reading levels affect the retention of political information is important for a variety of practitioners, including educators and the media and the EMBs themselves. Furthermore, scholars will benefit from a better understanding of how political information and knowledge translates into political behaviour, and understanding the ability of citizens to interpret written information is a key first step in this relationship.Footnote 3

Thus, four information texts were created: a control text and three treatment texts (see online Appendix A for full experiment materials). The control text is at a grade 6 reading level and considers a topic unrelated to electoral reform. The three treatment texts all consider electoral reform in Canada in the form of a shift to single-transferable vote (STV). This proposal was chosen since it has been suggested in the Canadian context but was not specifically the subject of Ontario's referendum on electoral reform (which considered a mixed-member proportional system).Footnote 4 The initial text was adapted from Samara Canada's guides on electoral reform (Samara Canada and Prest, Reference Canada and Prest2016).

A clear language consultant was hired to rewrite the text to a grade 6, grade 9 and grade 12 level. The final reading levels were close to the targeted grades (grade 5.9, 8.7 and 12). The process of revising the text began with text at the grade 12 level, which was then revised to a grade 6 and 9 level. The consultant took into consideration the principles of clear design: using more white space, down-style punctuation, shorter sentences, numerals, replacing multi-syllable words with direct language, and pulling information out of paragraphs into lists. While many consider reading level to refer only to the types of sentences and words used, it is important to note that clear design principles (for example, the amount of white space) are also key components of evaluating the reading level of a document. See online Appendix B for an outline of the process used by the clear language consultant.

To test the impact of the reading level of information provided, 474 student participants were recruited from an Ontario college through their instructors, who allowed the researcher or the researcher's assistant to administer the experiment (using traditional pen-and-paper texts and surveys) during the final 20 minutes of a class during the fall semester of 2018. The classes were diverse in their levels (ranging from first year to fourth year) and discipline (including classes in nursing, communications, business and policing). The first five minutes of the researcher's time spent in the classroom were used to explain the procedure. Participants in each class were then randomly handed a package containing a consent/information letter, one of three treatment texts or a control text, and a survey that included socio-demographic and attitudinal variables in addition to indicators of the main outcomes. Once the experiment began, the students were asked to wait the full 15 minutes of the experiment before leaving, if at all possible, to avoid disruption. While all students had the option to leave at any point in time (as per ethics requirements), nearly all students remained for the full 15 minutes. The control and treatment groups were similarly sized (each had between 110 and 127 participants).

Because the classes of participants were recruited from technical colleges and trades schools, rather than universities, the participants were slightly closer to a low-information population, although we recognize that college students are also highly educated, having completed high school. Education is known to be a predictor of both political knowledge (Barabas et al., Reference Barabas, Jerit, Pollock and Rainey2014) and voter turnout (Smets and van Ham, Reference Smets and van Ham2013), given both its direct impact on the skills of the respondent and indirect influence on other factors such as affluence and socialization. It is therefore important not to study only the most educated segment of the population (that is, university-level students).

The dependent variables (see Figure 1) were measured through the survey questions included with the text. Note that respondents had access to the text while responding to the survey. This access mirrors most individuals’ interaction with information provided by, for example, EMBs, which remain continually accessible up to a referendum or election. The variable measuring knowledge is constructed through a combined scale of two batteries of questions. The first battery is 10 true/false questions about the electoral reform proposal. The second battery of 5 questions more specifically addresses how a voter would interact with the new system. To avoid students’ guessing, a “don't know” option was also provided. See online Appendix C for a list of these questions. Mokken scaling was used to test the scalability of these questions. It finds that 14 of the 15 questions scale together. This article uses a scale ranging between 0–1 to report on the proportion of the 15 questions answered correctly. A robustness check with a scale consisting of the proportion of correct answers for only the 14 questions that scale together is found in online Appendix D.

Figure 1 Experimental Design

In addition to these true/false questions, respondents were also asked for a self-assessment on how informed they believed themselves to be on the issue of electoral reform. They were asked: “How informed do you think you are about the issue of electoral reform?” and were prompted to respond on a 0–10 scale, where 0 indicates “Not at all informed” and 10 indicates “Very well informed.” This article is also interested in whether the texts have an impact on the respondent's interest in electoral reform in general. Respondents were asked: “On a scale of 0–10, how interested are you on the issue of electoral reform?” where 0 indicates “Not at all interested” and 10 indicates “Very interested.” Finally, respondents were asked about their opinions on the proposed STV system explained in the treatment text. They were asked how likely they would be to vote in a referendum in Canada on electoral reform if held today, in which the options were: “The existing electoral system (First-Past-the-Post),” “Single transferable vote,” and “Don't know / no opinion.”

Several control variables are also considered in the analysis. These are drawn from the most important predictors of political knowledge and specific studies on knowledge of, and support for, electoral reform (Vowles, Reference Vowles2013). These variables include standard socio-demographic variables (gender, age, education), whether the respondent's first language is English, general political knowledge (measured through a battery of knowledge questions) (Vowles, Reference Vowles2013) and political interest, among others. This article checks for comparability between the groups receiving the control text and the three treatment texts using these control variables. See online Appendix E for tests demonstrating the comparability of control and treatment groups.

Results

The results reported here show simple analysis of variance and cross-tabulations. See online Appendix F for means and standard deviations in chart format. Further results with regression analysis and controls are available in online Appendix G, though the results are similar.

The mean knowledge scores (Figure 2) across groups help demonstrate the impact of the reading level of the text provided on the respondent's ability to answer questions about the electoral reform proposal. We hypothesized that those with the information that was easiest to digest (grade 6) would have the highest levels of both objective and subjective knowledge [H1]. Groups of respondents that received any of the three texts concerning electoral reform have higher means on the knowledge scale than the control text, as expected. However, there is a notably smaller mean score among those who received the grade 12 text. An analysis of variance (ANOVA) demonstrates statistically significant differences at the p < .10 level (Tukey's honestly significant difference) across all groups, except between the grade 6 and grade 9 reading levels. This suggests that the grade 6 and grade 9 reading level texts were better able to convey information about electoral reform than the grade 12 text. It is possible that this is due in part to the increased white space in the grade 6 and 9 texts, which was not present in the more prose-like grade 12 text. The respondents were better able to acquire information about the electoral reform proposal when the text, following the principles of clear design, included more white space and shorter sentences.

Figure 2 Mean Knowledge Scores on Electoral Reform Proposal

Note: 95% confidence intervals are depicted.

However, there is a different finding for subjective levels of information, as gauged through the question of how well informed respondents think they are about electoral reform.Footnote 5 Here, there are no statistically significant differences across the groups (see Figure 3 for group means; p < .10). This suggests that while objective measures of factual information differ across groups, respondents are unable to perceive these differences. This finding is consistent with research that shows that people tend to overestimate their own level of information (Ran et al., 2006; Dunning et al., Reference Dunning, Heath and Suls2004) and suggests that future study of civic literacy should consider objective measures rather than self-assessments.

Figure 3 Mean Subjective Assessments of How Well Informed the Respondent Is about Electoral Reform

Note: 95% confidence intervals are depicted.

Additionally, this study hypothesized that the reading level of texts provided to a respondent would have an impact on that person's interest in electoral reform, again with those who received the lowest reading level text (grade 6) having the greatest amount of interest because they could better understand the issue [H2]. In other words, those who understand more about the proposal (due to a more digestible text) will find the subject more interesting. However, the results of the experiment demonstrate that there is no statistically significant difference across groups according to the mean levels of interest in electoral reform (p > .10) (see Figure 4). It is important to note, however, that the findings for interest are only a snapshot taken immediately after the text was provided. We are unable to assess whether interest levels remained fixed over a longer period or whether this effect would differ across groups.

Figure 4 Mean Interest in Electoral Reform

Note: 95% confidence intervals are depicted.

Finally, we are interested in determining whether those who receive more digestible texts on an electoral reform proposal differ in opinions on that reform. We expected that those with grade 6 texts would be most likely to express an opinion in favour of the electoral reform proposed in the text [H3]. Figure 5 depicts the group means for responses to: “Which electoral system should Canada use to elect members to the House of Commons?”Footnote 6 A cross-tabulation with a chi-square test demonstrates that there are statistically significant variations in opinions across the different groups at the p < .05 level. Those who received any text on electoral reform were more likely to have an opinion (any opinion, whether positive or negative). We see that more comprehensible texts allow for the voter to form an opinion on STV, whether positive or negative (fewer respondents marked “don't know” or had “no opinion”). Those who received the grade 6 level text were the least likely to respond that they did not have an opinion on reform.

Figure 5 Opinions on Electoral Reform by Treatment Group

Additionally, those with the more comprehensible text were more likely to respond in favour of STV when compared with those in the control group. Specifically, those with the grade 6 level text were most likely to respond in favour of the STV proposal. This result suggests that reading a comprehensible text on electoral reform will increase support for the electoral reform proposal. This supports the idea that when a voter is unable to understand a proposal, that person will likely resort to the status quo—hence the theory of a status quo bias at play in previous electoral reform referendums.

Finally, it is interesting to note that when information is more easily understood—that is, at a grade 6 or 9 reading level—the proportion of those who report no opinion is lower, while the percentage of those in favour of STV is higher. It is possible (though we are unable to say for certain if this is the case) that those who would have reported having no opinion instead state they are in favour of STV when they are provided with more comprehensible text. The proportion against STV (in favour of the current FPTP system), however, is consistent across treatment and control groups.

Discussion and Conclusion

The results of this experiment suggest that when EMBs, civil society or the media are attempting to inform the public about complex political issues such as electoral reform, achieving a grade 6 or 9 reading level in written text is ideal for retention of information. The group that received information at a grade 6 reading level had the highest mean knowledge scores. This supports research indicating that clear language and design found at lower reading levels are key to facilitating citizens’ understanding of the information provided to them by any government agency, not only EMBs.

This research further provides evidence that objective and subjective levels of knowledge may be quite different. In this study, subjective levels of knowledge, measured by how much a respondent thought they knew about electoral reform, were not significantly different across the four texts provided. This result can lead us to conclude that voters may not always know what they don't know, which has implications for research design both for scholars studying political knowledge and for programme evaluators seeking to determine the effectiveness of information campaigns. It suggests that they should look beyond how much voters think they know about a subject or proposal to their objective knowledge.

The second set of analyses, considering interest in electoral reform, reveals that there is no guarantee that any information about electoral reform procedures, regardless of the reading level, will spark greater curiosity in the topic. Further research may consider different political issues to gauge whether this is specific to the issue of electoral reform. It may also consider whether more persuasive information, such as campaigns from different sides of a referendum debate, may spark greater interest in the issue.

Finally, this research supports previous findings that suggest that those who have learned more about an electoral reform proposal are more likely to support it. The group that received the grade 6 text had the highest percentage of those in favour of the STV system and the lowest percentage of those who did not report an opinion on electoral reform. This experiment supports the theory that there is a status quo bias, wherein lower-information voters may be more risk averse and less likely to vote in favour of an unfamiliar proposal (Hessami and Resnjanskij, Reference Hessami and Resnjanskij2019; Nadeau et al., Reference Nadeau, Martin and Blais1999).

There are, of course, some limitations to the present study. First, the study was conducted with college-level students; thus the findings are likely to be even more pronounced for citizens with lower education. However, our findings can be interpreted as a conservative estimate of any impact of readability on the ability to digest, retain and integrate political information. Additionally, we note that the survey was administered immediately after the text was read and that respondents had access to the text while completing the survey. We cannot comment on whether the results remain consistent if respondents were surveyed at some later point or if the text was removed prior to starting the survey.

What are the implications of this research for EMBs, civil society, the media and other educators seeking to inform citizens on complex political issues? A few recommendations can be suggested based on the findings of this research.

First, provide information at lower reading levels, keeping in mind clear design principles. In the Ontario referendum on electoral reform, for example, original materials were well beyond a grade 6 reading level. Our research supports the requests of referendum resource officers (who are on the front lines of referendum education) that educators be equipped with materials at lower reading levels (Garnett, Reference Garnett2014). Sacrificing greater complexity and nuance for the ability of voters to understand the information may be a worthwhile trade-off for voters’ overall ability to form an opinion. Rather than having lower levels of knowledge, those who received the less complex information had greater knowledge of the electoral reform proposal. There does not appear to be a loss in practical information when presenting the material at an easier-to-digest reading level.

However, it is also important to acknowledge that the type of information may have an impact on the results of a referendum. In this study, those who were provided with information at lower literacy levels were more likely to form an opinion and support the STV proposal. Thus, we conclude that governments and EMBs should be aware that doing nothing, or very little, is still a political decision. Low-information voters have a status quo bias, and thus decisions to provide little information, or provide information in an inaccessible manner, will support the status quo in a referendum. Thus, the provision of information is rarely apolitical, and choices about how information is provided are key to the result of a referendum campaign.

Finally, consider incorporating alternative strategies for educating citizens, perhaps ones that go beyond the provision of neutral information. In this study, the non-partisan technical information was not able to increase interest in the electoral reform proposal. We know that one of the greatest barriers in many referendums is low turnout. Consider, for example, the need for a second electoral reform referendum in Prince Edward Island after low turnout in the first led the premier to question the results (Sinclair, Reference Sinclair2016). While this study was unable to directly test whether neutral information was itself a problem, future research should consider the effectiveness of alternative education strategies that do not rest solely on neutral information, such as funding “yes” and “no” campaigns, as was done in the case in British Columbia (Canadian Press, 2018).

In conclusion, these findings not only contribute to the body of academic literature on political decision making and civic education but also provide useful advice for a variety of practitioners, including members of the media, scholars and election administrators who design civic education campaigns. The findings should be used to inform future campaigns, as well as scholarly debate over the role of information, political knowledge and civic literacy.

Acknowledgments

The research for this article was funded by a grant from the Centre for the Study of Democratic Citizenship, which is funded by the Fonds de recherche du Québec—Société et culture. With thanks to research assistants Samantha Twietmeyer, Joseph Szeman, Desmond Barton, Gant Crocker and Madison MacGreggor.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/S0008423920000591

Footnotes

1 Additional factors, such as the thresholds required for passing, the political context and the types of systems proposed, are also important (see Garnett, Reference Garnett2014; Lea, Reference Lea2006; Robertson, Reference Robertson2009; Stephenson and Tanguay, Reference Stephenson and Tanguay2009; Vowles, Reference Vowles2013).

2 The authors make a distinction between objective and subjective knowledge of the proposal and suggest that subjective knowledge is a more important predictor of a “yes” vote.

3 See, for instance, Williamson and Martin (Reference Williamson and Martin2010), who show that the majority of patient information leaflets provided by hospitals exceed patient comprehension.

4 Note that only 15 respondents stated they had voted in the 2007 Ontario referendum on electoral reform. Most respondents were too young at the time to take part in the 2007 referendum.

5 Subjective knowledge and objective knowledge are correlated (r = .43, p < .01).

6 Respondents were forced to choose either the existing system or the one explained in the text (since we could not assume that they knew about other electoral systems).

References

Barabas, Jason, Jerit, Jennifer, Pollock, William and Rainey, Carlisle. 2014. “The Question(s) of Political Knowledge.” American Political Science Review 108 (4): 840–55.CrossRefGoogle Scholar
Bartels, Larry M. 1996. “Uninformed Votes: Information Effects in Presidential Elections.” American Journal of Political Science 40 (1): 194230.CrossRefGoogle Scholar
Bittner, Amanda. 2007. “The Effects of Information and Social Cleavages: Explaining Issue Attitudes and Vote Choice in Canada.” Canadian Journal of Political Science 40 (4): 935–68.Google Scholar
Burden, Barry. 2009. “The Dynamic Effects of Education on Voter Turnout.” Electoral Studies 28 (4): 540–49.CrossRefGoogle Scholar
Campbell, Angus, Converse, Philip E., Miller, Warren E. and Stokes, Donald E.. 1960. The American Voter. New York: John Wiley.Google Scholar
Canada, Samara, and Prest, Stewart. 2016. What We Talk about When We Talk about Electoral Reform. August 25. http://www.samaracanada.com/docs/default-source/Reports/electoral-reform-report.pdf?sfvrsn=4.Google Scholar
Canadian Press. 2018. “Yes and No Groups Selected before Referendum on Electoral Reform in B.C.” National Post, July 12. https://nationalpost.com/pmn/news-pmn/canada-news-pmn/yes-and-no-groups-selected-before-referendum-on-electoral-reform-in-b-c.Google Scholar
Chief Electoral Officer of British Columbia. 2005. Statement of Votes: Referendum on Electoral Reform, May 17, 2005. Elections BC. http://www.elections.bc.ca/docs/rpt/SOV-2005-ReferendumOnElectoralReform.pdf.Google Scholar
Chief Electoral Officer of Ontario. 2008. Provincial Referendum on Electoral System Reform: Report of the Chief Electoral Officer, October 10, 2007. Toronto: Elections Ontario.Google Scholar
Christin, Thomas, Hug, Simon and Sciarini, Pascal. 2002. “Interests and Information in Referendum Voting: An Analysis of Swiss Voters.” European Journal of Political Research 41 (6): 759–76.Google Scholar
Clarke, Harold D., Goodwin, Matthew and Whiteley, Paul. 2017. “Why Britain Voted for Brexit: An Individual-Level Analysis of the 2016 Referendum Vote.” Parliamentary Affairs 70 (3): 439–64.CrossRefGoogle Scholar
Cutler, Fred and Fournier, Patrick. 2007. “Why Ontarians Said No to MMP.” Globe and Mail, October 25.Google Scholar
Downs, Anthony. 1957. An Economic Theory of Democracy. New York: Harper & Row.Google Scholar
Dunning, David, Heath, Chip and Suls, Jerry M.. 2004. “Flawed Self-Assessment: Implications for Health, Education, and the Workplace.” Psychological Science in the Public Interest 5 (3): 69106.CrossRefGoogle ScholarPubMed
Elkink, Johan A. and Sinnott, Richard. 2015. “Political Knowledge and Campaign Effects in the 2008 Irish Referendum on the Lisbon Treaty.” Electoral Studies 38 (5): 217–25.CrossRefGoogle Scholar
Garnett, Holly A. 2014. “Lessons Learned: Referendum Resource Officers and the 2007 Ontario Referendum on Electoral Reform.” Canadian Political Science Review 8 (1): 6384.Google Scholar
Hessami, Zohal and Resnjanskij, Sven. 2019. “Complex Ballot Propositions, Individual Voting Behavior, and Status Quo Bias.European Journal of Political Economy 58: 82101CrossRefGoogle Scholar
Hobolt, Sara B. 2007. “Taking Cues on Europe? Voter Competence and Party Endorsements in Referendums on European Integration.” European Journal of Political Research 46 (2): 151–82.CrossRefGoogle Scholar
Jamieson, Donald G. 2006. “Literacy in Canada.” Paediatrics & Child Health 11 (9): 573–74.CrossRefGoogle ScholarPubMed
Jenssen, Anders T. and Listhaug, Ola. 2001. “Voters’ Decisions in the Nordic EU Referendums of 1994: The Importance of Party Cues.” In Referendum Democracy: Citizens, Elites and Deliberation in Referendum Campaigns, ed. Mendelsohn, Matthew and Parkin, Andrew. London: Palgrave Macmillan UK.Google Scholar
Katz, Richard S. 2004. “Problems in Electoral Reform: Why the Decision to Change Electoral Systems Is Not Simple.” In Steps Towards Making Every Vote Count: Electoral System Reform in Canada and its Provinces, ed. Milner, Henry. Peterborough: Broadview Press.Google Scholar
Kincaid, J. Peter, Robert P. Fishbourne, Richard L. Jr. Rogers, Brad S. Chissom, . 1975. “Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel.” Research Branch Report 8-75. Institute for Simulation and Training, University of Central Florida. https://stars.library.ucf.edu/istlibrary/56/.Google Scholar
Lassen, David D. 2005. “The Effect of Information on Voter Turnout: Evidence from a Natural Experiment.” American Journal of Political Science 49 (1): 103–18.CrossRefGoogle Scholar
Lau, Richard R. and Redlawsk, David P.. 2006. How Voters Decide: Information Processing during Election Campaigns. New York: Cambridge University Press.CrossRefGoogle Scholar
Lea, Jeannie. 2006. “The Prince Edward Island Plebiscite on Electoral Reform.” Canadian Parliamentary Review 29 (1): 48.Google Scholar
LeDuc, Lawrence. 2011. “Electoral Reform and Direct Democracy in Canada: When Citizens Become Involved.” West European Politics 24 (3): 551–67.Google Scholar
Lupia, Arthur and McCubbins, Matthew D.. 1998. The Democratic Dilemma: Can Citizens Learn What They Need to Know? Cambridge: Cambridge University Press.Google Scholar
Matsusaka, John G. 1995. “Explaining Voter Turnout Patterns: An Information Theory.” Public Choice 84 (1): 91117.CrossRefGoogle Scholar
McAllister, Ian. 2001. “Elections without Cues: The 1999 Australian Republic Referendum.” Australian Journal of Political Science 36 (2): 247–69.Google Scholar
Nadeau, Richard, Martin, Pierre and Blais, André. 1999. “Attitude toward Risk-Taking and Individual Choice in the Quebec Referendum on Sovereignty.” British Journal of Political Science 29 (3): 523–39.CrossRefGoogle Scholar
Nagel, Jack H. 1994. “What Political Scientists Can Learn from the 1993 Electoral Reform in New Zealand.” Political Science & Politics 27 (3): 525–29.Google Scholar
Popkin, Samuel L. 1991. The Reasoning Voter: Communication and Persuasion in Presidential Campaigns. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Programme for the International Assessment of Adult Competencies (PIAAC). n.d. “Frequently Asked Questions.” http://piaac.ca/590/FAQ.html.Google Scholar
Robertson, A. B. a. J. R. 2009. Electoral Reform Initiatives in Canadian Provinces. http://www2.parl.gc.ca/Content/LOP/ResearchPublications/prb0417-e.htm#British2.Google Scholar
Roy, Jason. 2009. “Voter Heterogeneity: Informational Differences and Voting.” Canadian Journal of Political Science 42 (1): 117–37.CrossRefGoogle Scholar
Sinclair, Jesara. 2016. “Premier Calls Plebiscite Results ‘Debatable,’ Cites Low Turnout.” https://www.cbc.ca/news/canada/prince-edward-island/pei-premier-plebiscite-results-1.3842107.Google Scholar
Smets, Kaat and van Ham, Carolien. 2013. “The Embarrassment of Riches? A Meta-analysis of Individual-Level Research on Voter Turnout.” Electoral Studies 32 (2): 344–59.CrossRefGoogle Scholar
Statistics Canada. 2013. “Skills in Canada: First Results from the Programme for the International Assessment of Adult Competencies (PIAAC).” http://www.deslibris.ca/ID/240077.Google Scholar
Stephenson, Laura and Tanguay, Brian. 2009. “Ontario's Referendum on Proportional Representation: Why Citizens Said No.IRPP Choices 15 (10). https://irpp.org/wp-content/uploads/assets/research/strengthening-canadian-democracy/why-do-canadians-say-no-to-electoral-reform/vol15no10.pdf.Google Scholar
Verba, Sidney, Schlozman, Kay Lehman, and Brady, Henry E.. 1995. Voice and Equality: Civic Voluntarism in American Politics. Cambridge: Harvard University Press.Google Scholar
Vowles, Jack. 2013. “Campaign Claims, Partisan Cues, and Media Effects in the 2011 British Electoral System Referendum.” Electoral Studies 32 (2): 253–64.CrossRefGoogle Scholar
Williamson, James M. L. and Martin, Alexandra G. 2010. “Analysis of patient information leaflets provided by a district general hospital by the Flesch and Flesch–Kincaid method.” International Journal of Clinical Practice 64: 18241831. doi:10.1111/j.1742-1241.2010.02408.x.Google ScholarPubMed
Figure 0

Figure 1 Experimental Design

Figure 1

Figure 2 Mean Knowledge Scores on Electoral Reform ProposalNote: 95% confidence intervals are depicted.

Figure 2

Figure 3 Mean Subjective Assessments of How Well Informed the Respondent Is about Electoral ReformNote: 95% confidence intervals are depicted.

Figure 3

Figure 4 Mean Interest in Electoral ReformNote: 95% confidence intervals are depicted.

Figure 4

Figure 5 Opinions on Electoral Reform by Treatment Group

Supplementary material: File

Garnett and Blais Supplementary Materials

Garnett and Blais Supplementary Materials

Download Garnett and Blais Supplementary Materials(File)
File 91 KB