Collaborative methodologies provide a creative corrective to traditional approaches to measurement and evaluation in political science, as the contributions to this symposium attest. Unlike in typical quantitative research, these approaches treat participants as full collaborators rather than subjects. More specifically, the production of “participatory numbers” relies on participants to define key terms and potential measures and, at times, to collect data (Gaillard et al. Reference Gaillard, Cadag, Gampell, Hore, Le De and McSherry2016). In this symposium alone, authors apply collaborative methodologies to the study of political trust (Dorussen, Bakaki, and Kolbe Reference Dorussen, Bakaki and Kolbe2021), peace (Levy and Firchow Reference Levy and Firchow2021; Thomson Reference Thomson2021), and climate change (Asiamah, Awal, and MacLean Reference Asiamah, Awal and MacLean2021). Collaborative methodologies promise many benefits, but chief among them is a new balance between subjectivity and objectivity. On the one hand, collaborative methodologies have, at their core, an activist mission to decolonize knowledge production and empower researchers and communities in the Global South (but see Gellman’s Reference Gellman2021 discussion of the challenges to capturing these benefits). On the other hand, they also promise methodological rigor that frees findings from accusations of bias, as Kaplan (Reference Kaplan2021) emphasizes in his article.
Nevertheless, collaborative approaches to research have not yet caught fire in the study of democracy, which remains dominated by cross-national data collection and experimental analyses of causal relationships. These studies rely heavily on a standardized definition of democracy developed by Western scholars. Ironically, research on political democracy has been largely undemocratic and top down: rarely have scholars devoted meaningful energy to uncovering alternative understandings of democracy. Consequently, political scientists understand little about how citizens make meaning of democracy and how those meanings differ across time and space. Neither are these purely academic questions: standardized measures of democratic governance are an essential component of democracy promotion. These definitions have political power.
Ironically, research on political democracy has been largely undemocratic and top down: rarely have scholars devoted meaningful energy to uncovering alternative understandings of democracy. Consequently, political scientists understand little about how citizens make meaning of democracy and how those meanings differ across time and space.
This article proposes Everyday Democracy Indicators (EDI) to explore the benefits and challenges of using collaborative methods to study democracy. The thought experiment imparts new lessons for both practitioners of collaborative methodologies and students of comparative politics. EDI would address a significant blind spot in the study of democracy by exploring how citizens define and make meaning of democracy in their daily lives. It would allow a fuller exploration of whether standardized, top-down definitions of democracy actually resonate with citizens in diverse cultural settings. A project such as EDI could prove partially emancipatory by shifting the power of conceptual development and measurement away from the halls of Western universities and toward the locales being studied, with significant implications for the practice of democracy promotion.
Notwithstanding these benefits, collaborative methodologies also face an uphill battle to being adopted in the study of democracy—and, indeed, for an entire class of concepts that, like democracy, are foundational, multidimensional, and contested. EDI would do little to further the field’s understanding of how democratic practice changes over time—a pivotal question in studies of democracy. Furthermore, the contested nature of democracy as a concept raises further challenges: for example, micro-level variation within communities may result in more powerful subgroups driving indicator development.
This article begins with a brief survey of the state of measurement in the field of comparative democratization, emphasizing the field’s reliance on Western-derived, standard definitions of democracy that power measurement and efforts to promote democracy around the world. Next, it explores how that reliance limits scholars’ understanding of how citizens define and make meaning of democracy. The article then outlines a proposal for EDI as a means to precisely address these questions, particularly by allowing direct analysis of variation in definitions across time and space, as well as the emancipatory impact of elevating citizens’ voices. It concludes with a discussion of the challenges of fielding the EDI and the broader lessons for using collaborative methodologies in political science.
THE VALUE OF EDI
The study of democracy in political science has benefited surprisingly little from collaborative methodologies or how people’s conceptualizations of democracy might vary meaningfully across contexts. Instead, democracy traditionally has been measured mainly through institutionalized cross-national data-collection projects. Efforts such as Polity IV (Marshall and Jaggers Reference Marshall and Jaggers2002) and Varieties of Democracy (V-Dem) (Coppedge et al. Reference Coppedge2018) focus on creating standardized definitions of democracy (and political regimes, more broadly) and then employing regional and international experts to develop indicators and assess performance. Other projects capture data on specific elements of democratic practice, such as the National Elections Across Democracy and Autocracy (Hyde and Marinov Reference Hyde and Marinov2012) and the Electoral Integrity Project (EIP) (Norris and Grömping Reference Norris and Grömping2019) databases on electoral conduct. Projects such as V-Dem and EIP create detailed data on the de jure form and de facto function of formal democratic practice.
Scholars of democracy also rely on cross-national survey projects (e.g., the various Barometer Projects, the World Values Survey, and the Comparative Study of Electoral Systems) to assess voters’ attitudes toward democracy worldwide. Respondents rate their support of democracy relative to other regime types and their satisfaction with democracy’s general function and specific institutions (e.g., judiciaries). Cross-national surveys allow for micro-level analysis of support for democracy and have asked what democracy means to people (Diamond and Plattner Reference Diamond and Plattner2008).
The definitions underlying these projects inform democracy-promotion strategies, programming, and evaluation. For example, the United States Agency for International Development (USAID) framework for promoting democracy and human rights depends on definitions developed in political science. It also encourages assessment teams to consult statistical databases on democracy (USAID 2014, 4). Neither is this influence incidental: these projects transparently seek to promote democracy by carefully analyzing its spread and deterioration. The EIP, for example, states that a core mission is to encourage countries to adhere to international standards for electoral conduct.Footnote 1 Rigorous research into violations of electoral integrity helps democracy promoters understand where and how democracy backslides. V-Dem reports focus on the success of democracy and the threat of “autocratization.” Objectivity and methodological rigor are core to these projects’ efforts to analyze democracy. Nevertheless, a dedication to democracy promotion undoubtedly is also a foundational value. Kelley (Reference Kelley2017) analyzes this phenomenon, finding that international non-governmental organizations and Western governments exploit public grades of state performance on human rights to pressure governments to change their behavior.
Scholars have criticized top-down definitions of democracy for compressing spatial heterogeneity and limiting research into how conceptions of democracy differ across time and space (Schaffer Reference Schaffer2014). The hegemonic approach to the study of democracy relies on creating a standardized definition of democratic practice informed primarily by the Western experience of democracy and then assessing a country’s “performance” according to this definition. Comparative data on political institutions occasionally rely on local knowledge: V-Dem, for example, involves 3,000 country experts to produce its global dataset (Lührmann et al. Reference Lührmann2020, 5). They do little to assess how citizens in different cultural contexts understand democracy, however. Comparative survey projects provide deep insight into respondents’ attitudes toward democracy. Nevertheless, they cannot explore how respondents define and make meaning through democratic practice, precisely because they must rely on a standardized set of understandings of democracy in the first place. Yet, previous scholarship strongly indicates that people define democracy in critically different ways. Schaffer (Reference Schaffer2000), for example, reveals how Senegalese citizens understood their participation in elections as part of a process of creating collective security, as opposed to choosing leaders freely.
This lacuna represents an opportunity for EDI, which would center its investigation not on participants’ views on a standardized version of democracy but rather on how they define and make meaning regarding democratic practice in their daily lives. The general outline would work much like the Everyday Peace Indicators (EPI), whose collaborative approach to studying peace is a model (Firchow Reference Firchow2018).Footnote 2 First, focus groups in a locality would engage participants in a process of defining democracy and then brainstorming and prioritizing different indicators for it. This open process may generate measures of democracy that vary substantially from top-down, standardized definitions. Second, randomized surveys would measure community members’ perceptions of the strength of democracy in their daily lives.
This lacuna represents an opportunity for EDI, which would center its investigation not on participants’ views on a standardized version of democracy but rather on how they define and make meaning regarding democratic practice in their daily lives.
EDI’s usefulness likely would pivot on how it sampled. Ideally, a pilot project purposively would choose several localities within the same country to observe variation in the definitions and indicators across urban–rural, class, and ethnic divides. The choice of pilot country would be critical, especially given concerns about the global decline of democracy (Diamond and Plattner Reference Diamond and Plattner2008; Levitsky and Ziblatt Reference Levitsky and Ziblatt2019). A pilot project in the United States would illustrate how citizens make meaning of democracy during a period of threats to its practice. Alternatively, a pilot project in a country seen as struggling to build democratic rule (e.g., Pakistan) or with a decidedly nondemocratic government (e.g., Eritrea) might reveal how citizens define a hypothetical democratic order. The project would build on Schaffer (Reference Schaffer2000) by creating a bottom-up understanding of how citizens define democracy and by adding a quantitative measurement of their assessment of its function.
The proposed EDI might contribute significantly to the study of comparative democratization and the practice of democracy promotion. First, the project could shed light on how citizens define democracy, make meaning of it, and experience it in their daily lives. Democracy, of course, remains a contested concept in both scholarly and public discourse. That contestation, however, largely excludes citizens—particularly outside of the Western world. Elite-driven debates—for example, whether democracy is culturally appropriate (Fukuyama Reference Fukuyama1995) or politically viable (Mansfield and Snyder Reference Mansfield and Snyder2007; Paris Reference Paris2004; Snyder Reference Snyder2000) in all contexts—often pit Western democracy promoters against would-be electoral authoritarians. Those debates, however, depend crucially on a shared acceptance of a standardized view of liberal democracy. This assumption remains untested. How do citizens define democracy, build meaning through participating in it, and rate its function in their daily lives?
Second, by conducting the project in different localities within or across countries, the project also could shed light on how and why localities diverge in their understanding of democracy (as discussed previously), posing a challenge to cross-national survey efforts. The eventual inclusion of more localities in several countries could reveal systematic drivers of this variation. Do conceptions of democracy vary significantly across countries with different levels of democratic experience? Does a history of ethnic polarization alter citizens’ understandings of democracy? EDI could begin to answer these questions.
Third, EDI could assist in the process of democracy promotion by elevating citizens’ voices. Global regimes for defining and measuring democracy have, at their core, the hope that rating country performance will incentivize political elites to adhere to democratic norms. The irony of these efforts at democracy promotion through measurement is their dependence on top-down, largely undemocratic processes headed by Western elites. A bottom-up approach practices democracy by empowering citizens to define what it means to them and then assesses its function by those standards. In doing so, EDI would transfer power from Western universities to people in locales being studied. Democracy promoters would benefit from understanding which aspects of democratic governance matter most to citizens, suggesting new avenues to promote deeper engagement in democratic practices.
THE CHALLENGE OF EDI
EDI could address critical limitations in political science’s understanding of democracy. Nevertheless, this proposal also illustrates the challenge of using collaborative methodologies to study political concepts such as democracy. First, contention over the definition of democracy provides as many challenges as opportunities to mounting EDI. Democracy is the subject of long-running global debates, as illustrated in this article. For example, the Lee thesis that democracy impedes economic development remains politically relevant, given China’s growth and its perceived success in responding to the COVID-19 pandemic; scholars have cast empirical doubt on the thesis, however (Bizzarro et al. Reference Bizzarro, John, Carl Henrick, Allen, Michael, Svend-Erik, Michael and Lindberg2018; Knutsen Reference Knutsen2010; Nooruddin Reference Nooruddin2011). Democracy’s link to the West has opened opportunities for elites to cast democracy as an imposition. People may regard other concepts studied using collaborative methodologies—peace, for example, as well as health—as superordinate sociopolitical goals.
Second, the more conceptually fraught nature of democracy might affect the interplay between focus groups and survey methods inherent in EDI. Focus groups in EDI play an indispensable role in the project’s first stage, identifying how participants experience democracy daily and developing indicators to measure that experience. The success of this first stage depends on whose voices are included and the skill of the facilitator, however. If powerful subgroups (e.g., men and elders) take precedence, a biased list of indicators will result, particularly where democracy is a politically contested goal. For example, women may find it challenging to propose indicators of democracy that center on their civic participation in patriarchal societies. In countries where political leaders actively subvert formal institutions or repress dissent among specific groups, citizens might refrain from such a politically sensitive discussion. After the focus groups generate indicators, survey design must attend carefully to the types of framing and priming effects identified by Dorussen and Bakaki (Reference Dorussen, Bakaki and Kolbe2021) in this symposium.
Third, scholars and citizens alike tend to define democracy in national terms. Despite the importance of federalism and local governance, democracy still connotes national elections, political parties, and issues. Whether people understand democracy as something functioning daily in their communities and lives in the same way that economic development, health, and peace do is an open question. People simply may not identify democracy as an everyday process or feeling. If not, EDI would risk creating that feeling through the research process instead of uncovering preexisting meaning making by citizens.
CONCLUDING THOUGHTS
This article proposes EDI as a means to understand the promise and challenges of leveraging collaborative methodologies to study political democracy. A brief summary of the field of comparative democratization reveals real opportunities for learning from collaborative methodologies. Heavy reliance on a top-down, standardized definition of democracy means that political scientists understand surprisingly little regarding how citizens define, experience, and make meaning of democracy and how those processes differ across time and space. Furthermore, collaborative methodologies are more democratic, and EDI could elevate citizen voices to democracy promoters, informing their perspectives and priorities.
Nevertheless, the proposed EDI also reveals the challenges of applying collaborative methods to the study of foundational, multidimensional, contested political concepts such as democracy. Scholars of comparative democratization understandably often focus on understanding why formal democracy emerges (or not) and perseveres (or not) across time and space. Even after significant scaling up, EDI would contribute only tangentially to this debate. EDI’s meso-level focus on communities also might obscure micro-level variation. The study of public opinion, for example, suggests that people’s definitions of “squishy” concepts such as democracy might vary meaningfully within communities. EDI’s focus on community-level definitions of democracy also might minimize variation in these intra-community definitions.
Nevertheless, the proposed EDI also reveals the challenges of applying collaborative methods to the study of foundational, multidimensional, contested political concepts such as democracy.
Micro-level variation raises another potential pitfall around power and indicator development. One risk is that voices running counter to a national or community-wide consensus might be silenced in the development of indicators, risking the creation of biased measures. In other words, even collaborative methodologies might engage in a different type of conceptual “flattening” if the first-stage focus groups lose the voices of women, ethnic minorities, and/or young people. Avoiding these pitfalls requires a cultural anthropologist’s knowledge of the social terrain and careful attention to how subaltern voices are incorporated. Intentional collaboration with in-country partners could prove useful in this regard, as Asiamah, Awal, and MacLean (Reference Asiamah, Awal and MacLean2021) discuss in this symposium.
Democracy is not the only complex and contested concept studied in political science. Peace is another, as we have seen, but so are political trust and justice. Collaborative methodologies provide new opportunities to understand citizens’ understanding of politics but also must adapt to the challenges of building bottom-up indicators of foundational, contested concepts.
