In this symposium, 36 diverse scholars discuss implications of identity and positionality in field experiments, particularly those that evaluate policy interventions (also called “impact evaluations” and “randomized controlled trials”).Footnote 1 Field experiments share many common identity and positionality issues with observational field research yet are distinct in two unique ways: (1) they involve interventions with the dual purpose of testing social science hypotheses as well as improving human welfare according to normative goals; and (2) they involve many actors beyond principal investigators and research participants.
Field experiments share many common identity and positionality issues with observational field research yet are distinct in two unique ways: (1) they involve interventions with the dual purpose of testing social science hypotheses as well as improving human welfare according to normative goals; and (2) they involve many actors beyond principal investigators and research participants.
Although this discussion is already deeply considered by scholars, this symposium provides a foundational starting point for written scholarly discourse regarding how these unique features of field experiments yield multifaceted considerations of identity and positionality. Together, we underscore how identity and positionality can affect the distinct goals of scientific rigor and ethics in field experiments.
Moving forward, we argue that field experimenters should engage in a continually reflexive process regarding identity and positionality throughout the research process. We provide resources in an online appendix that facilitate a reflexive process during projects as well as student methods instruction.
OVERLAP WITH EXISTING SCHOLARSHIP ON OBSERVATIONAL FIELD RESEARCH METHODS
Positionality is defined in political science as a researcher’s social location, perspective, orientation, and situatedness vis-à-vis participants derived from their identities and experiences (Fujii Reference Fujii2017; Soedirgo and Glas Reference Soedirgo and Glas2020). Identities include multiple personal and social identities (e.g., gender, class, and ethnicity), which often combine to form intersectional identities. Self-assigned identities may differ from identities assigned by others, and the activation of such identities as salient depends on social context (Henry, Higate, and Sanghera Reference Henry, Higate and Sanghera2009).
Political scientists have published scholarly discourse on the role of identity and positionality in observational field research. This work typically focuses on how identity and positionality can affect a study’s scientific rigor, its ethics, or both. In this symposium, authors elaborate on how positionality issues affect a study’s scientific rigor when they distort research processes or outcomes to bias study findings. Furthermore, contributors discuss ethical issues that may arise due to positionality concerns, not only for research participants but also for other actors involved (e.g., implementing partners and donors).Footnote 2
Of course, much discourse regarding observational field methods easily carries over into field experiments, given that field experiments typically incorporate many other research methodologies. For example, a growing literature examines how survey-enumerator identity or positionality triggers social-desirability concerns or uneven response rates among participants, undermining large-N survey-data quality (Lupu and Michelitch Reference Lupu and Michelitch2018). This literature informs best practices in surveys conducted during field experiments.
Moreover, such discourse can be especially relevant for field experimenters. For example, the PS symposium, “Whose Research Is It?” (Michelitch Reference Michelitch2018), discussed how scientific rigor and ethics can be improved for diverse methodologies when Global North–based researchers in Global South contexts include local populations throughout the research process. Given that the majority of field experiments are conducted by Global North researchers in Global South contexts (Corduneau-Huci et al., this issue), it is no surprise that Global North–Global South positionality concerns in field experiments already have received attention (Davis Reference Davis2020) and receive continued attention in this symposium (Herman et al., this issue).
UNIQUE CONSIDERATIONS FOR FIELD EXPERIMENTS
We contend that there are two unique aspects of field experiments that warrant distinct discussion of identity and positionality concerns for scientific rigor and ethics, as follows:
-
• intervention(s) based jointly on social science and normative grounds
-
• many actors beyond researchers and participants
Interventions Based Jointly on Social Science and Normative Goals
Field experiments that evaluate policy interventions constitute “activist scholarship.” They typically have a dual purpose to test social science theory and to discover whether certain policies improve human well-being according to normative goals. In fact, many political scientists are drawn to academia and field experiments because of a commitment to have a positive impact on the world. Whereas scholars typically present academic research as emanating from neutral-value social science theory, normative goals become easily apparent in other writing geared toward policy-maker, donor, and lay-person audiences.Footnote 3
Field experimenters often elide normative goals due to colleagues’ critiques that political scientists should not be engaging in policy-impactful research aimed at generating positive change for human welfare. These critiques are an oddity given that most natural and social science disciplines—and the grants used to fund them—not only acknowledge but also encourage policy-engaged scholarship aimed at welfare improvements. Furthermore, many university missions formally state a goal of achieving positive societal impact through research. At a minimum and to our knowledge, no ethical requirements exist in our discipline or at universities that research should avoid societal impact or must be value free.
That activist scholarship remains unaccepted by some disciplinary colleagues has an unfortunate consequence. Namely, field experimenters can be dissuaded—at least publicly—from openly examining potential ways that our identity and positionality have implications for the scientific rigor and ethics of research involving normative goals.
This symposium begins a conversation about these considerations for both scientific rigor and ethics. As symposium contributors explicate regarding diverse aspects of field experiments, researcher identities and experiences can shape the normative goals that prompt which research questions we ask, which like-minded donors and implementing partners we seek, which interventions and outcomes we consider, and how (and to whom) we disseminate research results.
This symposium begins a conversation about these considerations for both scientific rigor and ethics.
One major risk to research, for example, is that a scholar’s normative agenda may not be shared with or prioritized by participant populations. When a researcher’s normative goals are unshared, it leads to ethical questions about positionality: Who has the power to decide which normative agenda is important? Are some populations involved in research that is considered important by researchers but not locally important to participants? Herman et al. (this issue) discuss such positionality in Global South development research, and Hartmann et al. (this issue) discuss it for gender research. Whereas considering whether participants share the researcher’s normative goals is not formally on the list of Institutional Review Board ethical standards, it is a type of ethics that is increasingly valued by field experimenters.
Moreover, scientific rigor can be affected when normative agendas are unshared among researchers and research participants. Herman et al. (this issue) points out that participants often are familiar with researchers’ normative agendas. When participant populations or subordinate members of local research teams are deferential to researchers due to positionality, they may not reveal their true feelings about the reception of or potential backlash toward interventions. Speaking up also may vary based on researcher identity; for example, participants may be deterred to warn female researchers against feasibility of gender-equality interventions. Hartman et al. (this issue) underscore that subordinate research team members may have their own feelings for or against an intervention and purposefully or inadvertently affect implementation in ways that may affect outcomes.
A solution suggested by many symposium contributors is to involve local actors in research from start to finish (see especially Herman et al.). However, which individuals constitute “local actors”? Indeed, we must consider that identity and positionality issues exist within local communities that may render it difficult to access certain subpopulations. Long-term engagement with local populations may be necessary for comfort levels to allow for true feedback, compromise, and onboarding of diverse stakeholders.
Conversely, a tradeoff can emerge when research participants know the normative goals of an intervention due to their involvement, thereby adversely affecting scientific rigor in a field experiment. In particular, “experimenter demand effects” could be prompted when participants influence outcomes based on treatment status to please the researcher’s agenda or influence policy making (e.g., Gerber and Green Reference Gerber and Green2012). This methodological issue already has received attention in published work, but the focus typically is on a specific (very common) dyad of identity and positionality—that is, white (male) researcher identity and participants from traditionally marginalized groups (e.g., Cilliers, Dube, and Siddiqi Reference Cilliers, Dube and Siddiqi2015). Symposium contributors expand this important conversation by considering diverse types of researchers (Kim et al. this issue) and implementing partner identities (Haas et al. this issue).
Identity and Positionality across Many Actors
Because field experiments require intervention(s), this methodology involves interfacing with a variety of actors in ways that are distinct from other methodologies. A triumvirate of partners often exists: (1) donor(s), (2) implementing partner(s), and (3) research-team members. Crucially, we make an important point: as with researcher identity, the identity and positionality of these partners vis-à-vis participant populations can have a significant influence on the research process, outcome, and ethics. Additionally, however, these partners’ identity and positionality vis-à-vis one another also may affect research process and outcomes. We focus on the latter point here.
Donors who fund interventions and research activities often are from the Global North (i.e., private, governmental, and international), particularly agenda setting. First, by controlling the funds, donors steer normative agendas and shape implementing partners’ incentives. Similarly, to finance research, scholars need to find overlap with donor agendas. Whereas implementing partners and scholars have agency to pitch ideas, donors have the ultimate veto power over which interventions are funded. Second, research results that do not align with a donor’s agenda often can be downplayed or censored in reporting.
As discussed by Haas et al. in this symposium, implementing partners can include a diverse range of actors from governmental agencies to local nongovernmental organizations (NGOs) to international NGOs that subcontract to local chapters to religious entities. Collaborations with these partners typically is beneficial for the normative goals of field experiments because they are deeply engaged in policy making for a much longer time than researchers. Not only do they provide expertise in intervention implementation; they also provide longevity for interventions (e.g., often scaling up where successful or eliminating unsuccessful policy). However, these organizations have their own incentives, interests, and ethical standards that may conflict with the researcher’s. Researchers must understand these incentives as well as the historical relationships that organizations have with donors when conducting field experiments. Furthermore, researchers must avoid the (unfortunately) common problem in which academics treat implementing partners as inferior or subordinate rather than as peers with different types of expertise.
Commonly, a field experimental research team might be composed of principal investigators, local supervisors, local research assistants, and sometimes research infrastructure (e.g., Innovations for Poverty Action and the Abdul Latif Jameel Poverty Action Lab). Identity and positionality concerns emerge within the research team among various actors. Field experiments are time and energy intensive, and subordinate research team members easily could be overworked—to not only their own detriment but also the detriment of implementation efforts. Conversely, Kim et al. (this issue) underscore that researchers holding atypical identities often receive less automatic credibility and authority, leading research assistants to sometimes defy instructions.
Clearly, given the multitude of actors involved in field experiments, researchers should consider issues of identity and positionality bilaterally among these actors, as well as in group settings involving all actors. For example, when researchers and donors share a normative agenda for change, implementing partners may be unduly pressured to accept the normative agenda, resulting in a poorly implemented or unethical intervention.
A major theme discussed by many symposium contributors concerns the issue of researcher “insider/outsider” status vis-à-vis other partners. We believe that research is enhanced when researchers from diverse backgrounds—both insiders and outsiders—study research topics. However, researchers can fall victim to implicit bias held by potential partners for certain field experiments.
For example, Kim et al. (this issue) relate how researchers with typical academic identities (e.g., white and male) rarely are questioned about whether or why they can or should study a particular intervention in a particular context. They (often implicitly) are seen by potential partners as legitimate, credible, and authoritative researchers. By contrast, researchers who hold atypical identities (e.g., persons of color, women, and foreign nationals) often have more difficulty in establishing partnerships (e.g., a person of Asian descent studying an African country). However, the reverse might be true when researchers with such atypical identities are studying certain contexts where they are perceived as insiders and/or where those with typical academic identities are viewed as threatening.
Another example concerns insider/outsider identity and positionality when conducting field experiments to reduce prejudice against historically marginalized groups, which frequently is pursued in field experiments. Harrison and Michelson (this issue), who focus on LGBTQ+ prejudice-reduction field experiments, relate that researchers from historically privileged groups (here, heterosexual cisgender individuals) often are suspected of lacking credibility to engage in prejudice-reduction interventions toward marginalized groups (although, oddly, most of this research involves affecting change among majority groups). Alternatively, researchers from the targeted historically marginalized group (here, LGBTQ+ individuals) are suspected of being mere activists that might engage in unsound methodology to find positive results.
THINKING REFLEXIVELY
Identity and positionality affect the scientific rigor and ethics of field experimental research from start to finish. We advocate for scholars who are undertaking field experiments to engage in a continuous reflexive practice about how their identity and positionality—as well as the identity and positionality of other actors involved—affects scientific rigor and ethics. A continually reflexive process means that researchers should frequently examine how their own beliefs, judgments, and practices—often emanating from their identity—influence research (Yanow and Schwartz-Shea Reference Yanow and Schwartz-Shea2015).Footnote 4
We advocate for scholars who are undertaking field experiments to engage in a continuous reflexive practice about how their identity and positionality—as well as the identity and positionality of other actors involved—affects scientific rigor and ethics.
In organizing this symposium, our goal is to capture a diverse set of field experimenters: contributors come from various backgrounds, range from graduate students to associate professors, and include practitioners as well as academics. Davis, a black woman, and Michelitch, a white woman, are both based at Western institutions and conduct field experiments in sub-Saharan Africa. Our own experiences informed our desire to organize this symposium because we believe identity and positionality have an integral role in our work. This symposium includes the following contributions:
-
• Corduneanu-Huci et al. empirically analyze patterns of identity and positionality: who undertakes field experiments, where they take place, and what topics they address.
-
• Haas et al. discuss identity and positionality considerations when working with a variety of different implementing partners.
-
• Kim et al. consider the role of insider/outsider identity and positionality as researchers of atypical scholarly identities.
-
• Harrison and Michelson describe the role of researcher gender and sexual identity in conducting anti-prejudice interventions regarding the LGBTQ+ community.
-
• Hartman et al. examine how a researcher’s gender identity and positionality shape gender scholarship.
-
• Herman et al. describe diverse experiences of identity and positionality when they were working in the Global South as researchers from diverse places.
CONCLUSION
This symposium is a starting point for written dialogue on how identity and positionality affect scientific rigor and ethics in field-experimental research that evaluates policy interventions. The symposium also is important for researchers using other methods to understand that these issues are considered carefully among field experimenters. Indeed, organizing this symposium apparently has been “waiting to happen” based on the sheer numbers of contributors who responded to a public call for expressions of interest in participating. Notably, they span diverse subfields, regional settings, and researcher identities.
Turning to practical considerations moving forward, we direct readers to resources in the online appendix: (1) a checklist of questions about identity and positionality; (2) a list of homework and discussion questions for classroom instruction; and (3) an extended bibliography.
Finally, we recognize that every researcher and research project is unique and that one symposium cannot provide definitive answers for all types of researchers and projects. Rather, this symposium’s contributors shed light on ways in which scholars can think through issues of identity and positionality drawing from their personal experiences. We hope that scholars will build on this symposium to discuss additional identity and positionality concerns moving forward.
Acknowledgments
Funding from the French Agence Nationale de la Recherche (under the Investissement d’Avenir programme, ANR-17-EURE-0010) is gratefully acknowledged.
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/S1049096522000671.
