The use of experimental methods to study questions of gender and politics has increased dramatically in the past decade.Footnote 1 As research that uses a deductive hypothesis-testing framework, these studies derive value from the idea of their objectivity and neutrality. Yet, reality is far more complex. This article argues that researchers conducting field experimental work on gender topics should confront their own normative commitments to produce empirically rigorous research of the highest standard with empathy and integrity. We explore how and why researchers’ gender politics matter and offer a way forward to reflect on how one’s own values shape a research project. We contend that recognizing and engaging with one’s own normative commitments and politics create the conditions for the most rigorous work.
…researchers conducting field experimental work on gender topics should confront their own normative commitments to produce empirically rigorous research of the highest standard with empathy and integrity.
The use of experimental methods to study questions of gender and politics includes field experiments studying the effects of interventions that aim to shift attitudes, preferences, and behaviors around gender.Footnote 2 For the most part, this research presents itself as an objective and neutral undertaking, using a deductive hypothesis-testing framework. Yet, this rarely is the case.
A rich body of scholarship establishes how researcher positionality matters for field research, including research on gender (e.g., England Reference England1994; Loftsdottir Reference Loftsdottir2002) and work that explores the positionality of researchers “studying their own societies.”Footnote 3 However, most of this scholarship focuses on aspects of positionality other than researcher politics and is most prevalent in qualitative and interpretivist methodological traditions. Less research exists on how and why to consider politics and positionality in experimental work, where part of the methodological value is derived from its perceived objectivity.Footnote 4 This article highlights the importance of researchers grappling with their own politics and normative commitments. Although we believe that this is important for all field research and experimental work, this article focuses on how it applies to field experiments on topics of gender.
We follow Htun (Reference Htun2005) in conceptualizing gender as a “social position and attribute of social structures,”Footnote 5 and we define researchers’ “gender politics” as the normative views and political commitments that they hold about gender. In the introduction to this symposium, Davis and Michelitch define positionality as “researcher social location, perspective, orientation, and situatedness vis-à-vis participants” (Fujii Reference Fujii2017; Soedirgo and Glas Reference Soedirgo and Glas2020). We posit that gender politics, as defined previously, is a core part of researcher positionality.
How does this aspect of positionality matter for field experiments on gender? Whereas all research arguably involves a degree of intervention into the subjects of inquiry, field experiments are unique in that they often are designed explicitly to study interventions seeking to change attitudes, preferences, individual behaviors, or features of institutional design and social structures. We argue that researchers’ politics and ideological commitments take on heightened salience when intervention is at the core of the research method. Coupled with the fact that gender-related interventions often seek to redress inequalities that are upheld and reproduced in intimate sites of home and family, this may render them especially fraught and worthy of attention (Burns Reference Burns2005).
The article begins by establishing that political commitments underlie field experimental research on gender; we then demonstrate how these commitments shape various aspects of the research process. We draw on our collective experiences of conducting field experiments on topics of women’s political participation; sexual violence; intra-household relationships; and women’s representation in policing in varied contexts including Pakistan, Liberia, Democratic Republic of the Congo, Côte d’Ivoire, and Myanmar. Finally, we argue that actively recognizing and reflecting on the gender politics implicit in our work can result in more rigorous and ethical experimental scholarship on gender. We conclude with suggestions for how to do so.
THE PERSONAL POLITICS THAT UNDERPIN POLITICAL SCIENCE RESEARCH
A brief review of our own recent experimental work on gender makes it clear that gender politics shapes our experimental research. Cheema et al. (Reference Cheema, Khan, Liaqat and Mohmand2021) assessed the efficacy of different approaches to increase women voters’ turnout in national elections. Karim (Reference Karim2020) and Karim et al. (Reference Karim, Gilligan, Blair and Beardsley2018) studied how externally imposed quotas for women police have the potential to improve police-unit functioning and perceptions of the police. Donald et al. (Reference Donald, Goldstein, Hartman, La Ferrara and O’Sullivan2020) examined how formalizing marriages might improve women’s economic position within the household and reduce intimate-partner violence. Pierotti, Lake, and Lewis (Reference Pierotti, Lake and Lewis2018) investigated how positive and gender-accountable masculinities can reduce unequal labor burdens in the household, reduce violence, and improve intimate partnerships.
Motivating researchers in each of these projects is a set of implicit normative beliefs about gender arrangements in society. These include that women’s equality with men in different arenas is beneficial and should be advocated; violence against women is harmful for individuals, families, and societies and should be prevented; organizations benefit from having multiple different perspectives and experiences; and men and women should share power equally and have an equal stake in decision making. However, following existing norms in political science and economics, we rarely make these beliefs or commitments explicit in our work.
Although scholars who research these topics share political commitments, it is worth recognizing that these ideas are not static and may be shaped by aspects of identity and lived experiences specific to individual researchers. For instance, Cheema found that his political experience as a student during a time of draconian anti-women legal reforms in 1980s Pakistan influenced his subsequent research on women’s political participation.Footnote 6 Hartman’s personal experience of marriage led her to pivot her research agenda to explore the differential experience of women and other groups frequently excluded from local power structures.Footnote 7
Explicitly recognizing our own gender politics also allows space for anticipating the implications of situations when normative commitments are not shared in the course of research—that is, between different researchers or between researchers and other key actors, including research assistants, implementation partners, enumerator teams, and research subjects. The complex relationships between our normative commitments and our research present a series of ethical and political quandaries that can unfold throughout the research process, and they should be discussed and engaged more transparently in the discipline.
HOW GENDER POLITICS SHAPES RESEARCH AND WHY IT MATTERS
Reflection on our own work reveals how gender politics has shaped the focus and process of research. Politics can push our inquiry in new directions and also create blind spots. During the research process, politics shapes almost all decisions around who can participate, what is “innovative,” and what we value in our research.
The Questions We Ask
We find that gender politics often leads us to pursue research questions aligned with our own experiences and beliefs. For example, while designing an intervention to increase women’s political participation in Pakistan, Cheema et al. (Reference Cheema, Khan, Liaqat and Mohmand2021) chose to focus on interventions that could address “everyday forms of male gatekeeping” in the urban center where they live rather than the extreme conservative “bans” on women’s voting in rural and tribal areas that have received much media and legislative attention. The decision to focus on these more quotidian constraints came at least in part from a lived familiarity with personally navigating them and knowing the real costs that they impose on women’s presence in public life. Such choices involve crucial tradeoffs: findings from field experiments conducted in a specific locality where context matters may have limited external validity (Pritchett and Sandefeur Reference Pritchett and Sandefur2015).
The identities of academic researchers and their positionality also can create blind spots that prioritize questions that ignore intersectional perspectives. Some approaches by experimentalists often have focused on an “add-women-and-stir” approach to change institutional gendered culture. These interventions, which randomize women’s inclusion in politics (e.g., Beath, Christia, and Enikolopov Reference Beath, Christia and Enikolopov2013), security (e.g., Karim Reference Karim2020), and economics (e.g., Hoogendoorn, Oosterbeek, and Van Praag Reference Hoogendoorn, Oosterbeek and Van Praag2013), do not interrogate what those interventions might mean for differently situated women in target communities, and neither do they decouple gender from women. In doing so, they inadvertently make assumptions about how the category of women is apolitical.
Our own situated gender politics as researchers also profoundly shapes which questions we consider legitimate and valuable. Many scholars have justified interventions and inquiries into the most intimate aspects of research participants’ private lives in the interest of changing gender norms (Pierotti, Lake, and Lewis Reference Pierotti, Lake and Lewis2018). Research questions that interrogate behavior relating to intimate-partner violence, sexual consent, and marital decision making is often legitimated by an implicit normative assumption that changed behavior is a societal good. Yet, interventions that seek to change violent or coercive gender behaviors often involve advocating for far-reaching sociocultural change. For example, Pierotti, Lake, and Lewis (Reference Pierotti, Lake and Lewis2018) studied an intervention by the International Rescue Committee designed to reduce intimate-partner violence by promoting positive masculinities through men’s discussion groups. Researchers can inadvertently find themselves legitimating projects that seek social and cultural transformation without being transparent about the values that underpin those projects. Furthermore, the subsequent consequences of these changes often are not included in research designs. Even if interventions to enhance women’s inclusion work in the short term, they might lead to future unintended consequences that experimentalists may not consider when designing the experiment, such as intimate-partner violence among women who become more politically active (Berry Reference Berry2018).
Gender politics affects who is eligible to participate in research. Historically, gendered inequality meant that most experimental research implicitly or explicitly focused on men; if impacts on women were considered, studies often sought men to speak on their behalf.Footnote 8 Women engage in more care work and often supplement household income (i.e., taking on multiple jobs), which can make them unavailable or inaccessible to researchers (Kumar et al. Reference Kumar, Nguyen, Harris, Harvey, Rawat and Ruel2018). Similarly, in some contexts, women may not be permitted to speak openly to enumerators without their husband present. Although norms that determine which topics are centrally relevant for political science have shifted, challenges remain. Research questions concerning women who have weaker ties to men (e.g., unmarried women and older women who may be perceived as unimportant for phenomena most central to much political and economic research) or those pertaining to women who are not members of the dominant social group typically continue to be excluded from mainstream political science scholarship.
The Ethics of Intervening
Gender politics also shapes how we understand ethical research. Field experiments raise particularly critical ethical questions, wherein active intervention into the lives of study participants often is implicitly justified by the perceived social benefits of the research.Footnote 9 Indeed, researchers who conduct field experiments on gender sometimes identify a commitment to transformative change. Yet, most interventions neither seek nor have the capacity to overhaul social and political structures or uproot interlocking hierarchies of oppression (Clayton and Anderson-Nilsson Reference Clayton, Anderson-Nilsson, Green and Druckman2021). It is essential, therefore, to engage thoughtfully with what is desirable and what is possible in the realm of what often are fairly short-term interventions. This links to thinking about appropriate outcome measures. Should we expect an intervention to lead to wide-sweeping social change or large-scale changes in attitudes? Perhaps not (Beath, Christia, and Enikolopov Reference Beath, Christia and Enikolopov2013). In developing outcome measures that are perhaps unreasonable, we risk setting up the experiment to fail. This means that experiments find null results for interventions that might have important effects but are discounted because they do not create the large-scale change that policy makers want to see.
Furthermore, interventions that lead to social transformation may have subsequent negative consequences. When intervening to change gender dynamics, researchers have an ethical obligation to consider carefully those interventions that could result in backlash for participants. It is essential to avoid harm; yet, backlash may be indicative of real change in patriarchal hierarchies. In our personal lived politics, it is precisely these “transgressive” actions that we may promote to transform existing power structures. However, we often are ethically obliged to do the opposite in our research: that is, negotiate our work within the status quo and limit risks to participants at the cost of transformational change. It is notable, however, that—like us—participants in research who seek to transform gender politics have their own political orientation, values, and preferences.Footnote 10 They may take risks of their own volition. As researchers, we must balance our own politics, the participants’ politics, and the risks of transgressive or transformative interventions.
Decisions Along the Way
Gender politics also shapes numerous decisions that researchers make throughout the research process. All stages reflect gender politics: the determination of who is a legitimate researcher; who should assist in the research process; who is eligible to participate in the research and why; which organizations and stakeholders can collaborate in the research as partners; what ethical considerations should be made; and who can benefit from the research.
These decisions within the research process sometimes are as important as the outcome. For example, when conducting qualitative fieldwork on the demand for formalizing marriages in Côte d’Ivoire, Hartman found that different political views held by members of the research team shaped opinion about which interventions were possible. The debate around this issue was critical to designing an intervention that sought radical change and that also was legitimate within the research context (Donald et al. Reference Donald, Goldstein, Hartman, La Ferrara and O’Sullivan2020). In one study, Karim (Reference Karim2020) worked with the police in Zambia and engaged in a validation workshop to present the results. The research proved to be a catalyst for important topics related to gender equality, including sexual harassment, paternity leave, and hazing. The very act of dialogue about gender equality among police officers proved to be transformative, creating buy-in for the study results. Similarly, during a debriefing session in Cheema et al.’s (Reference Cheema, Khan, Liaqat and Mohmand2021) study on women voter turnout in Pakistan, women canvassers noted their experience of traveling to parts of the city that they had never been to as part of implementing the canvassing treatment. Thus, working in the study expanded, if only briefly, their sphere of mobility in an urban public space that is costly for women to traverse.
As the discussion of ethics demonstrates, who becomes a researcher and who can engage in a research partnership matters because these relationships can bring resources, visibility, and strength to structures that continue to work for change after the research is completed. Partnering to conduct experimental research can be extremely costly, and it requires specific capacity, resources, and connections. All of these factors shape how researchers understand which organizations conduct good work. To work with partners who are women or who have women leaders is a political decision that requires certain material investments but that also can reap critical rewards. Cheema et al. (Reference Cheema, Khan, Liaqat and Mohmand2021) chose to work closely with organizations that resulted from the 1980s women’s movement and that are extremely active in mobilizing communities on women’s rights issues in Pakistan. Similar investments may be necessary to return research to those communities that participate in it if they include women. Lower levels of literacy or fluency in multiple languages and the relative exclusion of women from parts of the public sphere mean that inclusive research requires a thoughtful commitment of resources. Dissemination requires that same careful engagement. Given our politics, we hope that our projects inform policies in ways that align with our politics. Given that experiments often are understood to be the “gold standard” for policy research and may be treated more credibly or disseminated more widely than other forms of research, experimentalists arguably have even more responsibility to engage their own politics as the research process unfolds.
BALANCING OUR GENDERED POLITICS WITH RIGOROUS EXPERIMENTAL RESEARCH
We contend that gendered politics matter for every phase of experimental research. Many of us are drawn to our research on gender through feminist political commitments to gender equality. Engagement with our politics allows us to identify crucial gaps in our own research and in broader research agendas. Field experiments are resource intensive and may be driven by what is possible rather than what is required. Honest engagement with the normative political commitments that underpin our work can keep us aware of these tradeoffs. It also can serve to make us humble about the generalizability of our claims. Reflecting on our own positionality—how it has informed the baseline assumptions we make, afforded us access to particular research topics and spaces, and shaped our understanding of what is ethical—enables more awareness of our own biases and blind spots and, critically, helps us to avoid unintended harm. In the interests of ethics, empirical rigor, and researcher integrity, we call for researchers to grapple with normative politics in their work, and we advocate for a shift in research culture that values this form of researcher reflexivity.
Nevertheless, introducing such an explicit engagement with personal values into experimental research frameworks can be challenging. How can researchers concretely integrate considerations of gender politics into their work? The introduction to this symposium provides several helpful suggestions. We propose the following additional questions specific to gender politics for scholars who are seeking to conduct field experiments on gender:
-
• What are my personal politics about the aspect of gender under study in this project?
-
• Is the intervention that I am studying in this project aligned or at odds with these politics?
-
• How does my politics shape my priors, expectations, and investment in the efficacy of this intervention?
-
• Are my politics aligned or at odds with those of other actors in the project: other principal investigators, implementations partners, research staff (e.g., enumerators and research assistants), and project participants? How might this facilitate or create tension in the research process?
-
• How might my politics shape the way I interpret and frame the findings of this research in the reporting phase?
-
• How does politics shape how I am thinking about the intervention and outcomes?
-
• How might my politics shape the way I disseminate the findings of my research to different audiences?
When and where might such reflection occur? In our experience, we have engaged these questions only in informal conversation with our peers, friends, and family during or after conducting a project. We encourage scholars to consider creating space for these discussions before implementation of their research—for example, as part of methods training in the classroom, during research-design workshops for experimental work, and in reflexivity statements in pre-analysis plans. Finally, we believe that these discussions can yield the greatest reward when undertaken in diverse teams in which multiple perspectives are represented. Field experiments often are a collaborative enterprise. When we embrace the diversity and complexity of gender politics across time and space, experimental research on the politics of gender may provide a unique opportunity for learning about ourselves and the world.
When we embrace the diversity and complexity of gender politics across time and space, experimental research on the politics of gender may provide a unique opportunity for learning about ourselves and the world.
