Field experiments, also known as randomized controlled trials (RCTs), have emerged as a leading methodological tool to strengthen causal inference in the social sciences (Banerjee and Duflo Reference Banerjee and Duflo2009). Designed to make causal claims on research questions of interest to scientists, policy makers, and the public, RCTs often require significant resources, substantive interventions in participants’ lives, and partnership with nonacademic implementers (Davis and Michelitch Reference Davis and Michelitch2022; Haas et al. Reference Haas, Haenschen, Kumar, Panagopoulos, Peyton, Ravanilla and Sierra-Arévalo2022; Teele Reference Teele and Teele2014). Consequently, RCTs carry significant benefits but also costs and risks for participants, research staff, and researchers themselves (Kaplan, Kuhnt, and Steinert Reference Kaplan, Kuhnt and Steinert2020). Although all political science studies require ethical and cost–benefit evaluations, RCTs invoke special considerations of identity, positionality, and power dynamics (Haas et al. Reference Haas, Haenschen, Kumar, Panagopoulos, Peyton, Ravanilla and Sierra-Arévalo2022; Kim et al. Reference Kim, Badrinathan, Choi and Zhou2022), especially for studies conducted in the Global South. The resource-intensive nature of RCTs means that they often are conceived of and led by scholars from the Global North (Corduneanu-Huci, Dorsch, and Maarek Reference Corduneanu-Huci, Dorsch and Maarek2022; Panin Reference Panin2020). This has implications for question choice (Thachil and Vaishnav Reference Thachil and Vaishnav2018), site selection (Porteous Reference Porteous2020), and relationships with Global South research partners (Bleck, Dendere, and Sangaré Reference Bleck, Dendere and Sangaré2018; Fujii Reference Fujii2012; Mwambari Reference Mwambari2019).
We do not take for granted the Global North concentration of human capital leading RCTs.Footnote 1 This article draws on our interdisciplinary experiences to develop a set of questions for RCT research in the Global South,Footnote 2 suggesting ways to involve scholars and research staff who hail from the study site in every research stage. We maintain that these interactions should not be one-off exchanges but rather opportunities to foster meaningful collaboration. We view such efforts as complementary to institutional efforts to recruit and retain graduate students and junior faculty from the Global South. This article is organized in four distinct yet interrelated research stages: idea generation, planning, implementation, and dissemination.
IDEA GENERATION
Ideas for RCTs emerge from various contexts, sometimes removed from the communities being studied. Most proposed experiments are concentrated not only within five Global North countries but also within five academic institutions (Corduneanu-Huci, Dorsch, and Maarek Reference Corduneanu-Huci, Dorsch and Maarek2022). Regardless of the source of an idea, prioritizing early and ongoing inclusion of community voices is the first imperative for a study (Davis Reference Davis2020).Footnote 3 It is challenging to determine who counts as a relevant stakeholder; addressing questions of positionality, belonging, and “insider/outsider” status can be as complex and frustrating as it is essential (Kim et al. Reference Kim, Badrinathan, Choi and Zhou2022).Footnote 4 Nevertheless, all researchers who are planning to work in the Global South—whether 5,000 miles or five blocks away from home—should ask themselves tough questions before proceeding with any RCT (Cowen Reference Cowen2019). Is the question relevant to the communities where research will take place? Can I answer this question without intervening in human lives? Are my career goals and research priorities leading me to overlook ethical or moral concerns? Answers to these questions may clarify whether the benefits of a study outweigh the costs.
Community collaboration during idea generation cultivates relevant research and shared interest in preserving long-term partnerships, and it indicates respect for stakeholders. Occasionally, prioritizing ideas and research goals of in-country partners can have tradeoffs in later stages of research. However, the possibility of generating useful insights for host communities—while still providing theoretically important, internally valid academic contributions—should be viewed as central in modern social science research (Bleck, Dendere, and Sangaré Reference Bleck, Dendere and Sangaré2018).
Including community input is especially crucial when questions are generated externally. Such input can come from potential participants, activists, researchers, academics, policy makers, and others and can orient researchers toward salient issues and challenges (Asiamah, Awal, and MacLean Reference Asiamah, Awal and MacLean2021; Thachil and Vaishnav Reference Thachil and Vaishnav2018). Over time, investments in community relationships allow for frank pushback, responsiveness, and active collaboration with community partners, which can improve design, measurement, conceptualization, and buy-in.Footnote 5
Businesses, governments, international agencies, and civil-society organizations often choose RCTs for program evaluation, viewing them as the “gold standard.” They may evaluate an existing program or a novel intervention designed by the research and implementation teams. In either case, developing implementation partnerships requires evaluating the compatibility of objectives, benefits, risks, and ethical concerns (Levine Reference Levine, Druckman and Green2021). Researchers should assess the demand for evidence related to the proposed intervention, considering how their own inclusion could alter the program’s design, the data gathered, and the overall goals of the project. Researchers should scrutinize implementing partners’ track records and cease those partnerships if they find abuses.
PLANNING
Planning an RCT in the Global South requires thoughtful assessment of the intervention and research protocols. If costs to the participants and study communities outweigh potential benefits, researchers should not proceed with the project. Many experiments have distributional effects, reallocating benefits from one group of people to another. As a result, the benefits and harms of an experiment between groups often are in conflict.Footnote 6 Resolving these tensions requires making value judgments, such as prioritizing the interests of citizens over politicians. We suggest that researchers consult and engage with a wide array of stakeholders when making these choices and reporting on the values they discover, as well as their own, in their study findings.
Researchers can consider relevant stakeholders as persons affected both directly and indirectly by the study. The identity of those directly affected depends on research design, including sampling, treatment assignment, and measurement. This additionally includes those responsible for administering treatments and collecting data. It also is important to consider individuals who are impacted indirectly due to their links to those directly affected, including social, geographic, political, and economic networks. This includes people potentially affected by the release of study results. Speaking with potential research participants in pre-experiment surveys and focus groups, as well as with local experts, may be necessary to identify the set of affected groups and the benefits and risks to each.
Ex-post cost–benefit analyses of interventions are increasingly common (Brown and Tanner Reference Brown and Tanner2019). These tools could be adapted to provide informed ex-ante approximations of costs and benefits across different experimental designs, thereby guiding decisions of whether to undertake the study. Costs should include not only those related to the intervention (i.e., treatment) but also to conducting the experiment, including participants and others indirectly affected. Benefits should include welfare changes caused directly by the intervention and experiment, as well as subsequent outcomes, including policy changes if programs are demonstrated to be effective and are adopted more widely. Values ascribed to the status quo can be elicited from community stakeholders before research begins to identify how best to maximize benefits, minimize harms, and build in risk-reduction strategies.
In these cost–benefit analyses, multiple value dimensions and to whom they accrue should be considered. Scientific value can be assessed by potential learning relative to past studies; where publication bias is not an issue, this can be accomplished using meta-analysis. Simulation-based power analyses can quantify expected learning (Blair, Coppock, and Humphreys Reference Blair, Coppock and Humphreys2021).
Considering what and how to randomize—as well as what cannot and should not be randomized—also is central to planning an RCT. Many people in highly studied communities understand aspects of experimental design. They may view intervention as unfair or adopt a negative attitude if they perceive themselves as excluded from a study and its benefits (Karim Reference Karim2020). Alternatively, randomization can be perceived as a fairer allocation mechanism than other methods. Although such circumstances are relatively rare, when interventions are known to have positive impacts validated across a range of contexts, randomization by researchers constitutes withholding known benefits and thus is unethical.
The design of research instruments also should incorporate stakeholder feedback to minimize harm, meet scientific goals, and maximize learning (Thachil Reference Thachil2018). Supplementing surveys with qualitative data, including interviews, program facilitator notes, participant reflections, and psychophysiological data, facilitates validation and depth.
Hiring and collaborating with local researchers and project managers advances ground-up planning, including monitoring of research integrity and risks. Staff training should be expansive and responsive, with a view toward developing sustainable skills for community partners and researchers. Training modules can be determined in partnership with enumerators and program staff to determine what is valuable in local labor markets. Researchers should create an open environment in which all team members can share ideas and oppose designs that threaten scientific, practical, and ethical integrity.
IMPLEMENTATION
Things can go wrong in the field, presenting both risks and opportunities. Researchers may have to halt or pivot studies at any point in implementation; therefore, active monitoring is essential. The decision to halt or pivot should consider that by that point, implementing partners already may be deeply involved, participants’ lives may have been altered, and lessons may have been generated to disseminate. Researchers can improve the quality and speed of identifying problems or unexpected opportunities by centering community voices throughout the project. Consulting with participants—with the understanding that power imbalances may affect their willingness to disclose harm—can help researchers identify whether red lines have been crossed.
Addressing ethical implications and positionality is a dynamic process throughout the project life cycle, especially as contextual or circumstantial shifts change implementation plans. For example, violence may make it unsafe to collect data, new leadership of a partner organization may cancel a treatment arm, or a global pandemic multiplies stay-at-home orders for involved stakeholders. Internal and external factors such as these often can threaten completion of an experiment. In such cases, what should researchers do?
First, researchers should not simply withdraw. Foreign researchers often have greater travel flexibility, including diplomatic support for evacuations and financial resources. However, they also have responsibilities to implementation partners, research staff, and RCT participants. Even if they halt a study, researchers should continue to provide promised support, responsibly transition ongoing duties,Footnote 7 and share insights generated thus far. Second, researchers should consider how to leverage resources to support research staff. Recommendations and connections to facilitate future jobs, creating mechanisms to transfer outstanding funds safely and securely, and establishing a network for all partners can preserve benefits. Third, researchers should determine how they can engage and support ongoing concerns even after their withdrawal. Fourth, sharing what can be learned from a project that “fails” is imperative. This can be accomplished by sharing protocols, survey instruments, and/or a short report on the study failure to provide future recommendations. Silent failures beget repetition.Footnote 8
Throughout the project, research teams should reflect on positionality and center participant well-being, including mental health. This can be accomplished by conducting additional analyses to understand context-specific factors among vulnerable communities beyond Institutional Review Board standards (Cronin-Furman and Lake Reference Cronin-Furman and Lake2018) and ensuring that relevant care is accessible to participants throughout the project, with resources such as referral cards and connections to reliable healthcare providers. Local partners can highlight potential problems of accessing care, including stigma and logistical barriers (Khedari et al. Reference Khedari, Niconchuk, Beranbaum, Herman, Al-Kareem, Jaber and D’Andrea2021). Care for the physical and mental health of research staff also should be prioritized during planning and budgeting by accommodating data-collection schedules, conducting regular debriefings, and providing mental health resources.
Care for the physical and mental health of research staff also should be prioritized during planning and budgeting by accommodating data-collection schedules, conducting regular debriefings, and providing mental health resources.
SCIENTIFIC COMMUNICATION
A significant justification for RCTs is that they generate credible causal evidence that can inform policy. Therefore, the ability to clearly communicate research findings to policy makers and affected communities is central to the ethical and empirical frameworks guiding RCT design and implementation. Effective scientific communication should target key stakeholders—including participants, communities directly and indirectly affected, policy makers, implementation partners, and in-country academics—with tailored output including memos, pamphlets, and short videos publicized via local partners and media (Thachil and Vaishnav Reference Thachil and Vaishnav2018). Findings should be presented throughout the research cycle using context-specific language and with comprehensible research transparency. Research output relevant to study communities may differ from academic output; reporting on secondary analyses may be as useful as the experimental results (Herman Reference Herman2021). Funding and capacity for communication efforts should be incorporated into budgeting and hiring considerations from the outset. Creating compensated advisory committees, staffed by community stakeholders, can facilitate stakeholder feedback on input and output at regular intervals and support dissemination throughout the research cycle.
A significant justification for RCTs is that they generate credible causal evidence that can inform policy. Therefore, the ability to clearly communicate research findings to policy makers and affected communities is central to the ethical and empirical frameworks guiding RCT design and implementation.
CONCLUSION
Assessing risks, localizing benefits, and addressing positionality requires an explicit understanding of what not to do. Research questions that do not reflect priorities of community stakeholders, designs that fail to align with site-specific contexts, and an over-reliance on implementation partners and prioritization of scientific validity (Bedecarrats, Guerin, and Roubaud Reference Bedecarrats, Guerin and Roubaud2019) may be potential barriers to effective and ethical RCTs. Time horizons between researchers eager to register output may conflict with policy makers who want longitudinal insights. Journal reviewers’ priorities may misalign with policy applicability. Reports that use disciplinary jargon may dissuade nonacademic audiences from engaging. Centering community contexts, stakeholders, and demands at each research stage is key to ensuring that RCTs in the Global South are ethically sound and generating insights that serve the populations being investigated.
For these changes to take hold within academia, it is incumbent on journal reviewers, tenure committees, and other institutions of academic power to institutionalize incentives that center collaborative processes. Reporting guidelines that document who was consulted at each research stage should be instituted for papers, including discussion of ex-ante cost–benefit analyses and steps taken to identify and mitigate risks. Institutional change is vital because the efforts of individual researchers alone cannot produce changes to entrenched norms.
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/S1049096522000063.
