The exploding popularity of randomized field experiments in political science, the 2014 Montana mailer controversy, the methodological ignorance among some Institutional Review Boards (IRBs), and the paucity of case law involving scholars who have successfully sued in response to unfair IRB practices create a Gordian Knot for scholars. Before considering how to approach this challenge, I first address the caveats. IRBs are necessary and their work is important. Many of those serving on IRBs are good-intentioned in making what often are difficult decisions at the intersection of federal guidelines and idiosyncratic research designs across an array of disciplines, while working within a system that they did not create. Moreover, regardless of how frustrating an experience with an IRB is, scholars should not undertake a research project without IRB approval (even if the project does not involve federal funding). But, good intentions notwithstanding, any attempt by IRBs to place wholesale or capricious restrictions on a research methodology must be resisted and labeled as what they are: a frontal assault on academic freedom. It is time for scholars to be more assertive in challenging IRB decisions that threaten both their academic freedom and their career trajectories.
The lingering problem—and why we again are engaged in another IRB symposium in PS—is that there is no national or uniform set of IRB practices across institutions. Instead, there is what Yanow and Schwartz-Shea (Reference Yanow and Schwartz-Shea2008, 485) call “the classic implementation situation of a legislative patchwork of policy decisions.” In other words, rather than function like the FBI, individual university IRBs are free to behave like county sheriffs—with no rhyme or reason in the assessment decisions made between boards. In addition to other unsavory outcomes, this patchwork implementation creates an unequal research playing field for scholars, particularly those who cannot get their proposals approved for reasons that reflect bias or ignorance as it pertains to certain research methodologies. Clearly, some IRB denials are justified, but the very existence of this symposium suggests that scholars are becoming more aware that not all IRB decisions are steeped in transparent, empirically-grounded, and equitable decision criteria—especially when new or controversial research methodologies are concerned.
Field experiments are perhaps the most controversial of political science research methods, and my intention in this article is to encourage those at institutions where IRBs have proven hostile to the methodology. Although the Montana mailer controversy has made life difficult for scholars using field-based interventions, fallout may not be evenly distributed across institutions when sorted by type (e.g., R1s versus more teaching-oriented institutions). Although I am unaware of any available statistics on this issue, it is reasonable to assume that those at institutions where IRBs lack experience with field-experiment methodology (and, presumably, refuse to be educated) may be more likely to face a capricious denial of their research proposal. Given their increased classroom instruction and service responsibilities, IRB members at teaching institutions—on average—have less time and motivation to be familiar with cutting-edge research techniques versus those at research-intensive schools (exceptions notwithstanding). Unless these IRBs are willing to be educated on the method, a “perfect storm” of IRB ignorance and/or recalcitrance and the desire of newer faculty to make tenure and/or “move up” the institutional ladder means that bias against field experiments is likely to affect scholars at “teaching” institutions (i.e., liberal arts colleges and Master’s-granting institutions) the most. This means field experiments may become an off-limits methodology for a large swath of political scientists whose IRBs are less aware of discipline-specific research trends, although there is no guarantee that IRBs at R1 institutions are always supportive of field-based interventions either. And, it is why simply allowing IRBs to pass unfair judgment on field experiments as if the “I” stands for “infallible” is intolerable.
This idea of pushback against an IRB may be perceived as controversial, but it is steeped in years of academic angst about IRB behavior across institutions. The larger debate about whether university IRB oversight of social science research is consistent with federal IRB guidelines has existed for at least 15 years (Shea Reference Shea2000; Shweder Reference Shweder2006). Complicating matters is the combined unfamiliarity of IRB members with the research methods they are evaluating and the negligence of university administrators who do nothing to address inherent deficiencies within their IRBs. Nelson’s (Reference Nelson2003) essay for the American Association of University Professors made this point well:
. . . the growing literature on campus IRBs shows again and again that boards assembled to supervise biomedical research often haven’t a clue about the culture of history or anthropology or literature departments.
In the meantime, Musoba, Jacob, and Robinson’s (Reference Musoba, Jacob and Robinson2014) concern that IRB “mission creep” into nonmedical research portends academic censorship becomes more pressing by the semester. Therefore, it is incumbent on faculty to take greater ownership of the research approval and oversight processes at their institutions.
Exactly how to push back effectively on IRBs is unclear (although this article offers starting points for consideration). But this lack of clarity does not mean that one should simply accept what may be an unfair IRB limitation of one’s academic freedom. After all, it is not as if one will be given a pass on research productivity expectations as a result: administrators are unlikely to concede that an IRB’s prohibition of a scholar’s field-experiment agenda is related to publication output. Furthermore, “moving up” to a “better” job (i.e., an institution in which research is more highly valued) will likely require publications before landing the new gig. This means that taking an IRB rejection of a field experiment without pushback may not serve one’s professional interests in the short or long runs.
An equally frustrating reality is that IRBs are not always the enemies of good research: they can have a helpful function in refining field-experiment proposals in ways that actually improve the design. As such, scholars should avoid the false dichotomy of total acquiescence or defiance in regard to IRBs (regardless of whether a project is supported by federal funds). However, realizing the potential benefit of an IRB review requires that an IRB actually understands what field experiments are and does not expect that scholars will simply revise their research plans so that they no longer include a field-based intervention.
Now is the time for faculty across institutional types to force a dialogue about IRB oversight within and among institutions and disciplines. In doing so, it will be vital to differentiate between those research designs in which IRB scrutiny is both legally and ethically warranted and those in which institutional and administrative preferences alone have created an IRB disposition toward field experiments that is both unfair and a clear violation of academic freedom. That there is cause for continued comment on IRB behavior suggests that the exigencies the authors of the 2008 PS essays perceived have only increased in the ensuing years.
Now is the time for faculty across institutional types to force a dialogue about IRB oversight within and among institutions and disciplines.
Given the inherent collective-action problem involved in taking on a university IRB (perhaps as the first and only faculty member to do so at one’s institution), faculty pushback against IRB decisions is a relatively new and rare phenomenon. This may be due partly to the observational survey and laboratory-based approaches to data collection that are more amenable to the type of adjustments that IRBs feel compelled to require (i.e., informed consent). Scholars facing IRB rejections may have also decided to not “rock the boat.” After all, it often is possible to find a workaround to IRB rejections of field experiments while not defying the IRB. Yet, what is possible versus what is probable or preferred is not the same. These workarounds (e.g., farming out a research project to colleagues whose IRBs approve the design) also are costly and not always practical—especially when single-author publications are required for tenure, promotion, or a better job. This means that something other than retreat in the face of an IRB rejection is needed.
The case of Dr. Jun Li at Brown University is a prime example of using litigation against IRBs that have acted egregiously. Facing what she considered an undue burden on her academic freedom—not to mention the success of a research project in which she invested substantial time and resources—Li, an associate professor of education, filed a federal lawsuit in 2011 against the Brown University IRB (Rasmussen Reference Rasmussen2011). Among her claims, Li charged the IRB with overstepping its federal mandate in blocking her use of data collected under a compensation plan for subjects that the IRB did not like. Li also claimed that Brown’s IRB lacked due process because there was no adjudication mechanism in place to monitor IRB actions. Importantly, both of these claims may be the norm rather than the exception at US institutions. Although Li’s suit eventually was dismissed “with prejudice” (i.e., she had to pay her own legal fees), the effect on Brown was telling in that university officials appeared to agree to a number of IRB reforms, including establishing a separate approval board for social science research. This is similar to what Mayer describes in this issue as the University of Wisconsin’s IRB setup.
With attention to the Li case, the renewed focus on the role of IRBs in the wake of the Montana mailer controversy, and even the impact of this current PS symposium, the effort to reform university IRBs—including (and perhaps especially) those at “teaching” institutions—may gain traction. Ironically—and although negative press about the Montana project might make IRBs more recalcitrant concerning field experiments—the mailer episode also offers an opportunity to engage IRB members in conversation and education about the inherent advantages of sound, ethical field experiments. Although IRB reform is a matter of both national policy change and local university action, individual researchers—especially those at non-R1 schools—should consider one or more of the following recommendations when dealing with IRB decisions that threaten their research agendas and academic freedom.
EVALUATE THE COST OF INACTION
To paraphrase John F. Kennedy, fearing to negotiate is not a sound strategy. Realistically, the only party that ultimately is hurt by an unfair IRB decision is the researcher. The IRB members will carry on along their habitual path and university administrators will continue to operate under the unsubstantiated assumption that they have staved off a lawsuit by steadfastly supporting IRB denial of a design. If, in the grand scheme, one is fine with abandoning a research project, then it makes strategic sense to let go of the issue. Furthermore, if the field-experiment methodology is not what the IRB is objecting to but rather a specific element of a proposal that would maintain the scientific validity of the randomized field intervention if corrected, then it is in one’s best interest to meet the IRB at least halfway. In other words, always attempt to negotiate with the IRB and address its stated concerns, providing that doing so does not essentially result in abandoning the field experiment.
However, in cases in which a university IRB simply is ignorant, even willfully so, about field experiments—and it expects a scholar to alter a research proposal so that compliance makes the intervention a field experiment no longer—one must consider the cost to career trajectory due to the combination of IRB ignorance and inflexibility and reflexive administrative caution in backing up the IRB decision. Realistically, not all IRBs will deny field experiments outright, but even required major revisions that make the design less powerful or unfit for a general-interest publication are serious ramifications. Scholars should consider what the cost of inaction gets them.
In other words, always attempt to negotiate with the IRB and address its stated concerns, providing that doing so does not essentially result in abandoning the field experiment.
CONSIDER WHAT COLLEGIALITY REALLY GETS YOU
One of the first rules of “making it” in higher education is to be collegial. However, this should not be interpreted as conflict must be avoided at all costs, especially where research agendas and productivity are concerned. In most cases, being collegial will not save a tenure application if research output is lacking, and “moving up” is virtually impossible by merely being a “good department citizen.” Of course, being nice is helpful and getting along with administrators is good practice for obvious reasons. At the same time, letting IRB members know that the frustration is not with them personally also is a good idea. I cannot help but imagine, however, that most new and mid-career scholars will find collegial relationships a less-than-suitable consolation when someone else beats them to publishing what would have been their field-experiment findings in a general-interest journal—if only some dust had been kicked up in a recalcitrant IRB’s direction. Stirring things up in defense of a research agenda may cost some institutional “brownie points,” which is unfortunate, but being less liked is generally a worthy tradeoff in the face of immediately sacrificing research opportunities simply because some colleagues and administrators will consider one a troublemaker. Of course, one may kick up dust and still not convince the IRB to budge, but this outcome is fully assured if no action is taken.
PREPARE FOR PUSHBACK
Coincidentally (of course), university administrators may take increased interest in a faculty’s personnel file once a grievance and/or litigation is pursued due to unfair IRB actions. Furthermore, assuming that they even appreciate the methodology’s importance, it is generally not in administrators’ rational interest to side with a faculty member against the IRB (particularly if the IRB has been instructed to go well beyond the federal regulations in evaluating research proposals). At the same time—and, again, purely as a coincidence—expect varying degrees of harassment from administrators if one starts to complain about “unfair” IRB practices. This is “politics” in its most professional and literal sense.
COUNTER CYA ASSUMPTIONS ABOUT INSTITUTIONAL LIABILITY
It is worth remembering that IRB concern over the “harms” of “deception” in nonmedical research on nonvulnerable populations is well outside of the established federal guidelines that instituted IRBs in the first place. Nevertheless, university leaders are likely in favor of CYA (i.e., cover your administrator) policies, even as the rationale for these actions is likely not based on any assignable probability of a lawsuit being filed over deceptive treatments. A common critique of field experimentation is the unsubstantiated assumption that the lack of informed consent exposes the university to liability. This argument is usually preceded by hand-wringing about the use of deception in field experiments. Whereas it is true that certain types of research deception actually may harm subjects—and should be avoided—deception in political science field experiments often is minimal and does not automatically place subjects at any appreciable risk.
Scholars must be consistent in pushing back on the implied link between deception and subject harm when political science research is concerned. Researchers also should be forceful in reminding IRB members and campus administrators that any human-subjects research participant could sue the university, even in cases when informed consent was obtained as part of a more common methodology (e.g., observational survey research). There is simply no empirical evidence that universities face heightened legal exposure by allowing political science field experiments to take place, and this point should be articulated in discussions with IRB members and administrators.
SAY UNION YES
Labor unions are a mixed bag in many ways, but joining one representing college professors—especially before submitting a field-experiment proposal to an IRB—may be the best investment a scholar can make since student loans. This is because the National Education Association and similar organizations have in-house legal representatives who work to resolve issues with university administrators, including IRB chairs. Moreover, when an informal resolution of these issues cannot be achieved, the unions have legal counsels that may assist faculty in developing lawsuit petitions and other legal strategies to bring relief. Of course, this does not mean that suing an institution over an IRB disagreement is a preferred use of one’s time, but it may be a necessary action depending on the circumstances. Union membership will provide benefits that assist the scholar in being proactive about protecting a research agenda from IRB overreach. Having membership, particularly on campuses that are non-unionized, also may be enough of a signal to university administrators that one means business—and that the institution should attempt to address any reasonable concerns a scholar raises about IRB practices.
CALL ON APSA AND CONSIDER LEGAL REMEDIES
Although many APSA members may never use field experiments—and it is likely that some within the discipline will maintain an aversion to the methodology—it is incumbent on our national organization to establish a clear position on IRB processes and examples of overreach. At issue is the core of what it means to have academic freedom, and all political scientists should support efforts to inform university IRBs across the United States that, whereas research designs akin to the Montana mailer project cannot be justified, neither can the abuse of faculty research agendas based on IRB ignorance, overly cautious administrators, or both. The APSA Committee on Professional Ethics, Rights, and Freedoms is a good resource to explore, particularly at the outset of a situation in which academic freedoms are being impinged.
The problem in simply calling on APSA to help with reform is that IRB issues are, as stated, local in nature. This makes compliance across thousands of institutions a nonstarter, even if APSA made IRB reform its top priority. Therefore, issuing a task force report on IRBs is unlikely to make much of an impact on faculty in dealing with IRBs. A more useful service for APSA to provide is a list of potential litigators who are experienced in both employment and higher-education law (if a lawsuit against a university and an IRB becomes necessary). Realistically, the number of experienced lawyers in these specialties will not be significant or evenly distributed geographically. Because most lawsuits against an IRB will be heard in a federal district court, state licensing will not matter (although most other university-specific suits will need to be brought in state court). At the least, developing a list of attorneys who could be called on for their expertise and general recommendations will go far in supporting faculty who bring grievances.
At issue is the core of what it means to have academic freedom, and all political scientists should support efforts to inform university IRBs across the United States that, whereas research designs akin to the Montana mailer project cannot be justified, neither can the abuse of faculty research agendas based on IRB ignorance, overly cautious administrators, or both.
EDUCATE AND INFILTRATE
It is tempting to view the IRBs as an adversary when the board acts unfairly in handing down prohibitive decisions that affect a scholar’s research agenda. Although this may be a correct characterization of a local board (depending on the circumstances), it also may be that an institution’s IRB members are not intending to make a scholar’s life more difficult and are not deeply committed to a view that field experiments are incompatible with ethical research. Realistically, most IRB members are not trying to complicate their colleagues’ professional lives, even if making their decisions based on methodological ignorance. As such, an IRB rejection may be an opportunity to educate IRB members about field interventions and their common use in the discipline (ideally, board members would have already done their homework on the matter, but this returns us to the notion of how researchers at non-R1 institutions may suffer in greater proportion). Offering to explain the methodology in a nonthreatening manner—perhaps by showing examples of already-published field experiments that are close to the proposed design—may convince an IRB to reverse course. Another option is to inquire about how IRB members are selected at an institution and request the opportunity to serve on the board. In this case, it might be possible to encourage IRB reform from within.
DON’T GIVE UP
An often-overlooked reality about universities is that administrators are generally not in their position forever; this is true of IRB members as well. IRBs and administrations currently hostile to field experimentation may be replaced by those who—if not actual supporters of the methodology—are willing to be reasonable in working with faculty. Of course, this trend can work in the opposite direction as well, with previously hospitable IRBs and campus leaders being replaced by hostile personnel. This is why it is incumbent on faculty to make IRB reform part of an agenda on a par with the recent discipline-wide attention to proposed National Science Foundation cuts to social science. Given that IRBs affect virtually all political science data collection in some way, this is a fight that scholars cannot afford to abandon.
