Check 1

Begin by asking, “What do I want to be true and why?” Ideally, the scientific method is characterized by objectivity. Realistically, however, social psychological science is conducted by people who share many of the same biases as those they study (e.g., confirmation biases). It may therefore be useful for researchers, before going into the laboratory or the field, to strive to account for whatever biases they can by asking themselves, “What do I want to be true and why?” Although personal desires or preferences should have little sway in the scientific process, an early accounting of one's own explicit biases allows one to add design elements to ensure that all theoretically grounded hypotheses (not only those that are most palatable) are meaningfully considered and tested.

Check 2

Explicitly state the theoretical rationale for your hypothesis in the form of an if-then statement. Starting with a theoretical rationale for one's hypothesis is not only “good science,” but also a crucial part of avoiding bias. Theoretical foundations give a clear understanding of why one expects one's hypotheses to be true (Sutton & Staw Reference Sutton and Staw1995). One way to confirm that hypotheses are grounded in a theoretical rationale rather than ideological bias is to generate an if-then statement: If a given theoretical proposition is true, then the following effect should be observed. Generating an if-then statement requires researchers to zero in on the theoretical premise that grounds their prediction. Focusing on and explicitly stating the theoretical premise behind one's predictions helps ensure that hypotheses are not driven by preferences for what researchers want to find but are firmly grounded in theory.

Check 3

Generate theoretical arguments for competing hypotheses and design studies accordingly. McGuire's (Reference McGuire2004) perspectivist approach to research and knowledge acquisition argues that all possible hypotheses are true – one just needs to think through the moderators and conditions when one hypothesis is more likely to be true than another. For this reason, researchers should challenge themselves to generate theoretical rationales not only for their preferred hypothesis but also alternative hypotheses. Generating a strong theoretical explanation for different, if not opposite, patterns of results than those that are preferred or expected can attenuate tendencies toward hypothesis confirmation and advocacy, instead of hypothesis testing.

McGuire (Reference McGuire2004), for example, models the perspectivist approach by hypothesizing that viewing violence on television could lead to more aggressive behavior because exposure legitimizes violence as acceptable and therefore increases desires to behave aggressively (Berkowitz et al. Reference Berkowitz, Corwin and Heironimus1963). Alternatively, viewing violence on television could also lead to a reduction in aggressive behavior because exposure to violence provides a catharsis of hostile feelings and therefore reduces desires to behave aggressively (Feshbach & Singer Reference Feshbach and Singer1971).

Once researchers generate a theoretical rationale for competing hypotheses, they can adopt an appropriate empirical strategy. Testing competing hypotheses in one design allows the data to speak for themselves: Which account is most consistent with the data? Designing strong tests of alternative hypotheses, however, requires that each hypothesis have an equal opportunity to be supported. Sometimes this goal is best accomplished by designing multiple studies: one or more studies that provide a strong test of Hypothesis A, and one or more that provide equally strong tests of Hypothesis B or C (for examples, see Skitka & Tetlock Reference Skitka and Tetlock1993; Skitka et al. Reference Skitka, Mullen, Griffin, Hutchinson and Chamberlin2002). Thresholds for what counts as support for each hypothesis should be decided a priori and could be pre-registered to avoid moving the goalposts, or engaging in questionable research practices to favor one hypothesis over another (e.g., Simmons et al. Reference Simmons, Nelson and Simonsohn2011).

Check 4

Be open to adversarial collaborations. In a sense, this checklist provides steps for researchers to fight against their own biases and thus to be their own intellectual adversaries. Nonetheless, psychology has documented the power of motivated reasoning (Kunda Reference Kunda1990); it is possible that some biases will still slip through the cracks. Researchers should be open to pursuing adversarial collaborations as a fail-safe (see the Appendix in Mellers et al. [Reference Mellers, Hertwig and Kahneman2001] for a detailed example). One may ask a colleague who has different theoretical or partisan loyalties to review one's work, or, ideally, invite collaboration in each step of the research process. Being open to invitations from others' for adversarial collaboration would also be desirable.

The current checklist gives researchers tools to be more objective and skeptical architects of their own research. In the true spirit of scientific inquiry, social psychologists should aspire to put theories and hypotheses to the strictest of tests. Adhering to the above-mentioned guidelines may facilitate objectively motivated hypothesis testing rather than subjectively laden hypothesis advocacy or confirmation. Moreover, these suggestions represent more immediate solutions to the problem of ideological bias that do not require researchers to wait for a day when the field is marked by greater ideological diversity.