Mediation and causation

Murphy (Reference Murphy2021) notes that tests of mediation are notoriously difficult to interpret. He provides many reasons for this and discusses ways to avoid misinterpreting what Spencer et al. (Reference Spencer, Zanna and Fong2005) call measurement-of-mediation designs. Implicit in Murphy’s discussion of mediation is that in the vast majority of I-O studies of mediation, the relation between the mediating and dependent variable is examined via passive observation (Stone-Romero & Rosopa, Reference Stone-Romero and Rosopa2008). Mediation, however, implies causation. In simple mediation models, the independent variable causes the mediator (X→M) and the mediator causes the dependent variable (M→Y). Shadish et al. (Reference Shadish, Cook and Campbell2002) noted that there are three requirements for inferring causality. We list these as they apply specifically to mediation: (a) M precedes Y in time, (b), M and Y vary together, and (c) there are no plausible alternative explanations for the relationship between M and Y. Only randomized experiments satisfy all three requirements.

Therefore, measurement-of-mediation designs are vulnerable to misinterpretation. To believe the results of studies using nonexperimental designs, the reader must assume that the researcher has correctly specified the causal ordering of variables and has controlled all important confounds through design features or statistical means (Stone-Romero & Rosopa, Reference Stone-Romero and Rosopa2008). The assumption that no omitted variables influence the X→M→Y causal chain is referred to as the no omitted variable assumption or the no confounding assumption. (Pirlott & MacKinnon, Reference Pirlott and MacKinnon2016). This assumption is rarely justified and often violated. One way to provide evidence for the assumption is to follow up a study that used measurement of mediation with one that demonstrates the X→M→Y relationship experimentally (Podsakoff & Podsakoff, Reference Podsakoff and Podsakoff2019).

Manipulate the mediator

In the field of social psychology, attention to the overreliance on measurement-of-mediation designs led Spencer et al. (Reference Spencer, Zanna and Fong2005) to call for researchers to test mediation by establishing a causal chain through experimentation. This involves doing two sequential studies: in study 1, X is manipulated and M and X are measured, and in study 2, M is manipulated and X is measured. Pirlott and MacKinnon (Reference Pirlott and MacKinnon2016) distinguished between directly manipulating the presence or absence of the mediator versus encouraging or discouraging the mediator in manipulation-of-mediator designs. We discuss the direct approach first.

An example of directly manipulating the mediator is a study by Meyer and Gellatly (Reference Meyer and Gellatly1988). They examined the mediating effect of perceived performance norms on the relationship between assigned goals and performance. In the first experiment, the authors manipulated assigned goal (X; easy, difficult, impossible) and measured its effect on perceived performance norms (M) and performance (Y). In the second study, the researchers experimentally manipulated performance norms (M; low, high, no norm) to examine their effect on performance (Y). More recently, Nolan and Highhouse (Reference Nolan and Highhouse2014) examined the mediating effect of perceived autonomy on the relationship between standardized hiring practices and user resistance. In their first experiment, the authors manipulated type of hiring procedure (X; highly standardized, unstandardized) to examine the effects on perceived autonomy (M) and user resistance (Y). In the second experiment, the researchers manipulated the amount of autonomy (M) in the standardized hiring procedure to examine its effect on user resistance (Y).Footnote ¹

Imai et al. (Reference Imai, Tingley and Yamamoto2013) point out that directly manipulating the mediator is not always possible and that even when it is, the researcher must make a strong argument that the measured and manipulated versions of the mediator are consistent (i.e., they are the same construct and would have the same effect on the outcome). In these situations, they recommend employing “designs with imperfect manipulation” (p. 18), referred to as encouragement designs. These involve manipulations that are directed at increasing (encouraging) or decreasing (discouraging) the value of the mediator. Pirlott and MacKinnon (Reference Pirlott and MacKinnon2016) point to a study by Li et al. (Reference Li, Johnson, Cohen, Williams, Knowles and Chen2012), who manipulated their mediator “belief in a soul” by randomly assigning people to either write an essay suggesting that souls do exist (encouragement condition) or do not exist (discouragement condition). This approach involves strengthening or weakening the mediator, rather than fully changing the mediator.

Although manipulating the mediator is surely an improvement in terms of our ability to identify a causal mechanism, some experimentalists suggest that even establishing a causal chain falls short of demonstrating how an independent variable affects an outcome through the mediator (e.g., Imai, et al., Reference Imai, Tingley and Yamamoto2013). These authors suggest that parallel and crossover designs are better equipped to identify causal mechanisms.

In an experimental mediation using a parallel design, a sample is randomly split in two and two randomized studies are conducted in parallel: one study manipulates X and measures M and Y, and the other study simultaneously manipulates X and M, and measures Y. This is essentially a combination of a measurement-of-mediator design and a manipulation-of-mediator design. In a specific version of a crossover design suited to testing mediation, participants are exposed to both conditions of the independent variable in randomized order. In each condition, at stage 1, the M and Y are measured. Then, at stage 2 participants are assigned to the opposite condition and they are assigned to the value of M from the first stage—and Y is measured. An important assumption of crossover designs is no carryover effects. Both parallel and crossover designs can be used with direct or encouragement manipulations.

Anticipated objections to manipulating the mediator

Final thoughts

Although there are a number of limitations of experimental approaches that manipulate the mediator, many of these can be addressed by including within the same research program a complementary study using the measurement-of-mediation design. Both approaches to establishing a causal chain may be used to offset the limitations of each. Like Podsakoff and Podsakoff (Reference Podsakoff and Podsakoff2019), we believe that the benefits of experimental approaches to mediation outweigh the limitations. Researchers should consider this underused approach as a simple tool that can provide powerful evidence. Like Murphy (Reference Murphy2021), we encourage I-O researchers to go back to the basics and consider whether statistically simpler solutions may be appropriate.