The analysis of social and psychological mechanisms at the core of complex behavioral phenomena, such as persistence of group disparities (e.g., Duell & Valasek, Reference Duell and Valasek2019; Fershtman & Gneezy, Reference Fershtman and Gneezy2001; Fryer, Goeree, & Holt, Reference Fryer, Goeree and Holt2005; Haan, Offerman, & Sloof, Reference Haan, Offerman and Sloof2015; Landa & Duell, Reference Landa and Duell2015), is central to contemporary social sciences. Yet some of the important methodological elements of such studies may sometimes appear puzzling. Among such elements are those that concern the differences between features of laboratory studies that seek to instantiate and isolate specific mechanisms and the details of real-world interactions that these studies model. Failure to properly interpret such differences undergirds the following two claims, most recently advanced in Cesario:
(1) That, as a general matter, because laboratory studies draw on subject pools that are different from those in the modeled interactions, the laboratory results cannot effectively speak to real-world contexts with otherwise proximate decision situations.
(2) That (the laboratory) analysis of counterfactual conditions is irrelevant for understanding real-world social facts.
To see the implications of the first claim, suppose, first, that a social mechanism analyzed in the lab gives rise to predictable behavior B following treatment T, and that the experimental analysis shows that introducing treatment T′ leads to change in behavior, to B′. For Claim 1 to have force, the underlying assertion would have to be that as a rule, rather than an anomaly, T′ would be just as, if not more, likely to produce, in a sample more proximate to the target context, a change from B to some B′ that is in the opposite direction from B relative to B′. As a matter of evidence provided, this assertion is certainly under-determined: While Cesario lays out studies demonstrating that expert shooters tend to show no or little race-based bias in their decision to shoot, this finding is, plainly, not equivalent to an effect in the opposite direction from the seminal shooter bias studies.
Explaining the differences in these studies' findings is important, yet assuming that these differences, let alone putative behavioral patterns with the opposite sign of what is observed in the lab, is the right general expectation is deeply problematic. At the core of the concept of mechanisms is the idea of robust patterns of connections between causes and effects, driven by general properties of psychological, economic, or other social responses. In this way, portability – among others, from the lab to the world outside it; from the context with one set of subjects to a context with another set; and so on – is central to the very concept of mechanisms (Hedström & Ylikoski, Reference Hedström and Ylikoski2010; Hitchcock, Reference Hitchcock2012). In attacking this portability in principle, Claim 1 is, in effect, calling to abandon the study of social mechanisms as such – a position that today should strike many as, at least prima facie, implausible.
Opposing Claim 1 does not take away from the question of why expert shooters are less biased than many other groups. But a better way to conceive of such a question is as a targeted call for resolving a specific anomaly. Laboratory studies, including studies with varieties of convenience samples, frequently establish mechanisms by which group membership relationships inform behavior, and, depending on application and mechanism, sometimes predict in-group bias, sometimes no bias or even over-correction toward out-group favoritism. These predictions form baseline expectations when taken to alternative target populations, but they are, of course, not the final word for understanding behavior within those populations. Further research needs to establish which of the potential mechanisms have greater weight in a particular target population, resolving the anomalies that may arise from the disjunctions of the observed behaviors across populations. In fact, the research on shooters' bias (Correll, Hudson, Guillermo, & Ma Reference Correll, Hudson, Guillermo and Ma2014; Correll, Park, Judd, & Wittenbrink, Reference Correll, Park, Judd and Wittenbrink2002; Johnson, Cesario, & Pleskac, Reference Johnson, Cesario and Pleskac2018) exemplifies just such a practice, moving from a simplified choice situation in a laboratory setting to more and more contextually rich settings to understand why in a particular population, for example the police force, the expressions of a mechanism giving rise to group bias may sometimes depart from the predictions in a convenience sample.
Claim 2 states that the laboratory analysis of counterfactual conditions (e.g., of situations where there are equally violent black and white offenders or equally skilled black and white workers) is irrelevant for understanding patterns of group discrimination. Yet the study of social mechanisms often requires positing counterfactual possibilities as initial conditions that may, by way of the hypothesized mechanism, help explain observables. Just because most companies do not, per Cesario, choose between similarly skilled black and white workers, or most police officers do not consider how to respond to expectationally similar black and white potential offenders, does not mean there would not be bias if such decision situations emerged. The anticipation of such bias and the relevant individuals' responses to that anticipation, including possible underinvestment in productive capacity or in costly compliance with the state, are some of the central building blocks of important social mechanisms that have been posited to help account for the observable patterns of inequality, quite apart from other determinants of asymmetric standing and treatment of different subpopulations (Moro, Reference Moro, Durlauf and Blume2009). We learn whether such mechanisms – engendering what is known as strategic or equilibrium discrimination – are psychologically plausible (Duell & Landa, Reference Duell and Landa2021a) and how they respond to institutional interventions (Duell & Landa, Reference Duell and Landa2021b) by isolating determinants of particular posited mechanisms, while shutting down others that may create confounding effects. Where such determinants include particular existing distributions of attributes within demographic subpopulations, this means positing counterfactual conditions. In this way, a mechanism of strategic discrimination may, for example, be distinguished from that of statistical discrimination – something that would be impossible if experimentalists were to turn away from modeling conditions that are rare or not representative of what one may find contemporaneously outside the lab.
Effectively addressing social ills requires understanding of the causal mechanisms that bring them about, rather than mere descriptions of associations within immediate target populations. Done well, this is, undoubtedly, a painstaking process that demands both theoretical and experimental imagination, but there is little alternative to it.
The analysis of social and psychological mechanisms at the core of complex behavioral phenomena, such as persistence of group disparities (e.g., Duell & Valasek, Reference Duell and Valasek2019; Fershtman & Gneezy, Reference Fershtman and Gneezy2001; Fryer, Goeree, & Holt, Reference Fryer, Goeree and Holt2005; Haan, Offerman, & Sloof, Reference Haan, Offerman and Sloof2015; Landa & Duell, Reference Landa and Duell2015), is central to contemporary social sciences. Yet some of the important methodological elements of such studies may sometimes appear puzzling. Among such elements are those that concern the differences between features of laboratory studies that seek to instantiate and isolate specific mechanisms and the details of real-world interactions that these studies model. Failure to properly interpret such differences undergirds the following two claims, most recently advanced in Cesario:
(1) That, as a general matter, because laboratory studies draw on subject pools that are different from those in the modeled interactions, the laboratory results cannot effectively speak to real-world contexts with otherwise proximate decision situations.
(2) That (the laboratory) analysis of counterfactual conditions is irrelevant for understanding real-world social facts.
To see the implications of the first claim, suppose, first, that a social mechanism analyzed in the lab gives rise to predictable behavior B following treatment T, and that the experimental analysis shows that introducing treatment T′ leads to change in behavior, to B′. For Claim 1 to have force, the underlying assertion would have to be that as a rule, rather than an anomaly, T′ would be just as, if not more, likely to produce, in a sample more proximate to the target context, a change from B to some B′ that is in the opposite direction from B relative to B′. As a matter of evidence provided, this assertion is certainly under-determined: While Cesario lays out studies demonstrating that expert shooters tend to show no or little race-based bias in their decision to shoot, this finding is, plainly, not equivalent to an effect in the opposite direction from the seminal shooter bias studies.
Explaining the differences in these studies' findings is important, yet assuming that these differences, let alone putative behavioral patterns with the opposite sign of what is observed in the lab, is the right general expectation is deeply problematic. At the core of the concept of mechanisms is the idea of robust patterns of connections between causes and effects, driven by general properties of psychological, economic, or other social responses. In this way, portability – among others, from the lab to the world outside it; from the context with one set of subjects to a context with another set; and so on – is central to the very concept of mechanisms (Hedström & Ylikoski, Reference Hedström and Ylikoski2010; Hitchcock, Reference Hitchcock2012). In attacking this portability in principle, Claim 1 is, in effect, calling to abandon the study of social mechanisms as such – a position that today should strike many as, at least prima facie, implausible.
Opposing Claim 1 does not take away from the question of why expert shooters are less biased than many other groups. But a better way to conceive of such a question is as a targeted call for resolving a specific anomaly. Laboratory studies, including studies with varieties of convenience samples, frequently establish mechanisms by which group membership relationships inform behavior, and, depending on application and mechanism, sometimes predict in-group bias, sometimes no bias or even over-correction toward out-group favoritism. These predictions form baseline expectations when taken to alternative target populations, but they are, of course, not the final word for understanding behavior within those populations. Further research needs to establish which of the potential mechanisms have greater weight in a particular target population, resolving the anomalies that may arise from the disjunctions of the observed behaviors across populations. In fact, the research on shooters' bias (Correll, Hudson, Guillermo, & Ma Reference Correll, Hudson, Guillermo and Ma2014; Correll, Park, Judd, & Wittenbrink, Reference Correll, Park, Judd and Wittenbrink2002; Johnson, Cesario, & Pleskac, Reference Johnson, Cesario and Pleskac2018) exemplifies just such a practice, moving from a simplified choice situation in a laboratory setting to more and more contextually rich settings to understand why in a particular population, for example the police force, the expressions of a mechanism giving rise to group bias may sometimes depart from the predictions in a convenience sample.
Claim 2 states that the laboratory analysis of counterfactual conditions (e.g., of situations where there are equally violent black and white offenders or equally skilled black and white workers) is irrelevant for understanding patterns of group discrimination. Yet the study of social mechanisms often requires positing counterfactual possibilities as initial conditions that may, by way of the hypothesized mechanism, help explain observables. Just because most companies do not, per Cesario, choose between similarly skilled black and white workers, or most police officers do not consider how to respond to expectationally similar black and white potential offenders, does not mean there would not be bias if such decision situations emerged. The anticipation of such bias and the relevant individuals' responses to that anticipation, including possible underinvestment in productive capacity or in costly compliance with the state, are some of the central building blocks of important social mechanisms that have been posited to help account for the observable patterns of inequality, quite apart from other determinants of asymmetric standing and treatment of different subpopulations (Moro, Reference Moro, Durlauf and Blume2009). We learn whether such mechanisms – engendering what is known as strategic or equilibrium discrimination – are psychologically plausible (Duell & Landa, Reference Duell and Landa2021a) and how they respond to institutional interventions (Duell & Landa, Reference Duell and Landa2021b) by isolating determinants of particular posited mechanisms, while shutting down others that may create confounding effects. Where such determinants include particular existing distributions of attributes within demographic subpopulations, this means positing counterfactual conditions. In this way, a mechanism of strategic discrimination may, for example, be distinguished from that of statistical discrimination – something that would be impossible if experimentalists were to turn away from modeling conditions that are rare or not representative of what one may find contemporaneously outside the lab.
Effectively addressing social ills requires understanding of the causal mechanisms that bring them about, rather than mere descriptions of associations within immediate target populations. Done well, this is, undoubtedly, a painstaking process that demands both theoretical and experimental imagination, but there is little alternative to it.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflict of interest
None.