In his target article, Cesario argues that laboratory experiments cannot shed light on real group disparities because they leave out information, forces, and contingencies that operate in ordinary life. These so-called flaws motivate his recommendation to abandon the use of psychology experiments for understanding three topics related to racial discrimination. Even the least astute reader of this paper will ask the obvious question: Why are these three topics singled out as uniquely unsuitable for experimental treatment? Why would this question be raised about topics of study that came into being by the wizardry of a tribe called social psychologists, who in mid-twentieth century changed the world of science by boldly asserting that problems as complex as obedience to authority, bystander non-intervention, and minimal group effects, could be studied in the same way as atoms and cells. What in the name of Lewin, Heider, and Festinger does Cesario mean when he states that the experimental method is uniquely unsuited to the study of these three topics in social psychology? Why not add to his chosen topics for the garbage heap other equally complex problems such as climate change to be outside the bounds of the experimental method? Surely it's not easy to create the glaciers of the Uttarakhand in the lab for study so surely we should abandon all study of the effects of global warming! So, we must again reassert that the laziest reader of the target article will yawn out one question: Why is Cesario selecting the three topics from one subfield (social psychology) of one science (psychology), as uniquely unsuited for experimental treatment? We too wondered why.
In the rest of this comment, we do not engage with any of the specific areas of research Cesario selects, as that does not matter. Instead, we flatly state that if his thesis is to be taken at all seriously, we would need to reevaluate all of experimental psychology. In fact, if Cesario is to be taken seriously, it is not just three areas in psychology that should be abandoned, but the entire enterprise of physics, chemistry, and biology, the National Science Foundation, and BBS itself, that should each be abandoned and immediately.
Philosophers, mathematicians, and astronomers have observed regularities in the world for centuries. But discovery accelerated dramatically when natural philosophers began creating controlled conditions that abstracted away many details of ordinary experience in order to gain experimental control. The past 2000 years of the scientific method is the reason we boast of human progress, whether it be rockets, submarines, and airplanes; synthetic polymers, the cathode ray, or the periodic table; the discovery of antibodies, the sequencing of the genome, and vaccines that eradicate diseases such as smallpox and control viruses such as COVID.
Only when thought experiments based on intuition in the real world gave way to test tubes and Petri dishes did we have a chance to understand reality. Galileo told stories about dropping balls from the Tower of Pisa but he did his actual work on artificial equipment by rolling them down inclined planes at home. It was the only way to control the wind and slow the fall enough to measure with accuracy. Eventually, the physician's trial and error gave way to randomized trials. The randomized experiment, keeping everything constant and varying a single variable, remains not just a good way, but the dominant way to establish causality with confidence.
True, ordinary intuition has always had trouble with the scientific method and understanding the reason for varying single variables. Even today, reactionary forces impart the message that the earth is flat and that a god made humans and placed them on earth. But those of us who are the beneficiaries of basic education and reason roll our eyes at these misguided views. When a youngster makes such a remark, we explain that to answer a question such as “will a feather and a rock fall at equal or unequal speed” we must start with the idea of a vacuum and answer the question in a counterintuitive way. Left to Cesario's argument (that we must drop Galileo's balls from the Tower of Pisa given its real-world allure), we would have little to show for all our centuries of science.
What Cesario calls flaws – missing information, missing forces, and missing contingencies – are, in fact, the point of the experimental method. Researchers working in this area do not claim that laboratory experiments capture the complexity of the real world, because the purpose of experiments is not to reinstate the real world in the lab. Experiments in this tradition look at one factor at a time, such as race, but also time pressure, anxiety, motivations to be unbiased, the identity of the subject, and the difference between police officers and civilians, just to name a few (for a review see Payne & Correll, Reference Payne and Correll2020). For those doing this work, results of lab experiments are in constant conversation with other research, such as field experiments and observational studies of real-world disparities. For example, geographical patterning of race bias based on experimental tasks can be predicted by patterns of enslavement before the Civil War (Payne, Vuletich, & Brown-Iannuzzi, Reference Payne, Vuletich and Brown-Iannuzzi2019). Patterns of implicit bias across countries can predict educational disparities in actual standardized tests (Nosek et al., Reference Nosek, Smyth, Sriram, Lindner, Devos, Ayala and Greenwald2009). And cities where residents more easily associate Black people with weapons on laboratory tasks have larger racial disparities on actual police use of force (Hehman, Flake, & Calanchini, Reference Hehman, Flake and Calanchini2018). To claim that experimentalists make inferences from experiments to everyday discrimination without doing the work of empirically integrating data at multiple levels of analysis is both naïve and factually simply wrong.
The observation that experiments sometimes lack realism is not new. Gergen (Reference Gergen1973) argued that the findings of social psychology change with history and culture, casting doubt on whether laboratory experiments can produce insights that are general and cumulative. Neisser's (Reference Neisser, Gruneberg, Morris and Sykes1978) call for the study of everyday memory lamented that memory research doesn't answer enough interesting or socially significant questions. The trade-off between realism and experimental control is well understood (Banaji & Crowder, Reference Banaji and Crowder1989). But even these critics called for increased attention and integration, not simply abandoning the experimental method. That dubious innovation is new with Cesario's target article and it should be abandoned with haste unless the goal is to make a mockery of scientific psychology.
In his target article, Cesario argues that laboratory experiments cannot shed light on real group disparities because they leave out information, forces, and contingencies that operate in ordinary life. These so-called flaws motivate his recommendation to abandon the use of psychology experiments for understanding three topics related to racial discrimination. Even the least astute reader of this paper will ask the obvious question: Why are these three topics singled out as uniquely unsuitable for experimental treatment? Why would this question be raised about topics of study that came into being by the wizardry of a tribe called social psychologists, who in mid-twentieth century changed the world of science by boldly asserting that problems as complex as obedience to authority, bystander non-intervention, and minimal group effects, could be studied in the same way as atoms and cells. What in the name of Lewin, Heider, and Festinger does Cesario mean when he states that the experimental method is uniquely unsuited to the study of these three topics in social psychology? Why not add to his chosen topics for the garbage heap other equally complex problems such as climate change to be outside the bounds of the experimental method? Surely it's not easy to create the glaciers of the Uttarakhand in the lab for study so surely we should abandon all study of the effects of global warming! So, we must again reassert that the laziest reader of the target article will yawn out one question: Why is Cesario selecting the three topics from one subfield (social psychology) of one science (psychology), as uniquely unsuited for experimental treatment? We too wondered why.
In the rest of this comment, we do not engage with any of the specific areas of research Cesario selects, as that does not matter. Instead, we flatly state that if his thesis is to be taken at all seriously, we would need to reevaluate all of experimental psychology. In fact, if Cesario is to be taken seriously, it is not just three areas in psychology that should be abandoned, but the entire enterprise of physics, chemistry, and biology, the National Science Foundation, and BBS itself, that should each be abandoned and immediately.
Philosophers, mathematicians, and astronomers have observed regularities in the world for centuries. But discovery accelerated dramatically when natural philosophers began creating controlled conditions that abstracted away many details of ordinary experience in order to gain experimental control. The past 2000 years of the scientific method is the reason we boast of human progress, whether it be rockets, submarines, and airplanes; synthetic polymers, the cathode ray, or the periodic table; the discovery of antibodies, the sequencing of the genome, and vaccines that eradicate diseases such as smallpox and control viruses such as COVID.
Only when thought experiments based on intuition in the real world gave way to test tubes and Petri dishes did we have a chance to understand reality. Galileo told stories about dropping balls from the Tower of Pisa but he did his actual work on artificial equipment by rolling them down inclined planes at home. It was the only way to control the wind and slow the fall enough to measure with accuracy. Eventually, the physician's trial and error gave way to randomized trials. The randomized experiment, keeping everything constant and varying a single variable, remains not just a good way, but the dominant way to establish causality with confidence.
True, ordinary intuition has always had trouble with the scientific method and understanding the reason for varying single variables. Even today, reactionary forces impart the message that the earth is flat and that a god made humans and placed them on earth. But those of us who are the beneficiaries of basic education and reason roll our eyes at these misguided views. When a youngster makes such a remark, we explain that to answer a question such as “will a feather and a rock fall at equal or unequal speed” we must start with the idea of a vacuum and answer the question in a counterintuitive way. Left to Cesario's argument (that we must drop Galileo's balls from the Tower of Pisa given its real-world allure), we would have little to show for all our centuries of science.
What Cesario calls flaws – missing information, missing forces, and missing contingencies – are, in fact, the point of the experimental method. Researchers working in this area do not claim that laboratory experiments capture the complexity of the real world, because the purpose of experiments is not to reinstate the real world in the lab. Experiments in this tradition look at one factor at a time, such as race, but also time pressure, anxiety, motivations to be unbiased, the identity of the subject, and the difference between police officers and civilians, just to name a few (for a review see Payne & Correll, Reference Payne and Correll2020). For those doing this work, results of lab experiments are in constant conversation with other research, such as field experiments and observational studies of real-world disparities. For example, geographical patterning of race bias based on experimental tasks can be predicted by patterns of enslavement before the Civil War (Payne, Vuletich, & Brown-Iannuzzi, Reference Payne, Vuletich and Brown-Iannuzzi2019). Patterns of implicit bias across countries can predict educational disparities in actual standardized tests (Nosek et al., Reference Nosek, Smyth, Sriram, Lindner, Devos, Ayala and Greenwald2009). And cities where residents more easily associate Black people with weapons on laboratory tasks have larger racial disparities on actual police use of force (Hehman, Flake, & Calanchini, Reference Hehman, Flake and Calanchini2018). To claim that experimentalists make inferences from experiments to everyday discrimination without doing the work of empirically integrating data at multiple levels of analysis is both naïve and factually simply wrong.
The observation that experiments sometimes lack realism is not new. Gergen (Reference Gergen1973) argued that the findings of social psychology change with history and culture, casting doubt on whether laboratory experiments can produce insights that are general and cumulative. Neisser's (Reference Neisser, Gruneberg, Morris and Sykes1978) call for the study of everyday memory lamented that memory research doesn't answer enough interesting or socially significant questions. The trade-off between realism and experimental control is well understood (Banaji & Crowder, Reference Banaji and Crowder1989). But even these critics called for increased attention and integration, not simply abandoning the experimental method. That dubious innovation is new with Cesario's target article and it should be abandoned with haste unless the goal is to make a mockery of scientific psychology.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflict of interest
None.