The target article calls for “systematically dismantling” the “fundamentally flawed” practice of using implicit social cognition research to inform our understanding of real-world inequality. Sweeping conclusions and comprehensive recommendations of this kind, published in a leading journal of our discipline, should be supported by powerful arguments reflecting the latest state of the literature. Instead, the target article mischaracterizes the methods, goals, and state of implicit social cognition research while referencing a mere eight empirical papers, the most recent of which was published over a decade ago.
According to the target article, the “standard research cycle” begins with the observation that groups differ on some real-world outcome and has the goal of explaining, and eventually eliminating, such differences. This statement is misleadingly narrow. Not all memory research seeks to cure dementia; not all phonological awareness research tries to eradicate dyslexia; and not all auditory perception research contributes to the development of hearing aids. Similarly, much implicit social cognition research explores basic aspects of thought and behavior, including learning and representation (Kurdi & Dunham, Reference Kurdi and Dunham2020), social cognitive development (Dunham, Baron, & Banaji, Reference Dunham, Baron and Banaji2008), and cultural change (Charlesworth & Banaji, Reference Charlesworth and Banaji2019), without making any claim of immediate applicability to real-world problems. Thus, whether implicit social cognition research can explain real-world inequality should not be treated as its sole measure of success.
Of course, some of this literature does speak to real-world outcomes and behaviors. But here too the target article misses the mark. Specifically, according to the target article, researchers establish some experimental effect of social category knowledge in a small sample of naïve undergraduate participants in the lab and, without any further ado, conclude that the processes observed in the lab directly explain real-world disparities. In fact, as discussed below, much recent implicit social cognition research does not bear much resemblance to this description.
One relevant line of research has documented practical ramifications of basic implicit cognitive processes. For instance, transgender and cisgender children have been shown not to meaningfully differ from each other in implicit gender identity (Olson, Key, & Eaton, Reference Olson, Key and Eaton2015), thus providing a counterweight to prior claims of “psychological deviance.” In other cases, changes in implicit social cognition have been shown to track meaningful experiences in field settings: For example, exposure to female college professors in science, technology, engineering, and mathematics (STEM) fields can produce long-term effects on implicit gender stereotypes and self-concept (Dasgupta & Asgari, Reference Dasgupta and Asgari2004), implying that the social structures in which we are embedded shape the ways in which we envision our future possibilities.
Other research has investigated the relationship between implicit measures and ecologically meaningful measures of intergroup behavior (Kurdi et al., Reference Kurdi, Seitchik, Axt, Carroll, Karapetyan and Kaushik2019b). For example, implicit math–gender stereotypes predict actual academic achievement among high school students (Steffens, Jelenec, & Noack, Reference Steffens, Jelenec and Noack2010); implicit weight stereotypes predict actual callbacks of job applicants among human resources professionals (Agerström & Rooth, Reference Agerström and Rooth2011); managers' implicit competence stereotypes predict actual job performance of their minority employees (Glover, Pallais, & Pariente, Reference Glover, Pallais and Pariente2017); and doctors' implicit evaluations predict actual rapport, satisfaction, and treatment adherence among Black patients (Hagiwara et al., Reference Hagiwara, Penner, Gonzalez, Eggly, Dovidio and Gaertner2013; Penner et al., Reference Penner, Dovidio, Gonzalez, Albrecht, Chapman, Foster and Eggly2016, Reference Penner, Dovidio, West, Gaertner, Albrecht, Dailey and Markova2010).
Echoing an oft-repeated argument, the target article hastens to underscore that studies of predictive validity produce small correlations between implicit attitudes and intergroup behavior. The finding that the relationship between explicit attitudes and intergroup behavior is almost exactly the same size (Kurdi et al., Reference Kurdi, Seitchik, Axt, Carroll, Karapetyan and Kaushik2019b) receives no mention. What's more, the mean implicit–behavior correlation sits right around the 25th percentile of all effect sizes in social psychology, with the largest implicit–behavior correlations at the individual level approaching the 70th percentile of that distribution (Lovakov & Agadullina, Reference Lovakov and Agadullina2021).
Equally absent is any discussion of studies that investigate the association between implicit cognition and real-world inequality at the level of geographic units, which have produced large effects in multiple domains (Hehman, Calanchini, Flake, & Leitner, Reference Hehman, Calanchini, Flake and Leitner2019; Payne, Vuletich, & Lundberg, Reference Payne, Vuletich and Lundberg2017). For example, this work has demonstrated that regions with higher levels of implicit race bias are characterized by more frequent police killings of Black Americans (Hehman, Flake, & Calanchini, Reference Hehman, Flake and Calanchini2018), as well as more racial disparity in school disciplinary actions (Riddle & Sinclair, Reference Riddle and Sinclair2019) and upward mobility (Chetty, Hendren, Jones, & Porter, Reference Chetty, Hendren, Jones and Porter2020).
Finally, remarkable correspondence has also been found between individual-level conceptual associations indexed by implicit measures and cultural-level conceptual associations computationally derived from vast amounts of text produced spontaneously and outside any experimental setting (Caliskan & Lewis, Reference Caliskan and Lewis2020). Evidence for such alignment has been provided across different contexts, including a comprehensive examination of social group attitudes and stereotypes (Caliskan, Bryson, & Narayanan, Reference Caliskan, Bryson and Narayanan2017), the relationship between implicit beliefs and evaluations (Kurdi, Mann, Charlesworth, & Banaji, Reference Kurdi, Mann, Charlesworth and Banaji2019a), and the development of gender biases over the lifespan (Charlesworth, Yang, Mann, Kurdi, & Banaji, Reference Charlesworth, Yang, Mann, Kurdi and Banaji2021).
Little, if any, of the criticism formulated in the target article seems applicable to methodologically sound implicit social cognition research conducted over the past decade. Far from simply assuming a one-to-one correspondence between findings obtained with small undergraduate samples in artificial lab settings and real-world inequality, an increasingly large group of investigators have made serious efforts to establish connections between implicit measures of social cognition and group-based disparities. Specifically, all of the studies discussed above include at least one (but typically all) of the following elements: samples consisting of experts or members of the general public; real behaviors of consequence observed under ecologically realistic conditions; and the availability of ample individuating information during the decision-making process.
Implicit social cognition research has obviously not been immune to some of the same methodological missteps that have troubled much of psychology and the behavioral sciences over the past few decades. However, as should be clear based on even this brief review, there is considerable reason for optimism. Most importantly, further improvement and innovation won't be fueled by throwing out the baby with the bathwater. Instead, whether the goal is basic science or uncovering the antecedents, mechanisms, and consequences of real-world inequality, we urge renewed focus on theory building, study design, and statistical inference. And accurately characterizing the field that one critiques.
The target article calls for “systematically dismantling” the “fundamentally flawed” practice of using implicit social cognition research to inform our understanding of real-world inequality. Sweeping conclusions and comprehensive recommendations of this kind, published in a leading journal of our discipline, should be supported by powerful arguments reflecting the latest state of the literature. Instead, the target article mischaracterizes the methods, goals, and state of implicit social cognition research while referencing a mere eight empirical papers, the most recent of which was published over a decade ago.
According to the target article, the “standard research cycle” begins with the observation that groups differ on some real-world outcome and has the goal of explaining, and eventually eliminating, such differences. This statement is misleadingly narrow. Not all memory research seeks to cure dementia; not all phonological awareness research tries to eradicate dyslexia; and not all auditory perception research contributes to the development of hearing aids. Similarly, much implicit social cognition research explores basic aspects of thought and behavior, including learning and representation (Kurdi & Dunham, Reference Kurdi and Dunham2020), social cognitive development (Dunham, Baron, & Banaji, Reference Dunham, Baron and Banaji2008), and cultural change (Charlesworth & Banaji, Reference Charlesworth and Banaji2019), without making any claim of immediate applicability to real-world problems. Thus, whether implicit social cognition research can explain real-world inequality should not be treated as its sole measure of success.
Of course, some of this literature does speak to real-world outcomes and behaviors. But here too the target article misses the mark. Specifically, according to the target article, researchers establish some experimental effect of social category knowledge in a small sample of naïve undergraduate participants in the lab and, without any further ado, conclude that the processes observed in the lab directly explain real-world disparities. In fact, as discussed below, much recent implicit social cognition research does not bear much resemblance to this description.
One relevant line of research has documented practical ramifications of basic implicit cognitive processes. For instance, transgender and cisgender children have been shown not to meaningfully differ from each other in implicit gender identity (Olson, Key, & Eaton, Reference Olson, Key and Eaton2015), thus providing a counterweight to prior claims of “psychological deviance.” In other cases, changes in implicit social cognition have been shown to track meaningful experiences in field settings: For example, exposure to female college professors in science, technology, engineering, and mathematics (STEM) fields can produce long-term effects on implicit gender stereotypes and self-concept (Dasgupta & Asgari, Reference Dasgupta and Asgari2004), implying that the social structures in which we are embedded shape the ways in which we envision our future possibilities.
Other research has investigated the relationship between implicit measures and ecologically meaningful measures of intergroup behavior (Kurdi et al., Reference Kurdi, Seitchik, Axt, Carroll, Karapetyan and Kaushik2019b). For example, implicit math–gender stereotypes predict actual academic achievement among high school students (Steffens, Jelenec, & Noack, Reference Steffens, Jelenec and Noack2010); implicit weight stereotypes predict actual callbacks of job applicants among human resources professionals (Agerström & Rooth, Reference Agerström and Rooth2011); managers' implicit competence stereotypes predict actual job performance of their minority employees (Glover, Pallais, & Pariente, Reference Glover, Pallais and Pariente2017); and doctors' implicit evaluations predict actual rapport, satisfaction, and treatment adherence among Black patients (Hagiwara et al., Reference Hagiwara, Penner, Gonzalez, Eggly, Dovidio and Gaertner2013; Penner et al., Reference Penner, Dovidio, Gonzalez, Albrecht, Chapman, Foster and Eggly2016, Reference Penner, Dovidio, West, Gaertner, Albrecht, Dailey and Markova2010).
Echoing an oft-repeated argument, the target article hastens to underscore that studies of predictive validity produce small correlations between implicit attitudes and intergroup behavior. The finding that the relationship between explicit attitudes and intergroup behavior is almost exactly the same size (Kurdi et al., Reference Kurdi, Seitchik, Axt, Carroll, Karapetyan and Kaushik2019b) receives no mention. What's more, the mean implicit–behavior correlation sits right around the 25th percentile of all effect sizes in social psychology, with the largest implicit–behavior correlations at the individual level approaching the 70th percentile of that distribution (Lovakov & Agadullina, Reference Lovakov and Agadullina2021).
Equally absent is any discussion of studies that investigate the association between implicit cognition and real-world inequality at the level of geographic units, which have produced large effects in multiple domains (Hehman, Calanchini, Flake, & Leitner, Reference Hehman, Calanchini, Flake and Leitner2019; Payne, Vuletich, & Lundberg, Reference Payne, Vuletich and Lundberg2017). For example, this work has demonstrated that regions with higher levels of implicit race bias are characterized by more frequent police killings of Black Americans (Hehman, Flake, & Calanchini, Reference Hehman, Flake and Calanchini2018), as well as more racial disparity in school disciplinary actions (Riddle & Sinclair, Reference Riddle and Sinclair2019) and upward mobility (Chetty, Hendren, Jones, & Porter, Reference Chetty, Hendren, Jones and Porter2020).
Finally, remarkable correspondence has also been found between individual-level conceptual associations indexed by implicit measures and cultural-level conceptual associations computationally derived from vast amounts of text produced spontaneously and outside any experimental setting (Caliskan & Lewis, Reference Caliskan and Lewis2020). Evidence for such alignment has been provided across different contexts, including a comprehensive examination of social group attitudes and stereotypes (Caliskan, Bryson, & Narayanan, Reference Caliskan, Bryson and Narayanan2017), the relationship between implicit beliefs and evaluations (Kurdi, Mann, Charlesworth, & Banaji, Reference Kurdi, Mann, Charlesworth and Banaji2019a), and the development of gender biases over the lifespan (Charlesworth, Yang, Mann, Kurdi, & Banaji, Reference Charlesworth, Yang, Mann, Kurdi and Banaji2021).
Little, if any, of the criticism formulated in the target article seems applicable to methodologically sound implicit social cognition research conducted over the past decade. Far from simply assuming a one-to-one correspondence between findings obtained with small undergraduate samples in artificial lab settings and real-world inequality, an increasingly large group of investigators have made serious efforts to establish connections between implicit measures of social cognition and group-based disparities. Specifically, all of the studies discussed above include at least one (but typically all) of the following elements: samples consisting of experts or members of the general public; real behaviors of consequence observed under ecologically realistic conditions; and the availability of ample individuating information during the decision-making process.
Implicit social cognition research has obviously not been immune to some of the same methodological missteps that have troubled much of psychology and the behavioral sciences over the past few decades. However, as should be clear based on even this brief review, there is considerable reason for optimism. Most importantly, further improvement and innovation won't be fueled by throwing out the baby with the bathwater. Instead, whether the goal is basic science or uncovering the antecedents, mechanisms, and consequences of real-world inequality, we urge renewed focus on theory building, study design, and statistical inference. And accurately characterizing the field that one critiques.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflict of interest
Benedek Kurdi is a member of the Scientific Advisory Board of Project Implicit, a 501(c)(3) non-profit organization and international collaborative of researchers who are interested in implicit social cognition.