The aim of the social and behavioral sciences is to understand human behavior across a wide array of contexts. Our theories often make sweeping claims about human nature, assuming that our ancestors or offspring will be prone to the same biases and preferences. Yet we gloss over the fact that our research is often based on a single temporal context with a limited set of stimuli. Political and moral psychology are domains in which the context and stimuli are likely to matter a great deal (Brandt & Wagemans, Reference Brandt and Wagemans2017; Van Bavel et al., Reference Van Bavel, Mende-Siedlecki, Brady and Reinero2016). Understand differences between liberals and conservatives, when people are willing to sacrifice someone to save others, the behavior of political leaders, and the dynamics of intergroup conflict, all likely depend on the features of the research.
One of the major challenges of political psychology is understanding the generalizability of stimuli (see Yarkoni, this issue). Political psychologists often make broad claims about the differences (or lack thereof) between conservatives and liberals. However, the field's findings can vary widely depending on the specific stimuli used (e.g., topics, policies, etc.). Whether American Democrats or Republicans, for example, report higher belief superiority depends on the political issue researchers asked about (Toner, Leary, Asher, & Jongman-Sereno, Reference Toner, Leary, Asher and Jongman-Sereno2013). Scientists can therefore select different issues to show, alternatively, that people on the left or right of the political spectrum are more prone to a sense of superiority about their beliefs. It is only when looking across multiple topics that the overall quadratic relationship emerges (Harris & Van Bavel, Reference Harris and Van Bavel2021), suggesting the people on both ends of the spectrum are similarly prone to feelings of superiority when their attitudes are extreme.
This issue of non-generalizable stimuli extends to research on moral psychology as well. For instance, the victims of harm are often described using vague terminology, such as five nondescript workers on a track about to be run over by a trolley. In many studies, the victims are genderless, raceless, etc. (Hester & Gray, Reference Hester and Gray2020), which makes it difficult to generalize the findings to more vivid or real-world judgments (FeldmanHall et al., Reference FeldmanHall, Mobbs, Evans, Hiscox, Navrady and Dalgleish2012). Yet the research suggests that moral judgments are heavily influenced by contextual information and stimuli.
Another often-overlooked aspect of generalizability in both political and moral psychology is the changing temporal dynamics. Historical evidence demonstrates that core concepts studied by political psychologists, such as partisanship and polarization, are changing across time (Abramowitz & Webster, Reference Abramowitz and Webster2016; Baldassarri & Gelman, Reference Baldassarri and Gelman2008; Bartels, Reference Bartels2000; Kozlowski & Murphy, Reference Kozlowski and Murphy2020). Furthermore, the stimuli used to study these phenomena often include specific policies or politicians who are well known at a given time and elicit a host of associations related to that period of time.
Temporal factors occurring “outside the lab” may also influence the interactions between moral and political psychology. For instance, “political elites” in the United States use more moral language when their party is not in power (Wang & Inbar, Reference Wang and Inbar2020), demonstrating the changing nature of moral language usage due to external (and typically unacknowledged) factors (see Brady, Wills, Burkart, Jost, & Van Bavel, Reference Brady, Wills, Burkart, Jost and Van Bavel2019). Similarly, moral language changes over time (cf. Wheeler, McGrath, & Haslam, Reference Wheeler, McGrath and Haslam2019). The question of temporal generalizability is a foundational question of the historical versus lawlike nature of psychological findings (Gergen, Reference Gergen1973). A critical next step in psychological research must be examining whether trends related to political or moral reasoning are robust across time.
It is also important to avoid the assumption that reactions to political and moral stimuli in one cultural context reveal insights into individuals in another cultural context. For example, claims about growing rates of political polarization are often made based on US samples, with participants embedded within a two-party political system (Finkel et al., Reference Finkel, Bail, Cikara, Ditto, IIyengar, Klar and Druckman2020). Recent evidence suggests that other countries show varying trajectories of political polarization likely based on context-specific characteristics of their political system (Boxell, Gentzkow, & Shapiro, Reference Boxell, Gentzkow and Shapiro2020). Future work should include international samples or else constrain their claims to reflect the current sample.
In moral psychology, foundational research on moral intuitions largely ignored the prospect of cultural variation. For example, dual-processing theories developed by studying trolley problem dilemmas exclusively used evidence from WEIRD [western, educated, industrialized, rich, and democratic] samples (Greene, Sommerville, Nystrom, Darley, & Cohen, Reference Greene, Sommerville, Nystrom, Darley and Cohen2001), yet recent cross-cultural work demonstrates that participants' intuitions about trolly problem dilemmas are notably different in eastern cultures (Rehman & Dzionek-Kozłowska, Reference Rehman and Dzionek-Kozłowska2020) and non-industrialized cultures (Sorokowski, Marczak, Misiak, & Białek, Reference Sorokowski, Marczak, Misiak and Białek2020). Moral and political psychologists should be aware of the type of evidence required to make claims of strong or weak universality (Norenzayan & Heine, Reference Norenzayan and Heine2005), and make explicit reference to cultural generalizability when drawing broad conclusions from a small number of studies or restricted samples (Bago et al., Reference Bago, Aczel, Kekecs, Protzko, Kovacs, Nagy and Dutra2021).
Moral and political psychology face major generalizability challenges with regard to the content of research materials, temporal variability, and cultural differences. To address these challenges, part of the solution involves methodological and analytic changes (target article by Yarkoni). While important, these changes alone cannot fully address questions of generalizability. Here we join many recent observers in calling for better theory development and employment of formal tools to make more precise theoretical claims (Guest & Martin, Reference Guest and Martin2020; Muthukrishna & Henrich, Reference Muthukrishna and Henrich2019; van Rooij & Baggio, Reference van Rooij and Baggio2021). Improved theory will not only help close the distance between theoretical claims and empirical tests, but guide and facilitate interdisciplinary research. The latter may be particularly important in moral and political psychology, as its subject matter clearly intersects with explanatory efforts in many adjacent disciplines.
One hope is that, with changes to our empirical and theoretical practices, future researchers will be better placed to assess the generalizability of research in moral and political psychology. One potential future for this work is for researchers to organize themselves not only in cross-disciplinary and cross-cultural consortia but also cross-temporally. This would mean planning for research projects to investigate fundamental questions over longer time periods that span entire grant cycles or even individual careers. Conducting research with more diverse samples of participants and stimuli, across social and political contexts, will provide an exciting foundation for the future of the field.
The aim of the social and behavioral sciences is to understand human behavior across a wide array of contexts. Our theories often make sweeping claims about human nature, assuming that our ancestors or offspring will be prone to the same biases and preferences. Yet we gloss over the fact that our research is often based on a single temporal context with a limited set of stimuli. Political and moral psychology are domains in which the context and stimuli are likely to matter a great deal (Brandt & Wagemans, Reference Brandt and Wagemans2017; Van Bavel et al., Reference Van Bavel, Mende-Siedlecki, Brady and Reinero2016). Understand differences between liberals and conservatives, when people are willing to sacrifice someone to save others, the behavior of political leaders, and the dynamics of intergroup conflict, all likely depend on the features of the research.
One of the major challenges of political psychology is understanding the generalizability of stimuli (see Yarkoni, this issue). Political psychologists often make broad claims about the differences (or lack thereof) between conservatives and liberals. However, the field's findings can vary widely depending on the specific stimuli used (e.g., topics, policies, etc.). Whether American Democrats or Republicans, for example, report higher belief superiority depends on the political issue researchers asked about (Toner, Leary, Asher, & Jongman-Sereno, Reference Toner, Leary, Asher and Jongman-Sereno2013). Scientists can therefore select different issues to show, alternatively, that people on the left or right of the political spectrum are more prone to a sense of superiority about their beliefs. It is only when looking across multiple topics that the overall quadratic relationship emerges (Harris & Van Bavel, Reference Harris and Van Bavel2021), suggesting the people on both ends of the spectrum are similarly prone to feelings of superiority when their attitudes are extreme.
This issue of non-generalizable stimuli extends to research on moral psychology as well. For instance, the victims of harm are often described using vague terminology, such as five nondescript workers on a track about to be run over by a trolley. In many studies, the victims are genderless, raceless, etc. (Hester & Gray, Reference Hester and Gray2020), which makes it difficult to generalize the findings to more vivid or real-world judgments (FeldmanHall et al., Reference FeldmanHall, Mobbs, Evans, Hiscox, Navrady and Dalgleish2012). Yet the research suggests that moral judgments are heavily influenced by contextual information and stimuli.
Another often-overlooked aspect of generalizability in both political and moral psychology is the changing temporal dynamics. Historical evidence demonstrates that core concepts studied by political psychologists, such as partisanship and polarization, are changing across time (Abramowitz & Webster, Reference Abramowitz and Webster2016; Baldassarri & Gelman, Reference Baldassarri and Gelman2008; Bartels, Reference Bartels2000; Kozlowski & Murphy, Reference Kozlowski and Murphy2020). Furthermore, the stimuli used to study these phenomena often include specific policies or politicians who are well known at a given time and elicit a host of associations related to that period of time.
Temporal factors occurring “outside the lab” may also influence the interactions between moral and political psychology. For instance, “political elites” in the United States use more moral language when their party is not in power (Wang & Inbar, Reference Wang and Inbar2020), demonstrating the changing nature of moral language usage due to external (and typically unacknowledged) factors (see Brady, Wills, Burkart, Jost, & Van Bavel, Reference Brady, Wills, Burkart, Jost and Van Bavel2019). Similarly, moral language changes over time (cf. Wheeler, McGrath, & Haslam, Reference Wheeler, McGrath and Haslam2019). The question of temporal generalizability is a foundational question of the historical versus lawlike nature of psychological findings (Gergen, Reference Gergen1973). A critical next step in psychological research must be examining whether trends related to political or moral reasoning are robust across time.
It is also important to avoid the assumption that reactions to political and moral stimuli in one cultural context reveal insights into individuals in another cultural context. For example, claims about growing rates of political polarization are often made based on US samples, with participants embedded within a two-party political system (Finkel et al., Reference Finkel, Bail, Cikara, Ditto, IIyengar, Klar and Druckman2020). Recent evidence suggests that other countries show varying trajectories of political polarization likely based on context-specific characteristics of their political system (Boxell, Gentzkow, & Shapiro, Reference Boxell, Gentzkow and Shapiro2020). Future work should include international samples or else constrain their claims to reflect the current sample.
In moral psychology, foundational research on moral intuitions largely ignored the prospect of cultural variation. For example, dual-processing theories developed by studying trolley problem dilemmas exclusively used evidence from WEIRD [western, educated, industrialized, rich, and democratic] samples (Greene, Sommerville, Nystrom, Darley, & Cohen, Reference Greene, Sommerville, Nystrom, Darley and Cohen2001), yet recent cross-cultural work demonstrates that participants' intuitions about trolly problem dilemmas are notably different in eastern cultures (Rehman & Dzionek-Kozłowska, Reference Rehman and Dzionek-Kozłowska2020) and non-industrialized cultures (Sorokowski, Marczak, Misiak, & Białek, Reference Sorokowski, Marczak, Misiak and Białek2020). Moral and political psychologists should be aware of the type of evidence required to make claims of strong or weak universality (Norenzayan & Heine, Reference Norenzayan and Heine2005), and make explicit reference to cultural generalizability when drawing broad conclusions from a small number of studies or restricted samples (Bago et al., Reference Bago, Aczel, Kekecs, Protzko, Kovacs, Nagy and Dutra2021).
Moral and political psychology face major generalizability challenges with regard to the content of research materials, temporal variability, and cultural differences. To address these challenges, part of the solution involves methodological and analytic changes (target article by Yarkoni). While important, these changes alone cannot fully address questions of generalizability. Here we join many recent observers in calling for better theory development and employment of formal tools to make more precise theoretical claims (Guest & Martin, Reference Guest and Martin2020; Muthukrishna & Henrich, Reference Muthukrishna and Henrich2019; van Rooij & Baggio, Reference van Rooij and Baggio2021). Improved theory will not only help close the distance between theoretical claims and empirical tests, but guide and facilitate interdisciplinary research. The latter may be particularly important in moral and political psychology, as its subject matter clearly intersects with explanatory efforts in many adjacent disciplines.
One hope is that, with changes to our empirical and theoretical practices, future researchers will be better placed to assess the generalizability of research in moral and political psychology. One potential future for this work is for researchers to organize themselves not only in cross-disciplinary and cross-cultural consortia but also cross-temporally. This would mean planning for research projects to investigate fundamental questions over longer time periods that span entire grant cycles or even individual careers. Conducting research with more diverse samples of participants and stimuli, across social and political contexts, will provide an exciting foundation for the future of the field.
Financial support
This publication was made possible through the support of: John Templeton Foundation (#61378; PP and JVB), Swedish Research Council (2016-06793; PP), and the Social Science and Humanities Research Council of Canada (752-2018-0213; EH).
Conflict of interest
None.