In their focal article, Bergman and Jean suggest the need to increase the number of peer-reviewed journal articles that study workers and propose steps to achieve this goal. In this commentary, we first discuss how “workers” may on average diverge from “professionals, managers, and executives” (hereafter, for parsimony, “professionals”Footnote
2
) on important situational difference variables. We then elaborate on Bergman and Jean's suggestions regarding what journals can do to facilitate the study of workers—albeit in our case with a particular emphasis on strategies aimed at understanding the underlying differences in situations experienced by workers versus professionals.
Situational Differences That May Distinguish Workers From Professionals
In this section, we describe a few situational variables on which workers, on average, are likely to score differently from professionals. One obstacle to doing so is that neither I-O psychologists nor other psychologists (e.g., social and personality psychologists) have yet developed a thorough understanding of the structure of situations (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014). Therefore, we identify situational variables that not only seem likely to differ across worker and professional samples but that have also been identified as important situational variables, per se, either within I-O psychology or beyond.
The first such situational difference is socioeconomic status (SES). This construct is not studied widely in I-O psychology but is studied by sociologists, typically at the group or societal level, and is beginning to be studied with regard to its influence on individuals’ psychological states such as empathy (e.g., Kraus & Stephens, Reference Kraus and Stephens2012). SES is defined as an individual's current social and economic states (Rubin et al., Reference Rubin, Denson, Kilpatrick, Matthews, Stehlik and Zyngier2014) and includes objective measures of individual education, income, and occupational rank (Kraus & Stephens, Reference Kraus and Stephens2012). As can be seen from this definition, SES encompasses current pay, which has been an important topic of study within I-O psychology (Rynes, Gerhart, & Minette, Reference Rynes, Gerhart and Minette2004). Workers are expected to receive lower pay, and more generally to be of lower SES, than professionals.
When studying worker samples and their differences in SES compared with professional samples, new outcome variables may increase in relevance. Bergman and Jean suggest several important outcome variables (economic tenuousness, homelessness, bankruptcy, and second jobs), but the SES literature (e.g., Twenge & Campbell, Reference Twenge and Campbell2002; Wang & Beydoun, Reference Wang and Beydoun2007) suggests several additional outcomes of importance (e.g., subjective well-being, depression, self-esteem, and obesity). Moreover, the societal analogs of SES (using a dispersion composition model; Chan, Reference Chan1998) are income and wealth inequality—and these in turn are associated with outcomes such as lower social cohesion and population health (Kawachi & Kennedy, Reference Kawachi and Kennedy1997; Wilkinson & Pickett, Reference Wilkinson and Pickett2006). By neglecting worker samples, I-O psychology neglects these important individual and societal level outcomes. At a policy level, I-O psychology is therefore absent from important debates on issues such as the minimum wage. We have ceded the field to economists.
The second situational difference we emphasize is job complexity, which is frequently conceptualized in terms of the scope and information processing demands of the job (Xie & Johns, Reference Xie and Johns1995). The extent to which situations are complex is likely to be associated strongly with the extent to which situations require cognition and provide the opportunity to engage the intellect—that is, according to an influential new situational taxonomy developed by social and personality psychologists (the Situational Eight DIAMONDS; e.g., Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014; Rauthmann & Sherman, Reference Rauthmann and Sherman2015), the extent to which situations score high on the dimension of “intellect.” Complexity is likely to be lower in worker jobs than in professional jobs—an issue also alluded to briefly by Bergman and Jean in their focal article.
The third situational difference we emphasize is situational strength, which has a venerable history in the field of personality psychology (Mischel, Reference Mischel1973). Situational strength is defined as “implicit or explicit cues, provided by entities external to the individual, regarding the desirability of various forms of behavior” (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014, p. 1011). It has been referred to as “the most important situational moderating variable” (Snyder & Ickes, Reference Snyder, Ickes, Lindzey and Aronson1985, p. 904). Many aspects of situational strength (e.g., clarity of information received, consistency of information received, and constraints on decision and action) are likely to be higher in worker jobs than in professional jobs.
Why do job complexity and situational strength matter? As mentioned by Bergman and Jean, job complexity enhances intelligence–performance relationships. Situational strength, on the other hand, generally attenuates personality–performance relationships (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014). Therefore, intelligence and personality are less predictive of performance in strong and simple (i.e., worker) jobs than in weak and complex (i.e., professional) jobs. Moreover, both complexity and strength are likely to vary within jobs over time (Dalal, Bhave, & Fiset, Reference Dalal, Bhave and Fiset2014). However, we suspect that this is less likely to occur for worker jobs than for professional jobs because the tasks encountered by workers are probably fewer and more similar over time than those experienced by professionals. Therefore, the extent to which intelligence and personality are predictive of performance is likely to differ less across tasks within worker jobs than professional jobs.
Before ending this section, however, we would be remiss in noting that the lack of unanimity regarding situational taxonomies means that it is not possible, at the current stage, to articulate a comprehensive list of situational differences across worker versus professional samples. One option for researchers, therefore, would be to start with an extant taxonomy of situations (e.g., the aforementioned DIAMONDS taxonomy; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014) and to then theorize and empirically test for differences between worker and professional samples along these situational dimensions.
What Journals Can Do
In their focal article, Bergman and Jean proposed some ways to increase the number of peer-reviewed journal articles that study “workers.” We suggest several additional steps to achieve that goal, with a particular emphasis on strategies aimed at uncovering and ultimately better understanding the underlying differences in situations experienced by workers versus professionals. Our contention is that, if increasing our field's understanding of workers is a worthwhile goal, it is important to move beyond reliance on the goodwill of individual researchers who are amenable to studying workers when it is not too difficult. Instead, as Pfeffer (Reference Pfeffer1993) has so effectively pointed out, journal editors and reviewers would need to play an important “gatekeeping” role in this regard by indicating that research solely on workers, as well as research comparing workers and professional employees, is valued. The role of journal editors is also suggested by research on culture change (Hambrick & Mason, Reference Hambrick and Mason1984) if we make the additional assumption that journal editors represent “upper management” in the field.
Research Design Criteria in the Peer Review Process
When reviewing a manuscript that has been submitted to a journal, reviewers assess the manuscript on several criteria, including criteria related to research design. Reviewers are usually asked to provide a numerical rating regarding the strength of the manuscript's research design. In addition, reviewers provide narrative feedback regarding research design to the action editor and manuscript authors. Some of the common issues raised within the topic of research design include internal validity (e.g., limited basis for causal conclusions from cross-sectional studies) or external validity as it pertains to the sample (e.g., overall low response rate, low base rate for a not widely manifested phenomenon, the necessity for a broader sample; Green, Tonidandel, & Cortina, in press; Rogelberg, Adelman, & Askay, Reference Rogelberg, Adelman and Askay2009).
Researchers have emphasized the potential of the (low) external validity of the findings when the sample consists of student samples (Nadler et al., Reference Nadler, Bartels, Naumann, Morr, Locke, Beurskens and Ginder2015). However, reviewers typically do not (but should) comment on external validity when the sample is composed solely of professionals. In some cases, the findings of a study might be expected to differ as a function of the sample, either because of differences in the job settings associated with worker versus professional settings (e.g., the situational variables mentioned previously) or because of individual differences in worker versus professional samples or both. In such cases, reviewers ought to view external validity as low. Of course, discussions of external validity as a function of samples and associated settings are only possible if samples and settings are adequately described (reported) in journal submissions, an issue we turn to next.
Reporting Samples (and Associated Settings) in Primary Studies
Journals could require authors to ask research participants about their job titles and SES. Job title can be elicited via an open-ended question. Alternately, job category can be elicited via the categories provided by the U.S. Equal Employment Opportunity Commission (2011; e.g., executive, technician, sales, service worker). SES can be elicited via questions on income and education level (see Kraus, Piff, & Keltner, Reference Kraus, Piff and Keltner2009). Journals could likewise require authors to include measures of job complexity and situational strength. Authors would then report summary data on these variables as well as relationships (e.g., via a correlation matrix) with focal variables.
We must, however, acknowledge an important objection to these suggestions: Authors are often under pressure to keep survey length short, and requiring authors to include measures of these situational factors would increase the length of their surveys. We therefore suggest two possibilities for cases when authors believe that these situational factors are not the focus of their research. First, authors could include ultra-short measures of situational factors. In some cases, ultra-short measures already exist. For instance, SES can be measured via two items: one each on income and education level (see Kraus et al., Reference Kraus, Piff and Keltner2009). The extent to which situations provide the opportunity to engage the intellect can, if needed, be measured via a single item (see Rauthmann & Sherman, Reference Rauthmann and Sherman2015). In other cases, ultra-short measures can readily be constructed. For instance, the item quality scores presented in Table 1 of Meyer et al. (Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014) can be used to generate single-item measures of each of the four facets of situational strength. Second, if authors have collected job titles but not situational factors (e.g., strength, complexity) as part of their survey, they could subsequently obtain data on these situational factors through subject-matter experts.
Replication and Meta-Analyses
In their focal article, Bergman and Jean suggest that journals should encourage replication with worker samples. We agree, but we suggest that it would be more fruitful to compare extant research involving professional samples with new research that simultaneously examines both professional and worker samples, with other research design factors (e.g., measures and procedures) kept as similar as possible across the original and new studies and across the professional and worker samples in the new study.
Within this overall rubric, we emphasize two types of replication studies. First, consider those cases where there is reason to believe that the original findings may not generalize to worker samples, perhaps because of the potential for sample-based boundary conditions to the theoretical framework. In such cases, convincing evidence of a lack of generalizability of findings across samples would result from the original findings being replicated with a similar (i.e., professional) sample but not with a dissimilar (i.e., worker) sample. Second, consider those cases where there is reason to believe that the original findings may not replicate regardless of the sample used in the replication, perhaps because the original research involved hypotheses with a low probability of replication (e.g., extended mediation chains, four-way interactions, moderated mediation and mediated moderation designs). In such cases, convincing evidence of a lack of replicability would involve a failure to replicate the original findings in either the similar (i.e., professional) or the dissimilar (i.e., worker) sample.
Both types of studies would ideally also include measures of situational variables such as SES, situational strength, and job complexity. Both types of studies could, moreover, be conducted as a prelude to “prospective meta-analyses,” in which teams of researchers collectively design and conduct studies with the goal of including them in a meta-analysis (Ghersi, Berlin, & Askie, Reference Ghersi, Berlin, Askie, Higgins and Green2011). Prospective meta-analyses would allow researchers to coordinate their methods and the situational variables they measure as potential moderators. Prospective meta-analyses such as these would reduce the limitations associated with traditional meta-analyses (e.g., only a small number of primary studies using worker samples, only a small number of primary studies containing measures of situational variables believed to underlie worker versus professional sample differences). Journals could facilitate progress with long-term plans that involve first hosting special issues devoted to the two types of replication studies discussed previously and subsequently welcoming prospective meta-analyses on these topics.
There remains, nonetheless, an important role for traditional meta-analyses. We suggest that journals encourage meta-analyses to examine situational moderators. Even in cases where situational variables were not measured directly (or were not reported) in primary studies, it may be possible for meta-analysts to measure them as study-level (as opposed to employee level) moderators in primary studies that involved single jobs (e.g., hair stylists) or that involved multiple jobs in conjunction with a frequency breakdown of job titles (e.g., 60% hair stylists and 40% professors). This is because, as alluded to previously, meta-analysts can rely on subject-matter expert ratings of situational factors based on the job titles. Alternatively, meta-analysts could match the job titles reported in primary studies with those in O*NET (the Occupational Information Network; e.g., Peterson et al., Reference Peterson, Mumford, Borman, Jeanneret, Fleishman, Levin and Dye2001). O*NET reports average salary levels for each job. Moreover, scales measuring situational differences have been developed based on the information available on O*NET; see, for instance, Glomb, Kammeyer-Mueller, and Rotundo (Reference Glomb, Kammeyer-Mueller and Rotundo2004) for job complexity (specifically, cognitive demands associated with a job) and Meyer, Dalal, and Bonaccio (Reference Meyer, Dalal and Bonaccio2009) for job-level situational strength. O*NET-based scales could also be developed to measure additional situational constructs (e.g., the DIAMONDS dimensions; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014).
Journals could then ask meta-analysts to generate study-level scores for situational characteristics in cases where there is reason to believe that the original findings may not generalize at different levels of situational strength, job complexity, SES, and so forth. For instance, Meyer et al. (Reference Meyer, Dalal and Bonaccio2009) demonstrated meta-analytically that the conscientiousness–performance relationship was stronger for jobs low in situational strength than for jobs high in situational strength. Meyer et al. (Reference Meyer, Dalal and Bonaccio2009) did not further attempt to connect differences in job-level situational strength to differences in worker versus professional jobs, but journals could ask future meta-analysts to do so.
Bergman and Jean's (Reference Bergman and Jean2016) focal article decries the limited research attention of industrial and organizational (I-O) psychologists on “workers”—that is, employees such as wage earners, frontline workers, and contractors, who do not fill professional, managerial, or executive positions. We agree. In addition to the scientific and moral benefits of studying workers, there is a practical imperative. An academic discipline that comes across as being disinterested in workers may leave itself open to charges of being the “handmaiden” of management (Hulin, Reference Hulin, Brett and Drasgow2002, p. 12). Moreover, such an academic discipline may be ill prepared to provide evidence-based contributions to important societal debates on topics such as income inequality and immigration.Footnote 1
In their focal article, Bergman and Jean suggest the need to increase the number of peer-reviewed journal articles that study workers and propose steps to achieve this goal. In this commentary, we first discuss how “workers” may on average diverge from “professionals, managers, and executives” (hereafter, for parsimony, “professionals”Footnote 2 ) on important situational difference variables. We then elaborate on Bergman and Jean's suggestions regarding what journals can do to facilitate the study of workers—albeit in our case with a particular emphasis on strategies aimed at understanding the underlying differences in situations experienced by workers versus professionals.
Situational Differences That May Distinguish Workers From Professionals
In this section, we describe a few situational variables on which workers, on average, are likely to score differently from professionals. One obstacle to doing so is that neither I-O psychologists nor other psychologists (e.g., social and personality psychologists) have yet developed a thorough understanding of the structure of situations (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014). Therefore, we identify situational variables that not only seem likely to differ across worker and professional samples but that have also been identified as important situational variables, per se, either within I-O psychology or beyond.
The first such situational difference is socioeconomic status (SES). This construct is not studied widely in I-O psychology but is studied by sociologists, typically at the group or societal level, and is beginning to be studied with regard to its influence on individuals’ psychological states such as empathy (e.g., Kraus & Stephens, Reference Kraus and Stephens2012). SES is defined as an individual's current social and economic states (Rubin et al., Reference Rubin, Denson, Kilpatrick, Matthews, Stehlik and Zyngier2014) and includes objective measures of individual education, income, and occupational rank (Kraus & Stephens, Reference Kraus and Stephens2012). As can be seen from this definition, SES encompasses current pay, which has been an important topic of study within I-O psychology (Rynes, Gerhart, & Minette, Reference Rynes, Gerhart and Minette2004). Workers are expected to receive lower pay, and more generally to be of lower SES, than professionals.
When studying worker samples and their differences in SES compared with professional samples, new outcome variables may increase in relevance. Bergman and Jean suggest several important outcome variables (economic tenuousness, homelessness, bankruptcy, and second jobs), but the SES literature (e.g., Twenge & Campbell, Reference Twenge and Campbell2002; Wang & Beydoun, Reference Wang and Beydoun2007) suggests several additional outcomes of importance (e.g., subjective well-being, depression, self-esteem, and obesity). Moreover, the societal analogs of SES (using a dispersion composition model; Chan, Reference Chan1998) are income and wealth inequality—and these in turn are associated with outcomes such as lower social cohesion and population health (Kawachi & Kennedy, Reference Kawachi and Kennedy1997; Wilkinson & Pickett, Reference Wilkinson and Pickett2006). By neglecting worker samples, I-O psychology neglects these important individual and societal level outcomes. At a policy level, I-O psychology is therefore absent from important debates on issues such as the minimum wage. We have ceded the field to economists.
The second situational difference we emphasize is job complexity, which is frequently conceptualized in terms of the scope and information processing demands of the job (Xie & Johns, Reference Xie and Johns1995). The extent to which situations are complex is likely to be associated strongly with the extent to which situations require cognition and provide the opportunity to engage the intellect—that is, according to an influential new situational taxonomy developed by social and personality psychologists (the Situational Eight DIAMONDS; e.g., Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014; Rauthmann & Sherman, Reference Rauthmann and Sherman2015), the extent to which situations score high on the dimension of “intellect.” Complexity is likely to be lower in worker jobs than in professional jobs—an issue also alluded to briefly by Bergman and Jean in their focal article.
The third situational difference we emphasize is situational strength, which has a venerable history in the field of personality psychology (Mischel, Reference Mischel1973). Situational strength is defined as “implicit or explicit cues, provided by entities external to the individual, regarding the desirability of various forms of behavior” (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014, p. 1011). It has been referred to as “the most important situational moderating variable” (Snyder & Ickes, Reference Snyder, Ickes, Lindzey and Aronson1985, p. 904). Many aspects of situational strength (e.g., clarity of information received, consistency of information received, and constraints on decision and action) are likely to be higher in worker jobs than in professional jobs.
Why do job complexity and situational strength matter? As mentioned by Bergman and Jean, job complexity enhances intelligence–performance relationships. Situational strength, on the other hand, generally attenuates personality–performance relationships (Meyer et al., Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014). Therefore, intelligence and personality are less predictive of performance in strong and simple (i.e., worker) jobs than in weak and complex (i.e., professional) jobs. Moreover, both complexity and strength are likely to vary within jobs over time (Dalal, Bhave, & Fiset, Reference Dalal, Bhave and Fiset2014). However, we suspect that this is less likely to occur for worker jobs than for professional jobs because the tasks encountered by workers are probably fewer and more similar over time than those experienced by professionals. Therefore, the extent to which intelligence and personality are predictive of performance is likely to differ less across tasks within worker jobs than professional jobs.
Before ending this section, however, we would be remiss in noting that the lack of unanimity regarding situational taxonomies means that it is not possible, at the current stage, to articulate a comprehensive list of situational differences across worker versus professional samples. One option for researchers, therefore, would be to start with an extant taxonomy of situations (e.g., the aforementioned DIAMONDS taxonomy; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014) and to then theorize and empirically test for differences between worker and professional samples along these situational dimensions.
What Journals Can Do
In their focal article, Bergman and Jean proposed some ways to increase the number of peer-reviewed journal articles that study “workers.” We suggest several additional steps to achieve that goal, with a particular emphasis on strategies aimed at uncovering and ultimately better understanding the underlying differences in situations experienced by workers versus professionals. Our contention is that, if increasing our field's understanding of workers is a worthwhile goal, it is important to move beyond reliance on the goodwill of individual researchers who are amenable to studying workers when it is not too difficult. Instead, as Pfeffer (Reference Pfeffer1993) has so effectively pointed out, journal editors and reviewers would need to play an important “gatekeeping” role in this regard by indicating that research solely on workers, as well as research comparing workers and professional employees, is valued. The role of journal editors is also suggested by research on culture change (Hambrick & Mason, Reference Hambrick and Mason1984) if we make the additional assumption that journal editors represent “upper management” in the field.
Research Design Criteria in the Peer Review Process
When reviewing a manuscript that has been submitted to a journal, reviewers assess the manuscript on several criteria, including criteria related to research design. Reviewers are usually asked to provide a numerical rating regarding the strength of the manuscript's research design. In addition, reviewers provide narrative feedback regarding research design to the action editor and manuscript authors. Some of the common issues raised within the topic of research design include internal validity (e.g., limited basis for causal conclusions from cross-sectional studies) or external validity as it pertains to the sample (e.g., overall low response rate, low base rate for a not widely manifested phenomenon, the necessity for a broader sample; Green, Tonidandel, & Cortina, in press; Rogelberg, Adelman, & Askay, Reference Rogelberg, Adelman and Askay2009).
Researchers have emphasized the potential of the (low) external validity of the findings when the sample consists of student samples (Nadler et al., Reference Nadler, Bartels, Naumann, Morr, Locke, Beurskens and Ginder2015). However, reviewers typically do not (but should) comment on external validity when the sample is composed solely of professionals. In some cases, the findings of a study might be expected to differ as a function of the sample, either because of differences in the job settings associated with worker versus professional settings (e.g., the situational variables mentioned previously) or because of individual differences in worker versus professional samples or both. In such cases, reviewers ought to view external validity as low. Of course, discussions of external validity as a function of samples and associated settings are only possible if samples and settings are adequately described (reported) in journal submissions, an issue we turn to next.
Reporting Samples (and Associated Settings) in Primary Studies
Journals could require authors to ask research participants about their job titles and SES. Job title can be elicited via an open-ended question. Alternately, job category can be elicited via the categories provided by the U.S. Equal Employment Opportunity Commission (2011; e.g., executive, technician, sales, service worker). SES can be elicited via questions on income and education level (see Kraus, Piff, & Keltner, Reference Kraus, Piff and Keltner2009). Journals could likewise require authors to include measures of job complexity and situational strength. Authors would then report summary data on these variables as well as relationships (e.g., via a correlation matrix) with focal variables.
We must, however, acknowledge an important objection to these suggestions: Authors are often under pressure to keep survey length short, and requiring authors to include measures of these situational factors would increase the length of their surveys. We therefore suggest two possibilities for cases when authors believe that these situational factors are not the focus of their research. First, authors could include ultra-short measures of situational factors. In some cases, ultra-short measures already exist. For instance, SES can be measured via two items: one each on income and education level (see Kraus et al., Reference Kraus, Piff and Keltner2009). The extent to which situations provide the opportunity to engage the intellect can, if needed, be measured via a single item (see Rauthmann & Sherman, Reference Rauthmann and Sherman2015). In other cases, ultra-short measures can readily be constructed. For instance, the item quality scores presented in Table 1 of Meyer et al. (Reference Meyer, Dalal, José, Hermida, Chen, Vega and Khare2014) can be used to generate single-item measures of each of the four facets of situational strength. Second, if authors have collected job titles but not situational factors (e.g., strength, complexity) as part of their survey, they could subsequently obtain data on these situational factors through subject-matter experts.
Replication and Meta-Analyses
In their focal article, Bergman and Jean suggest that journals should encourage replication with worker samples. We agree, but we suggest that it would be more fruitful to compare extant research involving professional samples with new research that simultaneously examines both professional and worker samples, with other research design factors (e.g., measures and procedures) kept as similar as possible across the original and new studies and across the professional and worker samples in the new study.
Within this overall rubric, we emphasize two types of replication studies. First, consider those cases where there is reason to believe that the original findings may not generalize to worker samples, perhaps because of the potential for sample-based boundary conditions to the theoretical framework. In such cases, convincing evidence of a lack of generalizability of findings across samples would result from the original findings being replicated with a similar (i.e., professional) sample but not with a dissimilar (i.e., worker) sample. Second, consider those cases where there is reason to believe that the original findings may not replicate regardless of the sample used in the replication, perhaps because the original research involved hypotheses with a low probability of replication (e.g., extended mediation chains, four-way interactions, moderated mediation and mediated moderation designs). In such cases, convincing evidence of a lack of replicability would involve a failure to replicate the original findings in either the similar (i.e., professional) or the dissimilar (i.e., worker) sample.
Both types of studies would ideally also include measures of situational variables such as SES, situational strength, and job complexity. Both types of studies could, moreover, be conducted as a prelude to “prospective meta-analyses,” in which teams of researchers collectively design and conduct studies with the goal of including them in a meta-analysis (Ghersi, Berlin, & Askie, Reference Ghersi, Berlin, Askie, Higgins and Green2011). Prospective meta-analyses would allow researchers to coordinate their methods and the situational variables they measure as potential moderators. Prospective meta-analyses such as these would reduce the limitations associated with traditional meta-analyses (e.g., only a small number of primary studies using worker samples, only a small number of primary studies containing measures of situational variables believed to underlie worker versus professional sample differences). Journals could facilitate progress with long-term plans that involve first hosting special issues devoted to the two types of replication studies discussed previously and subsequently welcoming prospective meta-analyses on these topics.
There remains, nonetheless, an important role for traditional meta-analyses. We suggest that journals encourage meta-analyses to examine situational moderators. Even in cases where situational variables were not measured directly (or were not reported) in primary studies, it may be possible for meta-analysts to measure them as study-level (as opposed to employee level) moderators in primary studies that involved single jobs (e.g., hair stylists) or that involved multiple jobs in conjunction with a frequency breakdown of job titles (e.g., 60% hair stylists and 40% professors). This is because, as alluded to previously, meta-analysts can rely on subject-matter expert ratings of situational factors based on the job titles. Alternatively, meta-analysts could match the job titles reported in primary studies with those in O*NET (the Occupational Information Network; e.g., Peterson et al., Reference Peterson, Mumford, Borman, Jeanneret, Fleishman, Levin and Dye2001). O*NET reports average salary levels for each job. Moreover, scales measuring situational differences have been developed based on the information available on O*NET; see, for instance, Glomb, Kammeyer-Mueller, and Rotundo (Reference Glomb, Kammeyer-Mueller and Rotundo2004) for job complexity (specifically, cognitive demands associated with a job) and Meyer, Dalal, and Bonaccio (Reference Meyer, Dalal and Bonaccio2009) for job-level situational strength. O*NET-based scales could also be developed to measure additional situational constructs (e.g., the DIAMONDS dimensions; Rauthmann et al., Reference Rauthmann, Gallardo-Pujol, Guillaume, Todd, Nave, Sherman and Funder2014).
Journals could then ask meta-analysts to generate study-level scores for situational characteristics in cases where there is reason to believe that the original findings may not generalize at different levels of situational strength, job complexity, SES, and so forth. For instance, Meyer et al. (Reference Meyer, Dalal and Bonaccio2009) demonstrated meta-analytically that the conscientiousness–performance relationship was stronger for jobs low in situational strength than for jobs high in situational strength. Meyer et al. (Reference Meyer, Dalal and Bonaccio2009) did not further attempt to connect differences in job-level situational strength to differences in worker versus professional jobs, but journals could ask future meta-analysts to do so.
Conclusion
We have elaborated on Bergman and Jean's recommendations for journals to increase the representation of workers in the I-O psychology literature. Our recommendations focus on the underlying situational differences that may distinguish worker samples from professional ones (i.e., SES, job complexity, and situational strength). Over time, such practices should result in the development of a wealth of information about differences between workers and professionals in terms of both individual differences and situational (setting) differences. Ideally, a future review similar to Bergman and Jean's, but conducted, say, a decade from now, will not still find that only 9% of samples focus specifically on workers.