Bergman and Jean (Reference Bergman and Jean2016) suggest that an underrepresentation of nonmanagerial workers in industrial and organizational (I-O) research is hindering our contribution to practice and ultimately our legitimacy as a discipline. I agree with the target article authors’ overarching message insomuch as it serves as a reminder that we, as I-O psychologists, have responsibilities to understand and improve the work experience of a diverse range of populations. However, I also believe the authors’ claim that nonmanagerial workers are underrepresented may be overstated. Moreover, I offer a perspective suggesting that the current supply of research being produced in the name of I-O psychology is driven by real world demand, that this naturally occurring phenomenon is what maximizes our impact on practice, and that ignoring this phenomenon is actually what hinders the legitimacy of applied disciplines.
A Case for an Overstatement of the Problem
A call for change to a highly rigorous scientific field is no time to ignore scientific rigor. I have a number of concerns with regard to the manner in which the data presented in the target article were collected and interpreted. These are particularly critical issues, given that the data presented serve as the foundation for the subsequent arguments and call for change made by Bergman and Jean.
To begin, I appreciate the burden associated with conducting a more extensive literature search, but I do not believe that the journals selected to establish the rates with which I-O research has used managers versus nonmanagers is necessarily representative of the broader I-O literature. The target article authors chose five journals they purport to be the top I-O journals. Four of these also ranked among the top 10 management journals this past year (Thomson Reuters, 2015): Academy of Management Journal (no. 3), Journal of Management (no. 4), Journal of Applied Psychology (no. 6), and Personnel Psychology (no. 8). There is no doubt that I-O psychology and management are highly interconnected fields, but management is not the only field with which I-O shares significant commonalities. Characterizing I-O research, as a whole, only through journals that are as much management as I-O may overestimate the attention we have given to managers. Moreover, this characterization may ignore important I-O subdisciplines that give greater attention to nonmanagers but that may not feature as prominently in management-crossover journals.
To illustrate, I selected four highly ranked I-O journals (as rated by Thomson Reuters in the 2015 “Applied Psychology” category) that did not appear among the top 25 management journals: Journal of Vocational Behavior (no. 8), Journal of Occupational Health Psychology (no. 10), Work and Stress (no. 11), and Organizational Behavior and Human Decision Making (no. 13). In each of these, I coded the samples reported in the year 2014 using the same categories as the target article.Footnote
1
The total number of studies published was N = 204, resulting in a total of 304 samples. Although there were some similarities in the rates of sampling in management-centric and applied psychology-centric journals, there were also some notable differences. For example, Bergman and Jean found that nonworker samples (22%) were over twice as likely to be used as worker samples (9%). However, among the applied psychology journals I coded, these types of samples were used at the same rate (k = 27 for both; 9%). In addition, student samples were more prevalent in the applied psychology journals I coded (k = 125; 41%). Although the target article suggests that these individuals are likely to become managers, professionals, and executives, a greater majority of these individuals likely met the criteria for “worker” at the time they were studied (e.g., low to medium skill, wage earners, nonmanagers).
That said, for both sets of journals, the most commonly assessed type of sample consisted of a mix of workers and nonworkers. These “mixed” samples accounted for 42% and 41% of all data reported, which is greater than the rates for worker and nonworker samples combined. The target article described two general types of samples fitting this description: (a) those sampled from specific organizations and (b) those sampled from the broader labor population. I further distinguished between these subgroups in my coding procedures. I found 14% (k = 42) of the total samples were gathered from specific organizations. With the popularity of hierarchical linear modeling and the growing necessity to account for the nested nature of organizational data, there is surely a substantial portion of data within this subgroup of samples that speaks to how constructs and their relationships differ (or don't) between worker and nonworker populations. More important, though, I found 27% (k = 83) of the total samples came from individuals recruited from the broader labor population. The vast majority of these samples were obtained through survey websites (e.g., Amazon Mechanical Turk [MTurk]) with U.S.$0.50–$1.00 being compensated for participation. To paraphrase an anonymous journal reviewer's feedback on one of my own studies using an MTurk sample to study organizational phenomena, “What gainfully employed person would spend 20–30 minutes to complete a survey for $1.50?” The answer is likely not those in managerial or professional positions but possibly at best those working low-skill and/or part-time, wage-earning jobs. In other words, those the target article describes as “workers.”
I agree that it would be ideal to know more about different working populations based on any number of characteristics, professional status being just one of them. Given the issues highlighted above, however, I am not sure concluding that workers are highly underrepresented is necessarily accurate. At best, I would conclude that the perspective taken in the target article is exemplary of deeper issues that have been raised in previous volumes of this journal regarding the nature of I-O psychology's connection to the field of management (e.g., Aguinis, Bradley, & Brodersen, Reference Aguinis, Bradley and Brodersen2014).
I-O Psychology Is a Free Market Economy
One of the major concerns presented by Bergman and Jean was that the underrepresentation of nonmanager populations in I-O research “could minimize the impact the I-O discipline has on psychological sciences as a whole, organizations, and society, relegating I-O psychology to the fringe” (p. 91). Let us assume that a disproportion in the study of manager and nonmanager work populations exists in I-O, as suggested in the target article. Does this necessarily indicate dysfunction or that our discipline is at risk of losing whatever spot we currently hold at the scientific table? I believe the answer is no.
As an applied science, I-O psychology seeks to provide solutions to real world problems. In this way, the I-O community is naturally capitalistic. The topics published on (and not published on) during a given period are guided by the perceived demand from the working world. This phenomenon is analogous to the rise and fall of stock prices. Researchers invest their time and resources in solving the most novel and salient real world problems. Similarly, editors “buy up” manuscripts on topics that they view as gaining momentum and “sell off” manuscripts on topics for which they perceive demand to be weak or waning. A salient example is the increase in research on unemployment, underemployment, and downsizing survival in the wake of the Great Recession. I believe that, had the literature search presented in the target article been extended a few years prior to 2012, even the journals included for analysis would have reflected this discipline-wide trend (e.g., Creed, King, Hood, & McKenzie, Reference Creed, King, Hood and McKenzie2009; Song, Foo, Uy, & Sun, Reference Song, Foo, Uy and Sun2011; van Hooft & Noordzij, Reference van Hooft and Noordzij2009).
We must not forget that applied disciplines such as ours were born out of the ability to identify and meet salient real world demand (Roethlisberger & Dickson, Reference Roethlisberger and Dickson1939; Taylor, Reference Taylor1911), and we will be sustained through our ability to continue to do so. The authors of the target article suggest that this will require a major change in “who” we study, suggesting that a demand to do so exists. This may be true to a greater or lesser extent, but it is important to keep in mind that real world demand for practical solutions exists for a wide range of problems from a similar range of stakeholders. Being limited by size and resources, I-O psychology must effectively answer the questions of “where?” and “how much?” when determining how to allocate our attention in a way that maximizes our practical contribution. I believe that the capitalistic system that is in place will naturally drive these decisions.
I-O Psychology Is a Trading Post for Interdisciplinary Research
Let us again assume that managers, professionals, and executives are being oversampled within the I-O psychology literature. From a perspective of practical contribution, this would be a truly enviable problem, as managers and executives have the greatest influence in organizations. Thus, the better we understand their behavior and the environment in which they function, the better positioned we will be to improve the way work gets done and the work experience. Unfortunately, we all know how difficult it is to actually obtain data from such sources. Of course, this reality runs counter to the target article authors’ presumption that I-O is risking the othering, discounting, or pathologizing of nonmanager samples through the practice of oversampling manager samples.
In relation to this, I take issue with the target article's broader assertion that “When I-O psychology fails to study the full range of the labor market, it is likely that the field is not fully articulating the nomological network surrounding a construct of interest” (Bergman & Jean, p. 91). I do so because, in presenting this argument, the authors present a narrow characterization of I-O psychology. First, this characterization fails to give credence to the fact that a great amount of what we, as I-O psychologists, know about people in the work context comes from research conducted in other disciplines (e.g., clinical, cognitive, personality, and social psychology), wherein the typical study participant is much more likely to be a part-time wage earner in the service industry than a full-time salaried manager. Second, this characterization fails to acknowledge the diverse practical and scientific work in which I-O psychologists are involved. For example, I-O-trained academicians commonly work and collaborate with others in departments outside of psychology, and those who do call a psychology department home likely do not teach only I-O courses. Moreover, it is often a matter of opinion whether a particular journal is an “I-O” journal, but I-O psychologists publish and consume research in a vast array of journals (a subset of which has produced an impressive body of evidence from nonmanager samples). The eclectic space in which I-O psychologists are allowed to function is, I believe, one of the advantages of our discipline, and it is this collection of diverse perspectives that should quash concerns that we are developing too narrow of a view on workplace phenomena.
Concluding Remarks
I agree with the authors of the target article that the intentions of I-O psychologists should be to understand the attitudes, behaviors, and cognitions of individuals at all levels of the organization, not to mention in a diverse range of occupations. From a practical standpoint, there simply may not be enough journal outlets that are explicitly “I-O” to provide the attention that each of these populations deserves (I know what you are thinking here, but many lament that there are already too many of these journals of which to keep track). However, when one considers the broader social science literature that I-O psychologists both contribute to and consume, it is likely that we know far more about people who work in a diverse range of occupations than credence is given to in the target article. Because managers and executives have the greatest impact on workplace behavior and because researchers in other social science disciplines generally do not study these populations with great frequency, it makes economic sense for I-O psychologists to commit a significant amount of our limited resources to studying them in order to meet a practical demand. That said, the data presented in the target article do suggest that only 22% of the research published in the journals analyzed was actually conducted among the highly influential individuals at the upper organizational levels. This is a far cry from the type of monopoly on manager samples that would likely be needed in order to lead I-O psychologists to develop a skewed understanding of workplace behavior. In the end, I believe our legitimacy as a discipline is strengthened by the fact that our training and work has positioned us to meet an eclectic range of real world demands relating to the world of work.
You have
Access
Bergman and Jean (Reference Bergman and Jean2016) suggest that an underrepresentation of nonmanagerial workers in industrial and organizational (I-O) research is hindering our contribution to practice and ultimately our legitimacy as a discipline. I agree with the target article authors’ overarching message insomuch as it serves as a reminder that we, as I-O psychologists, have responsibilities to understand and improve the work experience of a diverse range of populations. However, I also believe the authors’ claim that nonmanagerial workers are underrepresented may be overstated. Moreover, I offer a perspective suggesting that the current supply of research being produced in the name of I-O psychology is driven by real world demand, that this naturally occurring phenomenon is what maximizes our impact on practice, and that ignoring this phenomenon is actually what hinders the legitimacy of applied disciplines.
A Case for an Overstatement of the Problem
A call for change to a highly rigorous scientific field is no time to ignore scientific rigor. I have a number of concerns with regard to the manner in which the data presented in the target article were collected and interpreted. These are particularly critical issues, given that the data presented serve as the foundation for the subsequent arguments and call for change made by Bergman and Jean.
To begin, I appreciate the burden associated with conducting a more extensive literature search, but I do not believe that the journals selected to establish the rates with which I-O research has used managers versus nonmanagers is necessarily representative of the broader I-O literature. The target article authors chose five journals they purport to be the top I-O journals. Four of these also ranked among the top 10 management journals this past year (Thomson Reuters, 2015): Academy of Management Journal (no. 3), Journal of Management (no. 4), Journal of Applied Psychology (no. 6), and Personnel Psychology (no. 8). There is no doubt that I-O psychology and management are highly interconnected fields, but management is not the only field with which I-O shares significant commonalities. Characterizing I-O research, as a whole, only through journals that are as much management as I-O may overestimate the attention we have given to managers. Moreover, this characterization may ignore important I-O subdisciplines that give greater attention to nonmanagers but that may not feature as prominently in management-crossover journals.
To illustrate, I selected four highly ranked I-O journals (as rated by Thomson Reuters in the 2015 “Applied Psychology” category) that did not appear among the top 25 management journals: Journal of Vocational Behavior (no. 8), Journal of Occupational Health Psychology (no. 10), Work and Stress (no. 11), and Organizational Behavior and Human Decision Making (no. 13). In each of these, I coded the samples reported in the year 2014 using the same categories as the target article.Footnote 1 The total number of studies published was N = 204, resulting in a total of 304 samples. Although there were some similarities in the rates of sampling in management-centric and applied psychology-centric journals, there were also some notable differences. For example, Bergman and Jean found that nonworker samples (22%) were over twice as likely to be used as worker samples (9%). However, among the applied psychology journals I coded, these types of samples were used at the same rate (k = 27 for both; 9%). In addition, student samples were more prevalent in the applied psychology journals I coded (k = 125; 41%). Although the target article suggests that these individuals are likely to become managers, professionals, and executives, a greater majority of these individuals likely met the criteria for “worker” at the time they were studied (e.g., low to medium skill, wage earners, nonmanagers).
That said, for both sets of journals, the most commonly assessed type of sample consisted of a mix of workers and nonworkers. These “mixed” samples accounted for 42% and 41% of all data reported, which is greater than the rates for worker and nonworker samples combined. The target article described two general types of samples fitting this description: (a) those sampled from specific organizations and (b) those sampled from the broader labor population. I further distinguished between these subgroups in my coding procedures. I found 14% (k = 42) of the total samples were gathered from specific organizations. With the popularity of hierarchical linear modeling and the growing necessity to account for the nested nature of organizational data, there is surely a substantial portion of data within this subgroup of samples that speaks to how constructs and their relationships differ (or don't) between worker and nonworker populations. More important, though, I found 27% (k = 83) of the total samples came from individuals recruited from the broader labor population. The vast majority of these samples were obtained through survey websites (e.g., Amazon Mechanical Turk [MTurk]) with U.S.$0.50–$1.00 being compensated for participation. To paraphrase an anonymous journal reviewer's feedback on one of my own studies using an MTurk sample to study organizational phenomena, “What gainfully employed person would spend 20–30 minutes to complete a survey for $1.50?” The answer is likely not those in managerial or professional positions but possibly at best those working low-skill and/or part-time, wage-earning jobs. In other words, those the target article describes as “workers.”
I agree that it would be ideal to know more about different working populations based on any number of characteristics, professional status being just one of them. Given the issues highlighted above, however, I am not sure concluding that workers are highly underrepresented is necessarily accurate. At best, I would conclude that the perspective taken in the target article is exemplary of deeper issues that have been raised in previous volumes of this journal regarding the nature of I-O psychology's connection to the field of management (e.g., Aguinis, Bradley, & Brodersen, Reference Aguinis, Bradley and Brodersen2014).
I-O Psychology Is a Free Market Economy
One of the major concerns presented by Bergman and Jean was that the underrepresentation of nonmanager populations in I-O research “could minimize the impact the I-O discipline has on psychological sciences as a whole, organizations, and society, relegating I-O psychology to the fringe” (p. 91). Let us assume that a disproportion in the study of manager and nonmanager work populations exists in I-O, as suggested in the target article. Does this necessarily indicate dysfunction or that our discipline is at risk of losing whatever spot we currently hold at the scientific table? I believe the answer is no.
As an applied science, I-O psychology seeks to provide solutions to real world problems. In this way, the I-O community is naturally capitalistic. The topics published on (and not published on) during a given period are guided by the perceived demand from the working world. This phenomenon is analogous to the rise and fall of stock prices. Researchers invest their time and resources in solving the most novel and salient real world problems. Similarly, editors “buy up” manuscripts on topics that they view as gaining momentum and “sell off” manuscripts on topics for which they perceive demand to be weak or waning. A salient example is the increase in research on unemployment, underemployment, and downsizing survival in the wake of the Great Recession. I believe that, had the literature search presented in the target article been extended a few years prior to 2012, even the journals included for analysis would have reflected this discipline-wide trend (e.g., Creed, King, Hood, & McKenzie, Reference Creed, King, Hood and McKenzie2009; Song, Foo, Uy, & Sun, Reference Song, Foo, Uy and Sun2011; van Hooft & Noordzij, Reference van Hooft and Noordzij2009).
We must not forget that applied disciplines such as ours were born out of the ability to identify and meet salient real world demand (Roethlisberger & Dickson, Reference Roethlisberger and Dickson1939; Taylor, Reference Taylor1911), and we will be sustained through our ability to continue to do so. The authors of the target article suggest that this will require a major change in “who” we study, suggesting that a demand to do so exists. This may be true to a greater or lesser extent, but it is important to keep in mind that real world demand for practical solutions exists for a wide range of problems from a similar range of stakeholders. Being limited by size and resources, I-O psychology must effectively answer the questions of “where?” and “how much?” when determining how to allocate our attention in a way that maximizes our practical contribution. I believe that the capitalistic system that is in place will naturally drive these decisions.
I-O Psychology Is a Trading Post for Interdisciplinary Research
Let us again assume that managers, professionals, and executives are being oversampled within the I-O psychology literature. From a perspective of practical contribution, this would be a truly enviable problem, as managers and executives have the greatest influence in organizations. Thus, the better we understand their behavior and the environment in which they function, the better positioned we will be to improve the way work gets done and the work experience. Unfortunately, we all know how difficult it is to actually obtain data from such sources. Of course, this reality runs counter to the target article authors’ presumption that I-O is risking the othering, discounting, or pathologizing of nonmanager samples through the practice of oversampling manager samples.
In relation to this, I take issue with the target article's broader assertion that “When I-O psychology fails to study the full range of the labor market, it is likely that the field is not fully articulating the nomological network surrounding a construct of interest” (Bergman & Jean, p. 91). I do so because, in presenting this argument, the authors present a narrow characterization of I-O psychology. First, this characterization fails to give credence to the fact that a great amount of what we, as I-O psychologists, know about people in the work context comes from research conducted in other disciplines (e.g., clinical, cognitive, personality, and social psychology), wherein the typical study participant is much more likely to be a part-time wage earner in the service industry than a full-time salaried manager. Second, this characterization fails to acknowledge the diverse practical and scientific work in which I-O psychologists are involved. For example, I-O-trained academicians commonly work and collaborate with others in departments outside of psychology, and those who do call a psychology department home likely do not teach only I-O courses. Moreover, it is often a matter of opinion whether a particular journal is an “I-O” journal, but I-O psychologists publish and consume research in a vast array of journals (a subset of which has produced an impressive body of evidence from nonmanager samples). The eclectic space in which I-O psychologists are allowed to function is, I believe, one of the advantages of our discipline, and it is this collection of diverse perspectives that should quash concerns that we are developing too narrow of a view on workplace phenomena.
Concluding Remarks
I agree with the authors of the target article that the intentions of I-O psychologists should be to understand the attitudes, behaviors, and cognitions of individuals at all levels of the organization, not to mention in a diverse range of occupations. From a practical standpoint, there simply may not be enough journal outlets that are explicitly “I-O” to provide the attention that each of these populations deserves (I know what you are thinking here, but many lament that there are already too many of these journals of which to keep track). However, when one considers the broader social science literature that I-O psychologists both contribute to and consume, it is likely that we know far more about people who work in a diverse range of occupations than credence is given to in the target article. Because managers and executives have the greatest impact on workplace behavior and because researchers in other social science disciplines generally do not study these populations with great frequency, it makes economic sense for I-O psychologists to commit a significant amount of our limited resources to studying them in order to meet a practical demand. That said, the data presented in the target article do suggest that only 22% of the research published in the journals analyzed was actually conducted among the highly influential individuals at the upper organizational levels. This is a far cry from the type of monopoly on manager samples that would likely be needed in order to lead I-O psychologists to develop a skewed understanding of workplace behavior. In the end, I believe our legitimacy as a discipline is strengthened by the fact that our training and work has positioned us to meet an eclectic range of real world demands relating to the world of work.