Scientific fields benefit when their researchers engage in self-reflection. Accordingly, we welcome the evidence gathered by Gardner, Ryan, and Snoeyink (Reference Gardner, Ryan and Snoeyink2018) on gender differences in our field, the field of industrial and organizational (I-O) psychology. In this commentary, we argue that such self-reflection processes can be further enhanced by taking advantage of the wealth and breadth of scientometrics, the quantitative study of science.
How I-O Psychology Can Benefit From the Science of Scientometrics
Scientometricians (i.e., researchers in the field of scientometrics) mainly study production and reception of scientific output (i.e., possible measures of scientific performance) and other forms of scientific communication. For instance, they investigate performance indicators (e.g., the h-index) and web-based supplements (“altmetrics”; for an introductory book see Thelwall, Reference Thelwall2016) and analyze different publication data sources (e.g., how dominated the database PsycINFO is by English-language publications; see Krampen, Reference Krampen2016). Moreover, they study similarities between papers (e.g., by using new techniques such as co-citation proximity analysis; Gipp & Beel, Reference Gipp, Beel, Larsen and Leta2009) as well as try to visualize knowledge domains (“science mapping”; see Chen, Reference Chen2017). For a better understanding of the field of scientometrics, readers are encourage to consult relevant journals such as Scientometrics, Journal of the American Society for Information Science and Technology, and Journal of Informetrics or introductory books such as the one by Qiu, Zhao, Yang, and Dong (Reference Qiu, Zhao, Yang and Dong2017).
The findings of scientometrics are relevant for I-O researchers interested in gender differences, as research citation metrics might reflect women's and men's scientific performance from a different perspective and they therefore complement results derived from other methods (e.g., Poon & Leeves, Reference Poon and Leeves2017). For example, our own analyses of gender differences among all members of the Society for Industrial and Organizational Psychology (SIOP) in the year 2013 showed gender differences in number of publications (favoring men) but not in the average journal impact factor (König, Fell, Kellnhofer, & Schui, Reference König, Fell, Kellnhofer and Schui2015), indicating the importance of different performance indicators, whereas Gardner et al. (Reference Gardner, Ryan and Snoeyink2018) looked at the number of publications in two journals, Journal of Applied Psychology and Personnel Psychology, as the only performance indicator in the area of publications.
To explore their research questions, scientometricians typically use large data sets. For example, our SIOP member analysis (König et al., Reference König, Fell, Kellnhofer and Schui2015) had an N of authors of 4,234 and an N of publications of 46,656, and a second publication based on this data set also included around 100,000 collaborators (Fell & König, Reference Fell and König2016). Another study on gender differences in all sciences was even based on an N of 5,483,841 papers (Larivière, Ni, Gingras, Cronin, & Sugimoto, Reference Larivière, Ni, Gingras, Cronin and Sugimoto2013). Such large Ns reduce the likelihood that results are biased by factors such as the peculiarities of chosen journals. In this realm, König and Bajwa (Reference König and Bajwa2018) showed that the journal Personnel Psychology (one of the two journals Gardner et al. analyzed) is more dominated by US authors than other top I-O psychology journals. This US dominance might have biased Gardner et al.’s results because international authors are (slightly) more likely to be female than US authors (König & Bajwa, Reference König and Bajwa2018). Gardner et al.’s choice of journals might also help explain why we found only minor gender differences in percentages of first authorships in our much larger data set on SIOP members (König et al., Reference König, Fell, Kellnhofer and Schui2015), unlike Gardner et al. who found 65% of first authors to be male. Furthermore, large Ns allow for insightful subgroup analyses. For example, although König et al. (Reference König, Fell, Kellnhofer and Schui2015) found clear gender differences in a number of publications in the total sample, nearly no gender differences were found among SIOP student members.
To run comprehensive scientometric analyses, researchers have developed special automation tools that are often freely available, like Publish or Perish (http://www.harzing.com/pop.htm) or SocSciBot (a web crawler and hyperlink analyzer, see http://socscibot.wlv.ac.uk/). Particularly helpful for the study of gender differences is https://genderize.io/, a webpage that enables individuals to determine people's gender by their first names, using a list of more than 200,000 distinct first names from nearly 80 countries (for an example see Fell & König, Reference Fell and König2016), and that frees researchers’ resources because they do not have to code gender by hand (as Gardner et al. did). Such tools will therefore make it easier for interested readers to run their own scientometric analyses.
Using scientometric approaches, I-O psychologists will be able to answer many questions to understand gender differences in our field. Some of these questions have already been asked by Gardner et al. (Reference Gardner, Ryan and Snoeyink2018). For example, they asked how (dis)similar women's and men's publication records are at different care stages, and they also asked about gender differences in research topics within I-O psychology (a first scientometric answer has already been found in König et al., Reference König, Fell, Kellnhofer and Schui2015). A myriad of additional questions could be explored. For example, scientometric studies could explore whether male (or female) researchers from our field remain focused on a low number of topics over the course of their careers versus diversifying their research interests (cf. Abramo, D'Angelo, & Di Costa, Reference Abramo, D'Angelo and Di Costa2018), whether papers in I-O psychology that receive nearly no immediate but much delayed recognition are more likely be written by female versus male researchers (cf. Bornmann, Ye, & Ye, Reference Bornmann, Ye and Ye2018), whether female and male I-O psychologists differ whom they choose as international collaborators (cf. Ni & An, Reference Ni and An2018), whether gender explains a significant amount of variance in reviewing activities and reviewing leniency or severity (cf. Ortega, Reference Ortega2017), and whether the relationship between number of publications and salary is moderated by gender (cf. Sandnes, Reference Sandnes2018). Surely, interested readers who make themselves a bit more familiar with the world of scientometrics will come up with many more topics worth being studied.
How Scientometrics Can Benefit From I-O Psychology
Fostering the link between I-O psychology and scientometrics will likely also benefit the field of scientometrics. In I-O psychology, team and collaboration processes, which are relevant for teams of authors as well, have been studied for a long time, with special interest toward gender diversity (e.g., Roberson, Ryan, & Ragins, Reference Roberson, Ryan and Ragins2017). For example, female authors in I-O psychology collaborate more intensively than male I-O psychologists (Fell & König, Reference Fell and König2016). Why is this the case? Answering such a question will be difficult or even impossible by conducting only scientometric analyses of collaboration patterns. Instead, it will likely require models and data with which I-O psychologists are familiar (e.g., data on agreeableness of male and female authors; see Feingold, Reference Feingold1994). In addition, the wealth of knowledge on career trajectories that our field has produced over the years (e.g., Sullivan & Baruch, Reference Sullivan and Baruch2009) could be helpful for guiding scientometric studies on the careers of female and male scientists.
You have
Access
Scientific fields benefit when their researchers engage in self-reflection. Accordingly, we welcome the evidence gathered by Gardner, Ryan, and Snoeyink (Reference Gardner, Ryan and Snoeyink2018) on gender differences in our field, the field of industrial and organizational (I-O) psychology. In this commentary, we argue that such self-reflection processes can be further enhanced by taking advantage of the wealth and breadth of scientometrics, the quantitative study of science.
How I-O Psychology Can Benefit From the Science of Scientometrics
Scientometricians (i.e., researchers in the field of scientometrics) mainly study production and reception of scientific output (i.e., possible measures of scientific performance) and other forms of scientific communication. For instance, they investigate performance indicators (e.g., the h-index) and web-based supplements (“altmetrics”; for an introductory book see Thelwall, Reference Thelwall2016) and analyze different publication data sources (e.g., how dominated the database PsycINFO is by English-language publications; see Krampen, Reference Krampen2016). Moreover, they study similarities between papers (e.g., by using new techniques such as co-citation proximity analysis; Gipp & Beel, Reference Gipp, Beel, Larsen and Leta2009) as well as try to visualize knowledge domains (“science mapping”; see Chen, Reference Chen2017). For a better understanding of the field of scientometrics, readers are encourage to consult relevant journals such as Scientometrics, Journal of the American Society for Information Science and Technology, and Journal of Informetrics or introductory books such as the one by Qiu, Zhao, Yang, and Dong (Reference Qiu, Zhao, Yang and Dong2017).
The findings of scientometrics are relevant for I-O researchers interested in gender differences, as research citation metrics might reflect women's and men's scientific performance from a different perspective and they therefore complement results derived from other methods (e.g., Poon & Leeves, Reference Poon and Leeves2017). For example, our own analyses of gender differences among all members of the Society for Industrial and Organizational Psychology (SIOP) in the year 2013 showed gender differences in number of publications (favoring men) but not in the average journal impact factor (König, Fell, Kellnhofer, & Schui, Reference König, Fell, Kellnhofer and Schui2015), indicating the importance of different performance indicators, whereas Gardner et al. (Reference Gardner, Ryan and Snoeyink2018) looked at the number of publications in two journals, Journal of Applied Psychology and Personnel Psychology, as the only performance indicator in the area of publications.
To explore their research questions, scientometricians typically use large data sets. For example, our SIOP member analysis (König et al., Reference König, Fell, Kellnhofer and Schui2015) had an N of authors of 4,234 and an N of publications of 46,656, and a second publication based on this data set also included around 100,000 collaborators (Fell & König, Reference Fell and König2016). Another study on gender differences in all sciences was even based on an N of 5,483,841 papers (Larivière, Ni, Gingras, Cronin, & Sugimoto, Reference Larivière, Ni, Gingras, Cronin and Sugimoto2013). Such large Ns reduce the likelihood that results are biased by factors such as the peculiarities of chosen journals. In this realm, König and Bajwa (Reference König and Bajwa2018) showed that the journal Personnel Psychology (one of the two journals Gardner et al. analyzed) is more dominated by US authors than other top I-O psychology journals. This US dominance might have biased Gardner et al.’s results because international authors are (slightly) more likely to be female than US authors (König & Bajwa, Reference König and Bajwa2018). Gardner et al.’s choice of journals might also help explain why we found only minor gender differences in percentages of first authorships in our much larger data set on SIOP members (König et al., Reference König, Fell, Kellnhofer and Schui2015), unlike Gardner et al. who found 65% of first authors to be male. Furthermore, large Ns allow for insightful subgroup analyses. For example, although König et al. (Reference König, Fell, Kellnhofer and Schui2015) found clear gender differences in a number of publications in the total sample, nearly no gender differences were found among SIOP student members.
To run comprehensive scientometric analyses, researchers have developed special automation tools that are often freely available, like Publish or Perish (http://www.harzing.com/pop.htm) or SocSciBot (a web crawler and hyperlink analyzer, see http://socscibot.wlv.ac.uk/). Particularly helpful for the study of gender differences is https://genderize.io/, a webpage that enables individuals to determine people's gender by their first names, using a list of more than 200,000 distinct first names from nearly 80 countries (for an example see Fell & König, Reference Fell and König2016), and that frees researchers’ resources because they do not have to code gender by hand (as Gardner et al. did). Such tools will therefore make it easier for interested readers to run their own scientometric analyses.
Using scientometric approaches, I-O psychologists will be able to answer many questions to understand gender differences in our field. Some of these questions have already been asked by Gardner et al. (Reference Gardner, Ryan and Snoeyink2018). For example, they asked how (dis)similar women's and men's publication records are at different care stages, and they also asked about gender differences in research topics within I-O psychology (a first scientometric answer has already been found in König et al., Reference König, Fell, Kellnhofer and Schui2015). A myriad of additional questions could be explored. For example, scientometric studies could explore whether male (or female) researchers from our field remain focused on a low number of topics over the course of their careers versus diversifying their research interests (cf. Abramo, D'Angelo, & Di Costa, Reference Abramo, D'Angelo and Di Costa2018), whether papers in I-O psychology that receive nearly no immediate but much delayed recognition are more likely be written by female versus male researchers (cf. Bornmann, Ye, & Ye, Reference Bornmann, Ye and Ye2018), whether female and male I-O psychologists differ whom they choose as international collaborators (cf. Ni & An, Reference Ni and An2018), whether gender explains a significant amount of variance in reviewing activities and reviewing leniency or severity (cf. Ortega, Reference Ortega2017), and whether the relationship between number of publications and salary is moderated by gender (cf. Sandnes, Reference Sandnes2018). Surely, interested readers who make themselves a bit more familiar with the world of scientometrics will come up with many more topics worth being studied.
How Scientometrics Can Benefit From I-O Psychology
Fostering the link between I-O psychology and scientometrics will likely also benefit the field of scientometrics. In I-O psychology, team and collaboration processes, which are relevant for teams of authors as well, have been studied for a long time, with special interest toward gender diversity (e.g., Roberson, Ryan, & Ragins, Reference Roberson, Ryan and Ragins2017). For example, female authors in I-O psychology collaborate more intensively than male I-O psychologists (Fell & König, Reference Fell and König2016). Why is this the case? Answering such a question will be difficult or even impossible by conducting only scientometric analyses of collaboration patterns. Instead, it will likely require models and data with which I-O psychologists are familiar (e.g., data on agreeableness of male and female authors; see Feingold, Reference Feingold1994). In addition, the wealth of knowledge on career trajectories that our field has produced over the years (e.g., Sullivan & Baruch, Reference Sullivan and Baruch2009) could be helpful for guiding scientometric studies on the careers of female and male scientists.
Conclusion
We hope that our commentary stimulates more exchange between I-O psychology and scientometrics, and we particularly hope that this exchange leads to more research that helps us assess the nature and progress of I-O psychology, in particular regarding gender issues.