Highhouse et al. (Reference Highhouse, Zickar and Melick2020) provided evidence of how the Society for Industrial and Organizational Psychology (SIOP) members’ perceptions of the prestige of academic journals are structured and how these perceptions play a role in scholarly communications in the field of industrial-organizational (I-O) psychology. Analyses of publication practices illustrate that business schools publish the clear majority of I-O-related articles, concurring with the conclusions of Aguinis et al. (Reference Aguinis, Bradley and Brodersen2014) that management scholarship will continue to lead future directions of I-O psychology. As initially deliberated by Aguinis et al. (Reference Aguinis, Bradley and Brodersen2014), one of the main driving forces triggering the move of I-O psychologists from university psychology departments to business schools is the vast availability of financial resources at the latter. The exodus to business schools is also in part because I-O psychology is treated in university psychology departments as inferior to other psychology disciplines. Compounding these difficulties are the existing reward and incentive systems at business schools that create pressure to publish in prestigious, macromanagement journals. Because specialized I-O outlets have less relevance for business schools, new, unexplored areas of I-O psychology are often overlooked in business research (Judge, Reference Judge2003; Ryan & Ford, Reference Ryan and Ford2010; Zickar & Highhouse, Reference Zickar and Highhouse2017). Yet, I-O scholars at business schools often face a quandary about what and where to publish to get hired, promoted, and rewarded. Consequently, the interplay among pressure, prestige, and self-guided interests arising from incentives available at business schools presents an engrossing problem for the future development of I-O scholarship. Focusing solely on journal prestige for determining publication practices may shed light on only one side of the story, if rankings, incentives, and their effects are neglected.
To illustrate this problem, I take a reflective and critical perspective by discussing how prestige and context-dependent metrics that signal prestige (i.e., rankings) determine publication practices in the I-O field at business schools, particularly in France. As one of the most competitive business education landscapes, the French higher-education system differs in how business schools operate as separate, autonomous entities, distinct from universities, which allows for a detailed examination of publication practices in isolation under the light of the centralized journal-ranking methodologies that affect researchers’ careers. As an I-O psychology scholar at a French business school, I demonstrate in this commentary how different interpretations of journal rankings define publication practices in the context of French business schools. I reveal that prestigious I-O journals garner considerably more attention from business scholars, as expected. Furthermore, I exhibit that less prestigious journals with similar reputations and metrics attract scholars differently depending on the availability of incentives. In closing, I argue that, in the long run, subjective prestige evaluations and the use of metrics hinder scientific progress more than helping it.
Journal prestige, scientific communication, and changing perceptions
It was not long ago that journal prestige bore an ultimately different meaning in the scientific community. Despite the credibility of prestigious journals having been challenged in recent years, it is still one of the most tangible proof points of high-quality scholarship, and competition for the limited space in these journals has become even fiercer. However, in a collective effort by Nosek et al. (Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler, Buck, Chambers, Chin, Christensen, Contestabile, Dafoe, Eich, Freese, Glennerster, Goroff, Green, Hesse, Humphreys and Yarkoni2015) and Open Science Collaboration (2012, 2015), it was documented that the reproducibility of studies in psychology, even in prestigious journals, is a worrying issue. A harsher criticism bestowed by Brembs (Reference Brembs2018) argued that journal reputation, one of the most critical factors determining scholars’ careers and success, does not guarantee high-quality scholarship. The results were conclusive for psychology, as increased journal rank did not correlate with increased methodological rigor. Furthermore, like Archambault and Larivière (Reference Archambault and Larivière2009), Brembs et al. (Reference Brembs, Button and Munafo2013) also heavily criticized the use of journal impact factor (JIF) as a key metric because it promotes practices that have detrimental effects on scientific progress. According to their study, not only is the way that JIF is calculated troublesome; the likelihood of fraudulent or less reliable studies, decreased ethical standards, scientific misconduct, and other types of questionable research practices are more frequently encountered in journals with high JIFs. It is further asserted that the existing, multirounded peer-review process for prestigious journals fails in both efficiency and effectiveness by slowing down the dissemination process, stealing the time of researchers (both reviewers and authors), and adding no meaningful value to the final quality. Conversely, Ioannidis and Boyack (Reference Ioannidis and Boyack2020) defended the merits of quantitative measures despite acknowledging the fact that the current system opens the door to novel possibilities of misusing metrics, such as artificially boosting them for unfair personal and institutional gain. Similarly, Aguinis et al. (Reference Aguinis, Cummings, Ramani and Cummings2020) cited negative consequences of evaluation of researchers’ publication quality based purely on the number of publications in high-prestige journals and criticized the new rules of the publishing game, in which quantity wins over quality.
Beyond these apparent problems, Edwards and Roy (Reference Edwards and Roy2017) identified the current reward structure and hypercompetition as the root causes of the pervasive use of quantitative measures as performance indicators. Undoubtedly, financial incentives play a major role in the publication process, as they do in all kinds of performance outcomes. In a meta-analysis, Jenkins et al. (Reference Jenkins, Mitra, Gupta and Shaw1998) concluded that financial incentives are great tools for increasing performance in terms of quantity. However, the same directional effect of financial incentives on performance quality outcomes could not be observed. The race for limited positions in research institutions, limited space in journal outlets, and limited sources of funding puts enormous pressure on all of academia. Publishing in prestigious journals and building a reputation are not necessarily byproducts of intrinsic motivation but rather a basic response to incentives triggered by extrinsic factors such as selection, promotion, and financial rewards for scholars. School rankings, which affect student choices and school finances, are also dependent upon research outcomes; therefore, beyond personal motives, institutional forces also create excessive pressure on business school researchers. However, that business schools monopolize publications in the I-O psychology field is not only a simple battle over prestige; it is also the natural consequence of a circular model between finances and incentives at business schools. To better illustrate the relationship between the desire to publish in well-reputed journals and the incentives and rewards system, I will provide a brief introduction to business schools in France.
The French business schools context
As Anseel et al. (Reference Anseel, Carette, Lang and Lievens2014) emphasized, there has been a growing competition among European business schools during the past decade that has influenced I-O psychology research. To attract international students, competition among French business schools (FBSs) has also proliferated (Blanchard, Reference Blanchard2009; Kaplan, Reference Kaplan2014; Thietart, Reference Thietart2009) and now mirrors the American business school landscape. However, France significantly differs from most other countries. Not only is France home to the world’s first business school; it has also historically treated business education as a vocation rather than an academic discipline (Kaplan, Reference Kaplan2014). In addition, the Grandes Écoles system presents a unique and exceptional example that influences how schools are administered, students are admitted, schools are funded, and relations between these schools and the corporate world are maintained (Blanchard, Reference Blanchard2009; Lichy & Pon, Reference Lichy and Pon2015). What distinguishes FBSs from others is that most are independent institutions, unconnected to a university system. Either they are associated with the local Chambers of Commerce or they act as private establishments with unique academic freedom and administrative autonomy (Blanchard, Reference Blanchard2009; Kumar & Usunier, Reference Kumar and Usunier2001; Lichy & Pon, Reference Lichy and Pon2015; Thietart, Reference Thietart2009). In response to a highly competitive environment, accreditations in business school education play a critical role in adopting a global perspective and signaling quality and reputation (Thomas et al., Reference Thomas, Billsberry, Ambrosini and Barton2014). As the largest and oldest accreditation body recognized worldwide, the Association to Advance Collegiate Schools of Business (AACSB) ensures that business schools meet certain standards. The ESSEC Business School (École Supérieure des Sciences Economiques et Commerciales), one of the top business schools in Europe according to several rankings, including Financial Times, Economist, and QS World University Rankings, is the first French business school to obtain this accreditation and the first institution outside of North America to be accredited (White et al., Reference White, Miles and Lavernier2009). Following in the footsteps of the ESSEC, there are currently 24 other FBSs with AACSB accreditation, making France one of the most competitive marketplaces for business education worldwide, after the US (with 534 schools), China (39), the UK (35), and Taiwan (26).
Why do different constituencies (FBSs) view the journals differently?
As FBSs became more competitive and internationally focused, research expectations also evolved (Thietart, Reference Thietart2009). Vocational schools that traditionally expected their professors to focus on teaching as opposed to conducting research have now been swept up in the “publish or perish” game due to internationalization concerns and ranking pressure (Dubois & Walsh, Reference Dubois and Walsh2017). Changing research expectations have pushed FBSs into isomorphic structures with publication patterns and interests in line with the international standards (Thomas et al., Reference Thomas, Billsberry, Ambrosini and Barton2014). Not surprisingly, professors who publish in prestigious journals both increase the visibility of business schools in the international arena and alter prestige perceptions of potential candidates seeking positions at business schools. For that reason, in most business schools in France, publication bonuses and reduction of teaching load further incentivize publication in high-prestige journals (Carton et al., Reference Carton, Dameron, Durand, Dameron and Duran2018).
Journal prestige is open to interpretation. Although some FBSs maintain their own internal rankings, two major institutions in France, CNRS (Centre National de la Recherche Scientifique) and FNEGE (Fondation Nationale pour l’Enseigement de la Gestion des Enterprises), oversee the classification of management and economics journals using different criteria. Although some objective bibliometric measures such as JIF or the SCImago h-index are used for journal classification by both institutions, other European and French institutions’ ranking lists are evaluated as well. Based on the assessment of scientific committees, CNRS and FNEGE classify journals using a quality rating from 1 (highest) to 4 (lowest). However, general interest journals, that is, more prestigious, are identified as exceptional by CNRS with “1g.” Similarly, FNEGE further distinguishes exceptional journals as 1* within the best-classified journals. Despite the controversy behind how these lists are determined, monetary and nonmonetary rewards heavily influence the publishing behavior of professors in FBSs (Carton et al., Reference Carton, Dameron, Durand, Dameron and Duran2018).
Judge (Reference Judge2003) recognized that I-O psychology has had difficulty finding its place in business schools because of its limited relevance for business research. The situation is no different in France. Until 2019, no psychology field was mentioned in either the FNEGE or CNRS rankings; FNEGE finally included a brand-new field category of applied psychology/sociology in 2019. New journals such as Applied Psychology: An International Review and the Journal of Applied Social Psychology (JASP) were created to accommodate this academic gap. The CNRS still does not consider applied psychology as a discipline that is independent from general management, organization studies, or human resources management.
Table 1 summarizes the current classification of the journals that are listed by Highhouse et al. (Reference Highhouse, Zickar and Melick2020) and their respective fields by FNEGE and CNRS, along with the prestige measure, JIF, and the SCImago h-index. For each journal, the number of papers (articles and reviews, excluding editorials) published by author(s) who are affiliated with an FBS and the total number of papers with author affiliation in France are also identified between the years 2010 to date (Feb 22, 2020) to conduct comparative publication behavior analyses. I used the Web of Science and Scopus databases to retrieve the number of publications and author affiliations.
Table 1. Journal Prestige, Bibliometric Indexes, Rankings and Publication Behavior in France
Table 2. Correlation Matrix
Organization Science (OrgSci) has received the highest number of publications from France, of which 62 come with an FBS affiliation. Similarly, the Academy of Management Journal (AMJ) received 60 publications from France, of which 56 of them were written by an author from an FBS. The following journals, Human Relations (HR), the Journal of Management (JOM), and Organizational Behavior and Human Decision Processes (OBHDP), are the three remaining outlets that received the highest number of publications by scholars from an FBS. Of the 38 journals, nine are missing from both the FNEGE and the CNRS rankings. These are Applied Psychological Measurement (APM), Consulting Psychology Journal (CPJ), Educational and Psychological Measurement (EPM), Journal of Organizational Behavior Management (JOBM), Journal of Personnel Psychology (JPP), Personnel Assessment and Decisions (PAD), Personality and Individual Differences (PID), Public Personnel Management (PPM), and The Industrial Psychologist (TIP). The journals APM, EPM, JOBM, PAD, and TIP did not contain any publication authored by a business school researcher in France. By the same token, PID contained three papers authored by a business school researcher from France, whereas 65 of the remaining publications were authored by non-business-school scholars who were affiliated with a French institution, contrary to its nonranked status. The journals, CPJ, JPP, and PPM that are not classified by FNEGE or CNRS only received a limited number of contributions from FBSs. One immediate conclusion that one can make is that prestige and relevant metrics are quite important in shaping the publication behavior in business schools using FNEGE and CNRS. However, the drawback is that business school scholars in France are not encouraged (if not discouraged) to publish in one of these nonclassified journals because publications appearing in those neither affect researchers’ promotions and bonuses nor improve institutional rankings as far as CNRS and FNEGE classifications are concerned. Even though they bear prestige and make an academic impact, from the perspective of most business schools, they will be found irrelevant. For top journals, there is a consensus about the prestige perceptions, relevant metrics, rankings, and classifications. However, when it comes to less prestigious journals, the situation becomes more ambiguous. Thus, there is a need to better understand how classifications determine publication decisions, especially in less prestigious journals.
Analysis of publication trends
The FNEGE and CNRS classifications are calculated based on metrics like the SCImago h-index and JIF. Table 2 illustrates the correlations between these metrics, classifications, and the number of publications by authors with an FBS affiliation. As indicated in the table, all correlations are significant at p < .05. Both institutions’ classifications correlate with SIOP members’ perception scores and two other bibliometric indexes strongly. The number of publications by scholars at FBSs is correlated with SIOP members’ prestige perceptions (r = 0.49, p = .002 < .01), JIF (r = 0.37, p = .028 < .05), the SCImago h-index (r = 0.7, p < .001), the FNEGE classification (r = -0.56, p = .002 < .01), and the CNRS classification (r = -0.51, p = .014 < .05). Figure 1 also indicates the differences in publication practices in classified versus nonclassified journals. The comparison of publication practices is also significant, t(36) = 2.38, p = .022 < .05. Figure 2 demonstrates the jittered density plot based on publication per journal classification, and Figure 3 indicates the relationship between the number of publications and JIF. This concurs with the fact that the higher the prestige, bibliometric, and classifications, the higher the number of the publications authored by scholars from FBSs. In other words, CNRS and FNEGE classifications capture most elite I-O journals correctly, in line with the SIOP members’ prestige perceptions, and set targets for business school scholars in France.
Figure 1. Publication analysis plot in classified vs. nonclassified journals.
Figure 2. Publication analysis plot based on CNRS and FNEGE classifications.
Figure 3. Publication analysis plot based on journal impact factor.
Although the picture is clear that both FNEGE and CNRS rankings are used as a proxy that signals prestige and impact, how does an author make a decision about where to publish if top journals could not be achieved? Do SIOP members’ prestige perceptions reflect the practices at FBSs? Is “journals’ prestige” translated directly as “le prestige des revues scientifiques,” or are the meaning and context lost in translation? To investigate this, I compared the above mentioned nine nonclassified journals against nine classified journals that are listed as 3 or 4. The classified journals are Applied Psychology: An International Review (APIR), European Journal of Work and Organizational Psychology (EJWOP), Human Performance (HP), International Journal of Selection and Assessment (IJSA), Journal of Applied Behavioral Science (JABS), Journal of Business and Psychology (JBP), Journal of Managerial Psychology (JMP), Journal of Occupational Health Psychology (JOHP), and Work & Stress (W&S). Although the Journal of Applied Social Psychology is classified in FNEGE in the third category, it was excluded from analysis, as it was added to the list in June 2019. Below, Table 3 indicates the t tests that were applied to compare the differences among prestige, JIF, SCImago indices, and the number of publications in classified versus nonclassified journals (see the descriptive statistics for these variables in Table 4). According to these results, the only significant difference is observed in the number of publications (p = .0452 < .05), whereas prestige perceptions scores, JIF, and the SCImago h-index did not differ based on the classification status of the journal.
Table 3. Independent t Tests for Bibliometric Indices and Publications Based on Classifications
Table 4. Descriptive Statistics for the Bibliometric Indices for Publications Based on Classifications
This presents an intriguing finding that may explain publication practices in the French business school context. Not surprisingly, when journals are listed, ranked, or classified, these measures serve as an indicator of prestige and quality of the journals. If these indices are objectively measured, the decision making about where to publish becomes less cumbersome. However, when assessment criteria include more subjective evaluations, they may introduce additional challenges to publication venue choices of researchers. The problem magnifies if these subjective assessments are linked to rewards and incentives. It is evident from our analysis that researchers at FBSs respond positively to incentives. When publications are classified by either FNEGE or CNRS, or both, regardless of objective bibliometric indices and SIOP members’ prestige perceptions, they are perceived to be superior based on publication numbers. As Jenkins and others argued, our finding substantiates the use of incentives as a regulatory tool affecting publication behavior externally, as we indicated the evidence of directionality of incentive responses for less prestigious journals with similar reputations and metrics.
Concluding remarks
Scholarly impact is one of the most important life goals of every researcher. The history of science is full of anecdotes that illustrate the paradigm of researchers who caught success in much later stages of their lives. Adopting objective measures allows researchers and administrators to set standards; correctly assess the value of research contributions; and eliminate unfair treatments in recruitment, selection, promotion, remuneration, and allocation of rewards. Along the same line, the rules of the game have changed. As Marshall McLuhan (Reference McLuhan1964, p.7) famously quoted, “the medium is the message.” This describes today’s academic communication model. Journals once perceived as channels of intellectual communication have now become the message itself. How many A-publications (or 4-star, or first class, etc.) one published is now a more crucial indicator of intellectual value of contribution. More disturbingly, “for management researchers, this categorization can translate into a stark dichotomy and imposed choice between scholarship that counts (i.e., published in A journals) and scholarship that does not count (i.e., published anywhere else)” (Aguinis et al. Reference Aguinis, Cummings, Ramani and Cummings2020, p. 136). Acknowledging the prestige battle stemming from researchers’ intrinsic motivations for scholarly communication would be an understatement because in the construction of prestige, external motives play a significant role.
Incentives that fetishize the use of rankings are heavily criticized in the literature. Business schools are not short of journal lists and rankings. Several countries, like France, have their own sources to evaluate journal quality. Willmott (Reference Willmott2011) noted that the one-size-fits-all logic penalizes academic fields such as sustainability, business communication, and tourism. Currently, I-O psychology suffers from the same problem. The existing list of CNRS and FNEGE classifications come with several limitations. The most obvious one is that the lists are not extensively covering the field of I-O psychology. Some key journals such as Journal of Counseling Psychology, Human Resource Development Quarterly, European Journal of Psychological Assessment, International Journal of Stress Management, Journal of Work and Organizational Psychology, and Small Group Research are among the high-impact journals that are not covered by those lists. Moreover, leading interdisciplinary journals that publish relevant I-O studies like Human Factors, New Technology, Work and Employment, Applied Ergonomics, Computers in Human Behavior, and Judgment and Decision Making are also kept off the list. Similarly, newly established, emerging venues and contemporary open-access outlets that may have a potential for interesting studies are currently not classified either. Second, the misuse of these rankings threatens the future development of scientific progress. Off-the-list publications that are considered as scholarship that does not count clearly present a hard-hitting issue. Journal prestige interpretations are usually context dependent. The meaning of prestige can be interpreted differently in different settings, as there are contextual boundaries, social contingencies, and local constraints. In the case of the field of I-O psychology, current incentives at FBSs do not fully encourage researchers to publish in diverse journals. My findings support the shortcomings of Highhouse et al. (Reference Highhouse, Zickar and Melick2020) in that business school researchers in France place greater emphasis on prestigious journals. However, when lower prestige journals are concerned, a unique pattern is observed, reflecting the practices in line with the availability of incentives, which ultimately guide scholars’ publication venue choices. Considering all these limitations of rankings and stigmas of superfluous metrics, we should focus on how we improve scientific communication and impact without fetishizing the (mis)use of metrics.
Highhouse et al. (Reference Highhouse, Zickar and Melick2020) provided evidence of how the Society for Industrial and Organizational Psychology (SIOP) members’ perceptions of the prestige of academic journals are structured and how these perceptions play a role in scholarly communications in the field of industrial-organizational (I-O) psychology. Analyses of publication practices illustrate that business schools publish the clear majority of I-O-related articles, concurring with the conclusions of Aguinis et al. (Reference Aguinis, Bradley and Brodersen2014) that management scholarship will continue to lead future directions of I-O psychology. As initially deliberated by Aguinis et al. (Reference Aguinis, Bradley and Brodersen2014), one of the main driving forces triggering the move of I-O psychologists from university psychology departments to business schools is the vast availability of financial resources at the latter. The exodus to business schools is also in part because I-O psychology is treated in university psychology departments as inferior to other psychology disciplines. Compounding these difficulties are the existing reward and incentive systems at business schools that create pressure to publish in prestigious, macromanagement journals. Because specialized I-O outlets have less relevance for business schools, new, unexplored areas of I-O psychology are often overlooked in business research (Judge, Reference Judge2003; Ryan & Ford, Reference Ryan and Ford2010; Zickar & Highhouse, Reference Zickar and Highhouse2017). Yet, I-O scholars at business schools often face a quandary about what and where to publish to get hired, promoted, and rewarded. Consequently, the interplay among pressure, prestige, and self-guided interests arising from incentives available at business schools presents an engrossing problem for the future development of I-O scholarship. Focusing solely on journal prestige for determining publication practices may shed light on only one side of the story, if rankings, incentives, and their effects are neglected.
To illustrate this problem, I take a reflective and critical perspective by discussing how prestige and context-dependent metrics that signal prestige (i.e., rankings) determine publication practices in the I-O field at business schools, particularly in France. As one of the most competitive business education landscapes, the French higher-education system differs in how business schools operate as separate, autonomous entities, distinct from universities, which allows for a detailed examination of publication practices in isolation under the light of the centralized journal-ranking methodologies that affect researchers’ careers. As an I-O psychology scholar at a French business school, I demonstrate in this commentary how different interpretations of journal rankings define publication practices in the context of French business schools. I reveal that prestigious I-O journals garner considerably more attention from business scholars, as expected. Furthermore, I exhibit that less prestigious journals with similar reputations and metrics attract scholars differently depending on the availability of incentives. In closing, I argue that, in the long run, subjective prestige evaluations and the use of metrics hinder scientific progress more than helping it.
Journal prestige, scientific communication, and changing perceptions
It was not long ago that journal prestige bore an ultimately different meaning in the scientific community. Despite the credibility of prestigious journals having been challenged in recent years, it is still one of the most tangible proof points of high-quality scholarship, and competition for the limited space in these journals has become even fiercer. However, in a collective effort by Nosek et al. (Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler, Buck, Chambers, Chin, Christensen, Contestabile, Dafoe, Eich, Freese, Glennerster, Goroff, Green, Hesse, Humphreys and Yarkoni2015) and Open Science Collaboration (2012, 2015), it was documented that the reproducibility of studies in psychology, even in prestigious journals, is a worrying issue. A harsher criticism bestowed by Brembs (Reference Brembs2018) argued that journal reputation, one of the most critical factors determining scholars’ careers and success, does not guarantee high-quality scholarship. The results were conclusive for psychology, as increased journal rank did not correlate with increased methodological rigor. Furthermore, like Archambault and Larivière (Reference Archambault and Larivière2009), Brembs et al. (Reference Brembs, Button and Munafo2013) also heavily criticized the use of journal impact factor (JIF) as a key metric because it promotes practices that have detrimental effects on scientific progress. According to their study, not only is the way that JIF is calculated troublesome; the likelihood of fraudulent or less reliable studies, decreased ethical standards, scientific misconduct, and other types of questionable research practices are more frequently encountered in journals with high JIFs. It is further asserted that the existing, multirounded peer-review process for prestigious journals fails in both efficiency and effectiveness by slowing down the dissemination process, stealing the time of researchers (both reviewers and authors), and adding no meaningful value to the final quality. Conversely, Ioannidis and Boyack (Reference Ioannidis and Boyack2020) defended the merits of quantitative measures despite acknowledging the fact that the current system opens the door to novel possibilities of misusing metrics, such as artificially boosting them for unfair personal and institutional gain. Similarly, Aguinis et al. (Reference Aguinis, Cummings, Ramani and Cummings2020) cited negative consequences of evaluation of researchers’ publication quality based purely on the number of publications in high-prestige journals and criticized the new rules of the publishing game, in which quantity wins over quality.
Beyond these apparent problems, Edwards and Roy (Reference Edwards and Roy2017) identified the current reward structure and hypercompetition as the root causes of the pervasive use of quantitative measures as performance indicators. Undoubtedly, financial incentives play a major role in the publication process, as they do in all kinds of performance outcomes. In a meta-analysis, Jenkins et al. (Reference Jenkins, Mitra, Gupta and Shaw1998) concluded that financial incentives are great tools for increasing performance in terms of quantity. However, the same directional effect of financial incentives on performance quality outcomes could not be observed. The race for limited positions in research institutions, limited space in journal outlets, and limited sources of funding puts enormous pressure on all of academia. Publishing in prestigious journals and building a reputation are not necessarily byproducts of intrinsic motivation but rather a basic response to incentives triggered by extrinsic factors such as selection, promotion, and financial rewards for scholars. School rankings, which affect student choices and school finances, are also dependent upon research outcomes; therefore, beyond personal motives, institutional forces also create excessive pressure on business school researchers. However, that business schools monopolize publications in the I-O psychology field is not only a simple battle over prestige; it is also the natural consequence of a circular model between finances and incentives at business schools. To better illustrate the relationship between the desire to publish in well-reputed journals and the incentives and rewards system, I will provide a brief introduction to business schools in France.
The French business schools context
As Anseel et al. (Reference Anseel, Carette, Lang and Lievens2014) emphasized, there has been a growing competition among European business schools during the past decade that has influenced I-O psychology research. To attract international students, competition among French business schools (FBSs) has also proliferated (Blanchard, Reference Blanchard2009; Kaplan, Reference Kaplan2014; Thietart, Reference Thietart2009) and now mirrors the American business school landscape. However, France significantly differs from most other countries. Not only is France home to the world’s first business school; it has also historically treated business education as a vocation rather than an academic discipline (Kaplan, Reference Kaplan2014). In addition, the Grandes Écoles system presents a unique and exceptional example that influences how schools are administered, students are admitted, schools are funded, and relations between these schools and the corporate world are maintained (Blanchard, Reference Blanchard2009; Lichy & Pon, Reference Lichy and Pon2015). What distinguishes FBSs from others is that most are independent institutions, unconnected to a university system. Either they are associated with the local Chambers of Commerce or they act as private establishments with unique academic freedom and administrative autonomy (Blanchard, Reference Blanchard2009; Kumar & Usunier, Reference Kumar and Usunier2001; Lichy & Pon, Reference Lichy and Pon2015; Thietart, Reference Thietart2009). In response to a highly competitive environment, accreditations in business school education play a critical role in adopting a global perspective and signaling quality and reputation (Thomas et al., Reference Thomas, Billsberry, Ambrosini and Barton2014). As the largest and oldest accreditation body recognized worldwide, the Association to Advance Collegiate Schools of Business (AACSB) ensures that business schools meet certain standards. The ESSEC Business School (École Supérieure des Sciences Economiques et Commerciales), one of the top business schools in Europe according to several rankings, including Financial Times, Economist, and QS World University Rankings, is the first French business school to obtain this accreditation and the first institution outside of North America to be accredited (White et al., Reference White, Miles and Lavernier2009). Following in the footsteps of the ESSEC, there are currently 24 other FBSs with AACSB accreditation, making France one of the most competitive marketplaces for business education worldwide, after the US (with 534 schools), China (39), the UK (35), and Taiwan (26).
Why do different constituencies (FBSs) view the journals differently?
As FBSs became more competitive and internationally focused, research expectations also evolved (Thietart, Reference Thietart2009). Vocational schools that traditionally expected their professors to focus on teaching as opposed to conducting research have now been swept up in the “publish or perish” game due to internationalization concerns and ranking pressure (Dubois & Walsh, Reference Dubois and Walsh2017). Changing research expectations have pushed FBSs into isomorphic structures with publication patterns and interests in line with the international standards (Thomas et al., Reference Thomas, Billsberry, Ambrosini and Barton2014). Not surprisingly, professors who publish in prestigious journals both increase the visibility of business schools in the international arena and alter prestige perceptions of potential candidates seeking positions at business schools. For that reason, in most business schools in France, publication bonuses and reduction of teaching load further incentivize publication in high-prestige journals (Carton et al., Reference Carton, Dameron, Durand, Dameron and Duran2018).
Journal prestige is open to interpretation. Although some FBSs maintain their own internal rankings, two major institutions in France, CNRS (Centre National de la Recherche Scientifique) and FNEGE (Fondation Nationale pour l’Enseigement de la Gestion des Enterprises), oversee the classification of management and economics journals using different criteria. Although some objective bibliometric measures such as JIF or the SCImago h-index are used for journal classification by both institutions, other European and French institutions’ ranking lists are evaluated as well. Based on the assessment of scientific committees, CNRS and FNEGE classify journals using a quality rating from 1 (highest) to 4 (lowest). However, general interest journals, that is, more prestigious, are identified as exceptional by CNRS with “1g.” Similarly, FNEGE further distinguishes exceptional journals as 1* within the best-classified journals. Despite the controversy behind how these lists are determined, monetary and nonmonetary rewards heavily influence the publishing behavior of professors in FBSs (Carton et al., Reference Carton, Dameron, Durand, Dameron and Duran2018).
Judge (Reference Judge2003) recognized that I-O psychology has had difficulty finding its place in business schools because of its limited relevance for business research. The situation is no different in France. Until 2019, no psychology field was mentioned in either the FNEGE or CNRS rankings; FNEGE finally included a brand-new field category of applied psychology/sociology in 2019. New journals such as Applied Psychology: An International Review and the Journal of Applied Social Psychology (JASP) were created to accommodate this academic gap. The CNRS still does not consider applied psychology as a discipline that is independent from general management, organization studies, or human resources management.
Table 1 summarizes the current classification of the journals that are listed by Highhouse et al. (Reference Highhouse, Zickar and Melick2020) and their respective fields by FNEGE and CNRS, along with the prestige measure, JIF, and the SCImago h-index. For each journal, the number of papers (articles and reviews, excluding editorials) published by author(s) who are affiliated with an FBS and the total number of papers with author affiliation in France are also identified between the years 2010 to date (Feb 22, 2020) to conduct comparative publication behavior analyses. I used the Web of Science and Scopus databases to retrieve the number of publications and author affiliations.
Table 1. Journal Prestige, Bibliometric Indexes, Rankings and Publication Behavior in France
Note. * = nonranked journals. ** = ranked journals included in the analysis. *** = excluded from the analysis because APP PSY was not a recognized field in FNEGE until June 2019. GM = general management, OS = organization studies, HRM = human resources management, Psy/Soc = psychology/sociology, and NR = no rank.
Table 2. Correlation Matrix
*p < 0.05. **p < 0.01. ***p < 0.001.
Organization Science (OrgSci) has received the highest number of publications from France, of which 62 come with an FBS affiliation. Similarly, the Academy of Management Journal (AMJ) received 60 publications from France, of which 56 of them were written by an author from an FBS. The following journals, Human Relations (HR), the Journal of Management (JOM), and Organizational Behavior and Human Decision Processes (OBHDP), are the three remaining outlets that received the highest number of publications by scholars from an FBS. Of the 38 journals, nine are missing from both the FNEGE and the CNRS rankings. These are Applied Psychological Measurement (APM), Consulting Psychology Journal (CPJ), Educational and Psychological Measurement (EPM), Journal of Organizational Behavior Management (JOBM), Journal of Personnel Psychology (JPP), Personnel Assessment and Decisions (PAD), Personality and Individual Differences (PID), Public Personnel Management (PPM), and The Industrial Psychologist (TIP). The journals APM, EPM, JOBM, PAD, and TIP did not contain any publication authored by a business school researcher in France. By the same token, PID contained three papers authored by a business school researcher from France, whereas 65 of the remaining publications were authored by non-business-school scholars who were affiliated with a French institution, contrary to its nonranked status. The journals, CPJ, JPP, and PPM that are not classified by FNEGE or CNRS only received a limited number of contributions from FBSs. One immediate conclusion that one can make is that prestige and relevant metrics are quite important in shaping the publication behavior in business schools using FNEGE and CNRS. However, the drawback is that business school scholars in France are not encouraged (if not discouraged) to publish in one of these nonclassified journals because publications appearing in those neither affect researchers’ promotions and bonuses nor improve institutional rankings as far as CNRS and FNEGE classifications are concerned. Even though they bear prestige and make an academic impact, from the perspective of most business schools, they will be found irrelevant. For top journals, there is a consensus about the prestige perceptions, relevant metrics, rankings, and classifications. However, when it comes to less prestigious journals, the situation becomes more ambiguous. Thus, there is a need to better understand how classifications determine publication decisions, especially in less prestigious journals.
Analysis of publication trends
The FNEGE and CNRS classifications are calculated based on metrics like the SCImago h-index and JIF. Table 2 illustrates the correlations between these metrics, classifications, and the number of publications by authors with an FBS affiliation. As indicated in the table, all correlations are significant at p < .05. Both institutions’ classifications correlate with SIOP members’ perception scores and two other bibliometric indexes strongly. The number of publications by scholars at FBSs is correlated with SIOP members’ prestige perceptions (r = 0.49, p = .002 < .01), JIF (r = 0.37, p = .028 < .05), the SCImago h-index (r = 0.7, p < .001), the FNEGE classification (r = -0.56, p = .002 < .01), and the CNRS classification (r = -0.51, p = .014 < .05). Figure 1 also indicates the differences in publication practices in classified versus nonclassified journals. The comparison of publication practices is also significant, t(36) = 2.38, p = .022 < .05. Figure 2 demonstrates the jittered density plot based on publication per journal classification, and Figure 3 indicates the relationship between the number of publications and JIF. This concurs with the fact that the higher the prestige, bibliometric, and classifications, the higher the number of the publications authored by scholars from FBSs. In other words, CNRS and FNEGE classifications capture most elite I-O journals correctly, in line with the SIOP members’ prestige perceptions, and set targets for business school scholars in France.
Figure 1. Publication analysis plot in classified vs. nonclassified journals.
Figure 2. Publication analysis plot based on CNRS and FNEGE classifications.
Figure 3. Publication analysis plot based on journal impact factor.
Although the picture is clear that both FNEGE and CNRS rankings are used as a proxy that signals prestige and impact, how does an author make a decision about where to publish if top journals could not be achieved? Do SIOP members’ prestige perceptions reflect the practices at FBSs? Is “journals’ prestige” translated directly as “le prestige des revues scientifiques,” or are the meaning and context lost in translation? To investigate this, I compared the above mentioned nine nonclassified journals against nine classified journals that are listed as 3 or 4. The classified journals are Applied Psychology: An International Review (APIR), European Journal of Work and Organizational Psychology (EJWOP), Human Performance (HP), International Journal of Selection and Assessment (IJSA), Journal of Applied Behavioral Science (JABS), Journal of Business and Psychology (JBP), Journal of Managerial Psychology (JMP), Journal of Occupational Health Psychology (JOHP), and Work & Stress (W&S). Although the Journal of Applied Social Psychology is classified in FNEGE in the third category, it was excluded from analysis, as it was added to the list in June 2019. Below, Table 3 indicates the t tests that were applied to compare the differences among prestige, JIF, SCImago indices, and the number of publications in classified versus nonclassified journals (see the descriptive statistics for these variables in Table 4). According to these results, the only significant difference is observed in the number of publications (p = .0452 < .05), whereas prestige perceptions scores, JIF, and the SCImago h-index did not differ based on the classification status of the journal.
Table 3. Independent t Tests for Bibliometric Indices and Publications Based on Classifications
aLevene’s test is significant (p < .05), suggesting a violation of the assumption of equal variances.
*p < 0.05.
Table 4. Descriptive Statistics for the Bibliometric Indices for Publications Based on Classifications
This presents an intriguing finding that may explain publication practices in the French business school context. Not surprisingly, when journals are listed, ranked, or classified, these measures serve as an indicator of prestige and quality of the journals. If these indices are objectively measured, the decision making about where to publish becomes less cumbersome. However, when assessment criteria include more subjective evaluations, they may introduce additional challenges to publication venue choices of researchers. The problem magnifies if these subjective assessments are linked to rewards and incentives. It is evident from our analysis that researchers at FBSs respond positively to incentives. When publications are classified by either FNEGE or CNRS, or both, regardless of objective bibliometric indices and SIOP members’ prestige perceptions, they are perceived to be superior based on publication numbers. As Jenkins and others argued, our finding substantiates the use of incentives as a regulatory tool affecting publication behavior externally, as we indicated the evidence of directionality of incentive responses for less prestigious journals with similar reputations and metrics.
Concluding remarks
Scholarly impact is one of the most important life goals of every researcher. The history of science is full of anecdotes that illustrate the paradigm of researchers who caught success in much later stages of their lives. Adopting objective measures allows researchers and administrators to set standards; correctly assess the value of research contributions; and eliminate unfair treatments in recruitment, selection, promotion, remuneration, and allocation of rewards. Along the same line, the rules of the game have changed. As Marshall McLuhan (Reference McLuhan1964, p.7) famously quoted, “the medium is the message.” This describes today’s academic communication model. Journals once perceived as channels of intellectual communication have now become the message itself. How many A-publications (or 4-star, or first class, etc.) one published is now a more crucial indicator of intellectual value of contribution. More disturbingly, “for management researchers, this categorization can translate into a stark dichotomy and imposed choice between scholarship that counts (i.e., published in A journals) and scholarship that does not count (i.e., published anywhere else)” (Aguinis et al. Reference Aguinis, Cummings, Ramani and Cummings2020, p. 136). Acknowledging the prestige battle stemming from researchers’ intrinsic motivations for scholarly communication would be an understatement because in the construction of prestige, external motives play a significant role.
Incentives that fetishize the use of rankings are heavily criticized in the literature. Business schools are not short of journal lists and rankings. Several countries, like France, have their own sources to evaluate journal quality. Willmott (Reference Willmott2011) noted that the one-size-fits-all logic penalizes academic fields such as sustainability, business communication, and tourism. Currently, I-O psychology suffers from the same problem. The existing list of CNRS and FNEGE classifications come with several limitations. The most obvious one is that the lists are not extensively covering the field of I-O psychology. Some key journals such as Journal of Counseling Psychology, Human Resource Development Quarterly, European Journal of Psychological Assessment, International Journal of Stress Management, Journal of Work and Organizational Psychology, and Small Group Research are among the high-impact journals that are not covered by those lists. Moreover, leading interdisciplinary journals that publish relevant I-O studies like Human Factors, New Technology, Work and Employment, Applied Ergonomics, Computers in Human Behavior, and Judgment and Decision Making are also kept off the list. Similarly, newly established, emerging venues and contemporary open-access outlets that may have a potential for interesting studies are currently not classified either. Second, the misuse of these rankings threatens the future development of scientific progress. Off-the-list publications that are considered as scholarship that does not count clearly present a hard-hitting issue. Journal prestige interpretations are usually context dependent. The meaning of prestige can be interpreted differently in different settings, as there are contextual boundaries, social contingencies, and local constraints. In the case of the field of I-O psychology, current incentives at FBSs do not fully encourage researchers to publish in diverse journals. My findings support the shortcomings of Highhouse et al. (Reference Highhouse, Zickar and Melick2020) in that business school researchers in France place greater emphasis on prestigious journals. However, when lower prestige journals are concerned, a unique pattern is observed, reflecting the practices in line with the availability of incentives, which ultimately guide scholars’ publication venue choices. Considering all these limitations of rankings and stigmas of superfluous metrics, we should focus on how we improve scientific communication and impact without fetishizing the (mis)use of metrics.