Introduction
For decades, several methods of bibliometric analysis have been suggested to estimate a particular journal's usefulness to the scientific community. Although many have been proposed and are available, impact factor remains the ‘gold standard’ surrogate indicator of quality in research.Reference Callaham, Wears and Weber1 Impact factor, first coined by Eugene Garfield in 1955, is calculated as the number of all current citations of source items published over the preceding two years.Reference Garfield2 Other forms of impact factor have since been described, including the five-year impact factor and the adjusted impact factor (which excludes citations to letters and editorials).
Despite an exponential growth in the number of articles published over the last decades in the field of otorhinolaryngology, very few studies have investigated journal impact factor in our specialty. In 2002, Roy et al. published their analysis of ENT journal impact factors from the publication year 1998.Reference Roy, Hughes, Jones and Fenton3 Since the publication of that work, the digital revolution has allowed for unprecedented amounts of information, with increased availability and access to that information. This is particularly true for scientific and medical literature, where we have witnessed dramatic changes in how we research, write, publish and, in particular, cite.Reference Coelho, Edelmayer and Fenton4 As the volume of source material grows, so too does the need for specific measures to establish relevance. This study aimed to determine whether ENT journal impact factors have changed since the publication of Roy and colleagues' original article, and, if so, ascertain how and why.
Materials and methods
Access to the Web of Science database and the database for the 2007 to 2012 online editions of Journal Citation Reports® was provided through the Virginia Commonwealth University library. Forty-three ENT journals with impact factors were located for the year 2012 (the most recent available year) in the Journal Citation Reports. As in the previous study,Reference Roy, Hughes, Jones and Fenton3 a total of eight audiology journals were excluded. The journal Dysphagia was considered a multidisciplinary journal and was excluded, as in the previous study. Out of the 34 remaining ENT journals, the top 15 were evaluated, as was also done in the previous study.
Citable items, as defined by Journal Citation Reports, are research articles and reviews. Journal Citation Reports classifies an item as a review if it meets any of the following criteria: the paper cites more than 100 references; it appears in a review section of a journal or within a review publication; the words ‘review’ or ‘overview’ are shown in the paper's title; or the abstract expresses that it is either a survey or a review.
The 2012 standard impact factor for each journal is calculated in terms of the number of citations that each journal received in 2012 for articles published in both 2010 and 2011; that number is then divided by the total number of articles published in 2010 and 2011 for each journal. Below is an example of how the 2012 standard impact factor for ‘Journal X’ would be calculated.
In 2012, Journal X was cited 175 times for papers published in 2010 and 125 times for papers published in 2011, with a total of 300 citations. One hundred articles were published in Journal X in 2010 and 50 articles were published in 2011, with a total of 150 articles. Based on this information, the 2012 impact factor for Journal X is calculated as:
$$\eqalign{& \left\{\matrix{ \hbox{Citations in 2012} \cr \, \hbox{for items published} \cr \hbox{in 2010 and 2011} } \right\} \div \left\{\matrix{ \hbox{Number of} \cr \hbox{items published} \cr \hbox{in 2010 and 2011}} \right\} \cr & = \displaystyle{{300} \over {150}}=2.0}$$The five-year impact factor was calculated in a similar way: the total number of citations in 2012 for articles published in the years 2007 to 2011 was divided by the total number of articles published in that journal from 2007 to 2011.
Unlike the original study, Journal Citation Reports calculations currently exclude editorials, letters, news items and meeting abstracts. Therefore, this study's impact factors are analogous to the adjusted impact factors in the original study.
Results
Data from a total of 15 ENT journals were retrieved from Journal Citation Reports. Citation information and impact factors are presented in Tables I and II, with comparisons to the original study. For all 15 journals studied, the average standard impact factor and 5-year impact factor increased by 2.72 and 2.05 fold respectively when compared with 1998.
Table I Ent journals ranked by standard impact factor for 2012
*Formally known as the American Journal of Otology
Table II Ent journals ranked by five-year impact factor for 2012
*Formally known as the American Journal of Otology
The Journal of the Association for Research in Otolaryngology (‘JARO’) had the highest standard impact factor (2.952) and five-year impact factor (3.244). In 1998, Head & Neck had the highest adjusted standard impact factor (1.325), whereas Archives of Otolaryngology – Head & Neck Surgery had the highest five-year impact factor (1.627). Laryngoscope had the highest total number of citations in 2012 for articles published in 2010 and 2011 (n = 1702), and the highest total number of articles published in 2010 and 2011 (n = 860). Laryngoscope also had the highest total number of citations from 1996 to 1997 (n = 640), although Acta Otolaryngologica had the greatest total number of articles published during the same time period.
Five journals new to the standard impact factor and five-year impact factor lists are the Journal of the Association for Research in Otolaryngology, Current Opinion in Otolaryngology & Head and Neck Surgery, Rhinology, the American Journal of Otolaryngology, and the International Journal of Pediatric Otorhinolaryngology. Current Opinion in Otolaryngology & Head and Neck Surgery was in the top 15 journals according to standard impact factor analysis, but not according to 5-year impact factor analysis as data were not available from Journal Citation Reports. The journals that no longer made the list were HNO, Acta Otolaryngologica, and the Journal of Laryngology & Otology.
The rank order of impact factor did not differ greatly between standard (two-year) impact factor and five-year impact factor analysis, and no journals (with the exception of the Annals of Otology, Rhinology & Laryngology) moved more than one position when the two methods were compared. This is in contrast to the 1998 paper, which saw much greater fluctuations in rank order when analysed using standard impact factor and five-year impact factor methods.
The average standard impact factor and 5-year impact factor for the top 15 journals were 1.82 and 1.99 respectively, reflecting a 9.3 per cent difference. In the 1998 analysis, the average standard impact factor and 5-year impact factor for the top 15 journals were 0.67 and 0.97 respectively, reflecting a 44.8 per cent difference.
Discussion
The journal impact factor is a measure of the frequency with which an average article in a particular journal has been cited over a given time frame. Theoretically, the more an article is referenced, the more important it is considered, and thus, by proxy, the more influential the journal in which it is published. Although the journal impact factor has come under scrutiny lately, it remains the most widely used of the bibliometric markers.Reference Oosthuizen and Fenton5
Impact factors are based on both a numerator (number of citations) and a denominator (total number of articles published), and a high number of citations for a journal does not always correlate with a high impact factor.Reference Garfield6 For example, the journal Laryngoscope had the highest number of citations in 2012 for the years 2010–2011, yet it is ranked fourth in terms of impact factor. Likewise, if the Journal of the Association for Research in Otolaryngology was ranked according to the number of citations it received in 2012 for the years 2007 to 2011 (n = 730), it would be ranked 11th. However, as only 225 articles were published in the Journal of the Association for Research in Otolaryngology during that time period, this journal has the highest standard impact factor and highest 5-year impact factor of any journal studied.
The Journal of the Association for Research in Otolaryngology, which debuted at the top spot for both standard impact factor and five-year impact factor has only been published since 2000. In a relatively brief timespan, it has become the premier forum for original experimental research in ENT, and a primary publishing outlet for the growing number of post-graduate and basic scientist members of ENT departments worldwide. Theoretically, its success may have been at the expense of the other journals. The journal publishes bi-monthly, with only 104 articles published between 2010 and 2012, and 225 published between 2007 and 2011. The Journal of the Association for Research in Otolaryngology had the second smallest number of articles published of any journal studied between 2010 and 2011, and smallest number of articles published of any journal between 2007 and 2011. This effectively resulted in a very small denominator on which its high standard impact factor and five-year impact factor are based.
Manipulation of the numerator and denominator can and has been used in the past by editors in order to influence impact factor.Reference Chew, Villanueva and Van Der Weyden7 High selectivity by the editors of journals toward lengthier articles would mean that fewer articles would fit into the journal, thus decreasing the denominator and increasing the total value of the impact factor. In the time period studied, four of the journals that had five-year impact factor scores calculated in the original study (from 1993 to 1997) decreased the total number of articles published between 2007 and 2011. Three of these journals (the Archives of Otolaryngology – Head & Neck Surgery, Clinical Otolaryngology and Otolaryngology Clinics of North America) had an average increase of 50.4 per cent in their five-year impact factor. The fourth, Annals of Otology, Rhinology & Laryngology, actually had a decrease in its five-year impact factor by 1.7 per cent. Interestingly, of the three journals that had increases in impact factor, two of the journals (the Archives of Otolaryngology – Head & Neck Surgery and Otolaryngology Clinics of North America) fell in the rank order list. This would suggest that forces which can influence article citations (i.e. the numerator) might have had a greater effect on impact factor calculations than manipulation of the denominator, as is likely believed by the editorial boards of these journals.
Another method of increasing impact factor is to publish scientific articles as correspondence. These items can be cited and hence count in the numerator of the impact factor calculation, but as correspondence they are not counted by Journal Citation Reports in the denominator. Some ENT journals have adopted this practice and have seen large increases in impact factor. An alternative theory is that there exists an equilibration point at which limiting the number of articles begins to have a positive effect on impact factor. Supporting this idea is the one journal for which decreasing published articles led to an increase in both impact factor and rank – Clinical Otolaryngology. This journal decreased the number of articles published between 2007 and 2011 by the most, with 170 fewer articles and at least 280 fewer articles than the other 3 journals that had also decreased their number of publications. Most journals, however, did not decrease the total number of articles published. In fact, nearly all journals had an increase in standard impact factor and five-year impact factor, and several journals are new to the lists. This is likely due to increases in citation rates (the numerator).
In general, citation counts are a surrogate identifier of scholarly impact. However, once a specific article's findings become established knowledge, it may be more likely to be included in a review article, which is then cited.Reference Bollen, Rodriguez and Van de Sompel8 Moreover, once the findings have been accepted as incontrovertible, the findings are absorbed as common knowledge and the original article may not be cited at all.Reference Coelho, Edelmayer and Fenton4
Changes in citation and publishing are by no means unique to ENT. The number of citations to internal and general medicine journals alone increased by 22 per cent between the years 2000 and 2005, and in that same time frame the total number of articles published decreased by roughly 11 per cent.Reference Chew, Villanueva and Van Der Weyden7 Compared to other small surgery specialties, ENT has made a substantial increase in its overall impact. When measured in terms of the cumulative or aggregate impact factors of all ENT journals from 2003 to 2012 (as depicted in Journal Citation Reports), ENT as a field has increased its impact factor by 51.3 per cent since 2003. Other smaller specialties such as dermatology, ophthalmology, orthopaedics, obstetrics and gynaecology, transplant surgery, and urology and nephrology have increased by 27.1 per cent, 19.5 per cent, 67.3 per cent, 30.7 per cent, 43.4 per cent and 14.1 per cent, respectively.
• Impact factor remains a valuable metric by which an individual journal's influence can be measured
• Over the last 15 years, journal impact factors have changed for the field of ENT
• Changes in impact factor may reflect changing clinical and research interests within ENT
• Like other specialties, the internet and digital revolution have had profound effects on how ENT articles are read, written, published and cited
• As a result, five-year intervals may no longer be needed to adequately reflect journal impact for smaller specialties
In the original article, the authors concluded that a base of five years is more appropriate for ENT journals for calculating impact factor.Reference Roy, Hughes, Jones and Fenton3 As a small specialty, they suggested that delays in publication, time to disseminate (and cite) published works, and the overall volume of articles published precluded reasonable comparisons to larger specialties. The authors supported this conclusion by noting marked changes in the rank order of journals when comparing standard impact factors to five-year impact factors. Interestingly, in the current study, we found relatively little change in either the absolute value or the rank order when comparing standard impact factor to five-year impact factor. The most likely explanation for this has been the monumental increase in access to information through the internet and the utilisation of internet-based referencing software. The speed and ease with which articles can now be retrieved, read, written, referenced and published has afforded an unprecedented and geometric rise in the number of articles cited. In our study of ENT citation classics, we found that between 1999 and 2009, the number of articles cited over 100 times had increased over 1000 per cent.Reference Coelho, Edelmayer and Fenton4 Indeed, in this new fast-paced reality, a five-year interval may no longer be necessary to adequately determine the true impact of a specific article or journal.
Conclusion
Impact factors for ENT journals have significantly increased over the past decade. Although many remain highly ranked, others have fallen off the list and have been replaced. Changes in the number of articles cited and published clearly have a direct effect on impact factor; however, exactly how these changes affect rank order remains to be elucidated. How we access, read, write and publish has been significantly influenced by the digital revolution, as have impact factors. As a result, using the alternative five-year impact factor may no longer be necessary for small specialties.
Impact factor is not a perfect bibliometric indicator of influence; authors should carefully weigh the prestige of a journal (by impact factor) against its propriety as an appropriate platform when submitting their work. Likewise, editorial boards should focus solely on the scientific merit of submitted articles, and not on a perceived assumption that impact factors can be manipulated.
