Introduction
The frequency with which a scientific article is cited by other studies is one way to measure its academic influence. Although the academic influence of an article does not exactly correspond to its quality, it does reflect how that particular study has generated discussion, controversy and further research in its field. Moreover, citation analysis has become a common evaluation method for scientific journals, articles and authors.
Recently, several medical specialties have ranked articles within their fields by citation frequency.Reference Quinn, Hensey and McDowell1–Reference Heldwein, Rhoden and Morgentaler6 An analysis of citation classics (defined as articles that received at least 100 citationsReference Garfield7) in the field of otorhinolaryngology revealed a dramatic increase in the number of publications over the last few decades.Reference Fenton, Roy, Hughes and Jones8, Reference Coelho, Edelmayer and Fenton9 This may be due to significant advancements in information technology and communication that allow studies and experiments to be rapidly performed, written, reviewed, published, and cited.
This study aimed to identify the 100 most cited articles in the field of otolaryngology – head and neck surgery, and to analyse the characteristics that made them important to the ENT community. This could help future research have higher relevance to the scientific community.
Materials and methods
A total of 44 journals are included in the ‘otorhinolaryngology’ subject category of the 2013 Journal Citation Reports Science Edition (Thompson Reuters, New York, New York, USA). Using all databases in the Web of Science (Thompson Reuters), a comprehensive search for articles published in the journals in the otorhinolaryngology subject category of the 2013 Journal Citation Report Science Edition over the last 30 years (1985–2014) was performed in May 2015.
The Thompson Reuters search engine provides information on the number of times a particular article has been cited by other articles. The 100 most cited articles were selected and data was collected for each article on the journal title, impact factor (in the Journal Citation Report 2013) and five-year impact factor, author names, country of origin of the authors (if there was more than one country of origin, then the country of origin of the first author was considered), publication year, research type (basic science or clinical research), article subtype (original study, review article, case report, expert opinion) and the research field (otology and neurotology, rhinology and/or endoscopic skull base surgery, head and neck surgery, laryngology, sleep disordered breathing, and general ENT surgery).
Pearson's correlation analysis of the impact factor of the journal of publication and number of citations was performed. In addition, correlation between the publication year and number of citations of the top 100 articles was assessed. A p value of less than 0.05 was considered statistically significant. Data were collected and analysed using Microsoft Office Excel 2007 (Microsoft, Redmond, Washington, USA).
Results
From 1985 to 2014 a total of 238 125 articles were published in the 44 journals in the otorhinolaryngology subject category of the 2013 Journal Citation Report Science Edition.
The article title, author names, publication year, title of the journal in which the article was published, country of origin, number of citations and citation density for the 100 most cited articles in the field of otorhinolaryngology are listed in Table I. All were published in 15 of the 44 journals in this category (Table II); all were published in English. Ninety-nine of the papers were published between 1985 and 2007; the other was published in 2012. The year with the highest number of top 100 publications was 1986, with 10 publications (Figure 1). The number of citations ranged between 208 and 1559 (mean 327.7), and the citation density ranged between 6.7 and 89.2 (mean 17.2).
Fig. 1 Graph showing the number of articles in the top 100 cited articles by study year.
Table I List of top 100 cited otolaryngologic articles
Table II Number of articles in the top 100 list by source journal
Most papers in the list were clinical articles (n = 77); the other 23 were basic science articles. The leading research field was otology and neurotology (n = 50), followed by rhinology (n = 23) and head and neck surgery (n = 11; shown in Table III). The country of origin of these articles was mainly the USA, followed by European countries (Table IV).
Table III Number of articles in the top 100 list by subspecialty
Table IV Top 100 publications by country of origin
The effect of the journal impact factor on ranking was evaluated by correlation analysis; no correlation was found between the journal impact factor for 2013 (r = 0.027, p = 0.787) or the five-year impact factor (r = 0.021, p = 0.834) and the number of citations. In addition, there was no correlation between publication year and the number of citations (r = −0.130, p = 0.196).
Discussion
The importance and the influence of an article in the medical literature can be estimated by the number of citations. The number of citations indicates the number of times the article has been cited in subsequent publications. It depends on both the article topic and the influence it has on the work of other authors (who cite it in their own publications). However, the number of citations and the impact factor of the journal of publication do not always indicate the quality of the original article. Indeed, both the author name(s) and journal of publication may significantly influence the number of citations. This study aimed to determine which articles published in otorhinolaryngological journals have had the most influence by ranking the 100 most cited works. In addition, the characteristics of these articles were analysed to determine which qualities make an ENT article important to other researchers working in the specialty.
Some problems are associated with the use of this method alone to evaluate the importance of an article, an author's publication record or the overall performance of a journal. Firstly, this type of citation analysis does not account for self-citation, citation in textbook and lectures, and the possibility that authors will cite articles published in a journal in which they seek to publish their own work.Reference Lefaivre, Guy, O'Brien, Blachut, Shadgan and Broekhuyse10–Reference Seglen12 Secondly, the time lag inherent in citation analysis places the most recently published articles at a disadvantage.Reference Callaham, Weber and Wears13–Reference Cole15 Indeed, despite the evolving focus on improving research quality and the importance of evidence-based practice, most of the top 100 cited papers were published before 2007 and the year with the highest number of publications cited (n = 10) was 1986. Thirdly, this study was limited by the categorisation of journals in the Web of Science. By design, this study considered only journals in the otorhinolaryngology subject category of the 2013 Journal Citation Report Science Edition. Although this represents a complete list of the high-impact journals in the otorhinolaryngology field, it means that articles published in the general medical and basic science literature and those published in journals covering related disciplines (e.g. neurosurgery and maxillofacial surgery) were not considered. Therefore, some articles that have influenced ENT specialists have probably been left out. Fourthly, there was an obvious bias toward articles published in English language journals. Finally, authors are more likely to cite articles because they were previously cited rather than for their content or quality, thus causing a ‘snowball effect’ on the number of citations.
Of the top 100 articles in the otorhinolaryngology specialty, the most cited article had 1559 citations and the least-cited article had 208 citations. Therefore, every article on the list can be considered a ‘citation classic’, because the arbitrary but frequently used benchmark is 100 citations.Reference Garfield7, Reference Coelho, Edelmayer and Fenton9
Characteristics of the top 100 otorhinolaryngology articles were similar to those reported for other disciplines. Most articles originated in the USA, similar to the findings of reviews of anaesthesia, paediatrics, plastic surgery, emergency medicine, general surgery and orthopaedic surgery.Reference Quinn, Hensey and McDowell1–Reference Baltussen and Kindler5, Reference Lefaivre, Shadgan and O'Brien11 This indicates that US authors may have an advantage when publishing articles in international journals. If the publication number per million of population is considered, then some European countries do as well as or even better than the USA (Table IV). However, there are no publications from Asia or Africa in the top 100 list. The predominance of clinical articles was also noted in other reviews.Reference Loonen, Hage and Kon2–Reference Baltussen and Kindler5, Reference Lefaivre, Shadgan and O'Brien11 When the most common ENT subspecialities are considered, otology and neurotology is best represented in the top 100 list (50 per cent of articles), followed by rhinology (23 per cent) and head and neck surgery (11 per cent).
Although the journal impact factor did not correlate with the number of citations, all articles in the top 100 list were published in only 15 of the total 44 journals in the otorhinolaryngology subject category of the 2013 Journal Citation Report Science Edition. Thus, the journal of publication is an important factor in determining the possibility that an article will be cited.
Conclusion
This review of the 100 most cited articles in otorhinolaryngology shows that the journal of publication and country of origin are the most important factors in determining the possibility of citation. Future bibliometric analyses could help confirm these observations or identify changes in the characteristics of the most influential publications.
