Introduction
In the face of constricting research budgets coupled with increased service demands, there are mounting concerns about the decline of research output within the UK.Reference Bell1 Correspondingly, studies have demonstrated a fall in published research output from UK-based authors, in both general surgical and anaesthetic journals.Reference Phillips, Macgregor and Mihai2, Reference Feneck, Natarajan, Sebastian and Naughton3 With pressure on surgical trainees to demonstrate evidence of publications as part of job applications, this paradox presents a growing challenge. For otorhinolaryngology trainees in particular, a decline in the planned number of annual national training positions (from an indicative 45 per year in 2012 to 33 per year in 2013) serves to compound the competition for specialty training and consultant posts.4
With published articles representing a quantifiable proxy of research activity, the aim of this study was to analyse the research output in leading otorhinolaryngology journals, evaluating the geographical mix of authors and type of article published, in order to confirm or refute fears that published otorhinolaryngology research output from the UK is declining.
Materials and methods
Ethical considerations
This was a retrospective literature analysis so no formal ethical approval was required.
Approach
This study was based upon analysis of the research output of 12 otorhinolaryngology journals over a 10-year period (2000 to 2010). The journals were selected based upon highest five-year average journal impact factor, as presented within the otorhinolaryngology section of Thompson Reuters Web of Knowledge 2010 Journal Citation Reports.5 The five-year journal impact factor is the average number of times articles published in the past five years from a given journal have been cited in the index year (e.g. 2010). It is calculated by dividing the number of citations in a given year by the total number of articles published in the preceding five years. The 10 otorhinolaryngology journals with the highest five-year average impact factor were selected (Table I). As none of these top-10 journals were UK-based, the output of the 2 leading UK otorhinolaryngology journals was also considered.
Fig. 1 Percentage change in research output between 2000 and 2010, by region, for (a) the leading 10 global journals and (b) the leading 2 UK journals.
Table I Journal ranking by 5-year average impact factor
5-y av IF = 5-year average impact factor
Outcome measures
For each journal, citable articles were considered for four years: 2000, 2004, 2007 and 2010. Output from each volume was categorised into one of five geographical regions based upon the affiliation of the corresponding author: Europe; UK and Ireland; Japan; USA and Canada; and the rest of the world. Articles were categorised as one of five types: case reports, research, systematic reviews or meta-analyses, other citable articles (e.g. special commentaries and debates), and other non-citable articles (e.g. book reviews and conference abstracts). For further details, see Appendix 1. Articles included in the ‘other non-citable’ category were excluded from the analysis, as were articles from certain supplements (e.g. conference specials). Journal contents were sourced from the online electronic journal archives whenever possible, with PubMed or hard copies used when online archive information was incomplete. Journal name changes were accounted for during data collection.
Results and analysis
Absolute output
In the 10-year period considered, the leading 10 journals increased their absolute published citable output by 15.9 per cent (from 1664 citable articles in 2000 to 1928 in 2010). However, the two leading UK journals demonstrated a decrease in absolute citable output of 11.4 per cent (from 361 articles in 2000 to 320 in 2010) (Table II).
Table II Citable articles published in leading journals
Data indicate total number of articles.
Geographical mix
The share of total citable output from UK and Irish authors grew 22.8 per cent among the leading 10 global otorhinolaryngology journals (from 3.1 per cent in 2000 to 3.8 per cent in 2010), but fell 28.6 per cent among the leading 2 UK journals (from 60.4 per cent in 2000 to 43.1 per cent in 2010). The converse was true for USA and Canadian authors, whose output fell 12.4 per cent among the leading 10 journals (from 62.6 per cent in 2000 to 54.8 per cent in 2010), but grew 167.9 per cent among the leading 2 UK journals (from 2.2 per cent in 2000 to 5.9 per cent in 2010). Japanese authors' share of output fell 30.7 per cent (from 3.7 per cent in 2000 to 2.5 per cent in 2010) among the leading 10 journals and 20.4 per cent (from 4.7 per cent in 2000 to 3.8 per cent in 2010) among the leading 2 UK journals. European authors demonstrated 10.4 per cent growth among the leading 10 journals (from 19.4 per cent in 2000 to 21.4 per cent in 2010) and 5.8 per cent growth among the leading 2 UK journals (from 17.7 per cent in 2000 to 18.8 per cent in 2010). The proportional output of authors from the rest of the world showed a similarly positive trend, growing a considerable 56.1 per cent among the leading 10 journals (from 11.3 per cent in 2000 to 17.6 per cent in 2010) and 90.1 per cent among the leading 2 UK journals (from 15.0 per cent in 2000 to 28.4 per cent in 2010) (see Table III and Figure 1).
Table III Published, citable articles by geographical region
*Percentage of annual global total.
Article type
In absolute terms, articles classified as ‘research’ remained the overall majority among the leading 10 global journals (for authors of all regions combined). However, the relative proportion of research articles fell amongst authors of most regions (from a total of 80.0 per cent in 2000 to 73.0 per cent in 2010), with a corresponding growth in the total proportion of articles classified as systematic reviews or meta-analyses (from 1.5 per cent in 2000 to 4.5 per cent in 2010) and ‘other citable’ (from 9.9 per cent in 2000 to 13.9 per cent in 2010). The proportion of case reports remained unchanged over the 10-year period (8.7 per cent).
The leading two UK journals demonstrated an overall increase in the proportion of both research articles (from 51.2 per cent in 2000 to 56.2 per cent in 2010) and systematic review or meta-analysis articles (from 5.8 per cent in 2000 to 8.4 per cent in 2010), but a reduction in articles classed as either case reports (from 3.6 per cent in 2000 to 3.5 per cent in 2010) or ‘other citable’ (from 6.6 per cent in 2000 to 0.6 per cent in 2010).
For UK and Irish authors, research articles remained the most prevalent type among the leading 10 journals, albeit with an overall decline over the 10-year period (from 85 per cent in 2000 to 72 per cent in 2010). The proportion of systematic reviews and meta-analyses increased (from 0 per cent in 2000 to 9 per cent in 2010), the proportion of case reports declined (from 10 per cent in 2000 to 7 per cent in 2010) and the proportion of ‘other citable’ articles increased (from 6 per cent in 2000 to 12 per cent in 2010).
Among the leading 2 UK journals, the proportion of case reports from UK and Irish authors was greater than among the leading 10 global journals (with 38 per cent of articles classed as case reports in 2010 in the leading 2 UK journals, versus 7 per cent in the leading 10 global journals), with a correspondingly lower proportion of research articles in the leading 2 UK journals (46 per cent, versus 72 per cent in the 10 global journals). However, the proportion of case reports and research articles among the leading 2 UK journals remained stable over the 10-year period (Figure 2).
Fig. 2 Types of articles published in 2000 and 2010, by authors of all regions, in (a) the leading 10 global journals and (b) the leading 2 UK journals.
Discussion
Synopsis of key findings
In a gloomy era of economic austerity and associated mounting financial restrictions for both clinicians and researchers, the results of this study present a glimmer of optimism for UK otorhinolaryngologists. A 22.8 per cent increase in UK and Irish authors' share of total citable content among the 10 globally leading otorhinolaryngology journals is reassuringly significant, as it refutes the findings of other studies. The decline in UK and Irish authors' share of total output among the leading two UK journals should not necessarily be a cause for concern if it continues to remain accompanied by corresponding shifts in UK and Irish articles towards global journals with the highest impact factors; it could simply reflect greater international interest in UK journals and a corresponding rise in the international diversity of published content (content from authors in the USA and Canada, Europe, and the rest of the world increased by 108, 6 and 90 per cent, respectively, in the leading two UK journals). Furthermore, this sub-analysis only considered 2 UK-based journals (ranked 11th and 26th in the world, based upon 5-year average impact factor) and not the entire pool of UK otorhinolaryngology journals.
Comparison with other studies
These findings differ from those of studies reporting a decline in UK-based general surgical and anaesthetic research output, as well as a decline in the proportion of abstracts from UK medical conference meetings which went on to full publication.Reference Phillips, Macgregor and Mihai2, Reference Feneck, Natarajan, Sebastian and Naughton3, Reference Hopper, Atkinson, Razak, Rahim, Perera and Jones6 One proposed hypothesis is increasing clinical sub-specialisation in the USA and UK, leading to a shift in publications away from general to more sub-specialist journals.
An anecdotally voiced concern among senior clinicians is trainees' desire to ‘get published quick’, with a full article deemed to not be worth much more than an abstract. It is unclear how our findings would change if other types of variably cited and less academically rigorous articles, such as abstracts, letters and commentaries, were to be included. However, while this study reported a small drop in the overall proportion of articles classified as research and a rise in articles classified as systematic reviews or meta-analyses among the leading 10 journals, the overall proportion of articles classified as case reports remained static over the 10-year period. Output from UK and Irish authors followed a similar trend, albeit with more marked growth in the proportion of systematic reviews and meta-analyses and an overall decline in the proportion of case reports. Thus, within UK otorhinolaryngology at least, the fear that trainees are feeling the need to place greater emphasis on simpler, desk-based articles and case reports, rather than on more rigorous, laborious and unpredictable laboratory or clinical trial based research studies, may be unfounded. Two factors may account for the global growth in systematic reviews and meta-analysis articles. On the one hand, this may reflect an increasingly evidence-based world, in which the combined conclusions of multiple rather than individual studies help influence clinical practice. However, with meta-analyses cited more than primary research articles, this may also reflect the editorial decisions of journals keen to improve their impact factors.Reference Feneck, Natarajan, Sebastian and Naughton3
Despite a relative decline in global market-share of published research output in the past 10 years, the USA still remains the dominant force in terms of absolute numbers of published research articles in leading otorhinolaryngology journals. However, it is important to note that while the UK has far fewer researchers compared with other leading research nations, a recent UK Government Department of Business, Innovation and Skills report concluded that the UK was far more efficient in terms of output per researcher.7 The UK is also reportedly more efficient than other nations in terms of research output, leading seven comparator countries (Canada, China, France, Germany, Italy, Japan and the USA) on citations per unit spend on gross expenditure on research and development. In the 2000 to 2010 period, articles in the overall health and medical sciences field increased their overall average impact factor from 0.93 to 1.05. Of greater concern is the fact that surgical specialties fare particularly poorly in the funding game: despite more than one in four care episodes within the UK National Health Service being surgical, only a small fraction of research funding is invested in surgical research. Of the £1.53 billion spent by the UK Medical Research Council and the National Institute for Health Research between 2008 and 2009, only £25.5 million went to surgical research.Reference McCulloch8
Study limitations
This study was by no means exhaustive and had several limitations.
Firstly, only a limited number of journals were considered (the majority of which were USA-based) rather than the entire pool of otorhinolaryngology literature. Furthermore, analysis of only 4 individual years within a 10-year period may have presented a somewhat less dynamic view of overall trends.
The second limitation was the consideration of journals based upon impact factor; this is regarded as an imperfect metric which reflects the popularity of a paper rather than specifically evaluating its academic rigour.Reference Seglen9 Akin to the analogy of ‘the rich getting richer’, articles published in journals with high impact factors are themselves more likely to be cited, making a high impact factor somewhat of a self-fulfilling prophecy. Furthermore, a high impact factor does not necessarily correlate with a large local readership or circulation of a journal; a further criticism is that the influence of local journals on local practice is ignored. Other metrics such as the Eigenfactor score and article influence factor may be more suitable in overcoming such bias.Reference Brown10
Thirdly, as with other, similar studies, this analysis did not make any distinction between clinical and academic publications.
Fourthly, multi-author publications now play a growing role, and this limits the usefulness of determining an article's geographical origin solely from the corresponding author's affiliation. Analysis of the leading 10 journals considered in this study showed a 33 per cent rise in the number of articles with authors from more than one region (from 11.7 per cent in 2000 to 15.5 per cent in 2010). The UK is likely to house smaller academic institutions than the USA or mainland Europe, and it may be the case that more UK academics are participating in research as contributors to studies hosted by overseas institutions. The UK Government Department of Business, Innovation and Skills report confirmed that the UK researcher population is highly mobile internationally: almost 63 per cent of researchers who were or had been affiliated with UK institutions had also published articles while working at institutions outside the UK.7
Finally, it is important to note that the quantity of published articles is neither the sole measure of research output nor necessarily correlates with the quality of research output.
• Surgical trainees face growing professional pressure to publish
• UK research output has reportedly declined in the general surgical and anaesthetic literature
• In this study, UK and Irish authors' share of otorhinolaryngology research output rose 22.8 per cent in the top 10 global journals but fell 28.6 per cent in the top 2 UK journals (2000–2010)
• Research articles remained the main type but their proportion declined (from 80.0 to 73.0 per cent; top 10 global journals)
• These encouraging results refute the fall in research output from UK and Irish authors observed by others
The future
While these results for the otorhinolaryngology literature are encouraging for the UK, it is important to consider what factors could prevent the decline observed in published UK research output among other specialties, in order to avoid a similar fate in otorhinolaryngology. As per the recommendations of the Modernising Medical Careers and Walport reports, increasing numbers of academic foundation programmes and academic clinical fellowships for specialty training may play an important role in integrating dedicated research time with clinical experience, producing ‘academic’ clinicians.11 As proposed in a Royal College of Surgeons report, a change in the surgical culture to accommodate research alongside ‘cutting time’ will be necessary, as will fostering greater collaboration between different surgical units.12 This report suggested that the present culture within the surgical profession was not always conducive to supporting research, with surgeons themselves not always effective advocates for surgical research and evidence-based surgery. With ongoing global economic uncertainty, it is inevitable that research budgets are likely to remain fragile for some years to come. Perhaps the UK will need to cautiously embrace the USA model and strengthen links between academic research institutions and commercial developers of drugs and technology, in order not only to reinforce research funding but also to help propagate research from bench to clinical practice. In the past, innovation, rather than robust clinical trials, has led to safer and less invasive surgical procedures with better outcomes. However, in an evidence-based world, further advances will require a change in mindset to appreciate that a strong research base with a focus on high quality research publications is crucial in order for further advances to reach clinical practice. Hopefully, UK otorhinolaryngologists will continue to ‘do their bit’ for UK research.
Appendix 1. Classification of article types
Case reports: included small case series (i.e. 10 cases or fewer).
Research: included all types of quantitative and qualitative clinical trials, laboratory research, retrospective studies, analysis of larger case series and demonstrations of new surgical devices or techniques.
Systematic review or meta-analysis: included articles citing over 100 references, featuring the words ‘review’ or ‘overview’ in the title, or appearing in the review section of a journal.
‘Other citable’: included certain special commentaries, debates and reports subjectively deemed to be citable but not fitting one of the above categories.
‘Other non-citable’: included types of articles deemed by Thomson Reuters to be generally excluded from external citation (such as letters and other correspondence, editorials, book reviews, conference abstracts, news, errata, and other journal ‘house-keeping’ articles). Articles included in the ‘other non-citable’ category were excluded from the analysis, as were articles from certain supplements (e.g. conference specials).
