Introduction
Surgical training in the UK has undergone significant reform, particularly in the last decade following the introduction of the Modernising Medical Careers scheme and European Working Time Directive. These changes have resulted in the reduction of training opportunities for those in Higher Surgical Training programmes, an issue which has been widely discussed.Reference Morris-Stiff, Sarasin, Edwards, Lewis and Lewis1–Reference Tami4
There is now growing concern over the standards of UK Higher Surgical Training programmes amid the current work-based assessment competency system. Significant challenges and responsibility are placed on the trainee to be sufficiently experienced to become a consultant.Reference Black2
Laryngology and phoniatrics is an important yet often overlooked subspecialty within otolaryngology surgical training programmes. Trainees have stated that specific laryngology training in the UK is not always as comprehensive as the other subspecialties.
The awarding of the Certificate of Completion of Training requires trainees to be trained in all the ENT curriculum subjects, including otology, neurotology and skull base surgery, paediatrics, benign head and neck surgery, head and neck oncology, rhinology and sinus surgery, and facial plastic surgery. Although laryngology is encompassed within several of the subjects, there is no indication within the syllabus as to how much training time should be attributed to each subject.5
Within the current UK Higher Surgical Training programmes, workplace-based assessments are the only online assessment tool used in the curriculum. Under the current syllabus for trainee otolaryngologists, trainees are graded 1–4 on all workplace-based assessments according to their level of competence. Level 4 competency indicates that the trainee is judged to be at Certificate of Completion of Training level by their trainer. From the list of key conditions for which a trainee is required to have level 4 competency, only 3 out of 14 (21 per cent) are laryngology-related topics.6
The ability to perform flexible laryngoscopy and diagnostic adult microlaryngoscopy procedures features under the required technical skills for the awarding of the Certificate of Completion of Training, but no quantitative logbook values for procedural numbers are stated. Prior to the Modernising Medical Careers (MMC) programme, the curriculum stipulated that trainees perform at least 10 microlaryngoscopy procedures as the principal surgeon. This is no longer the case, but procedures such as major neck resection and septorhinoplasty currently have recommended logbook numbers for the Certificate of Completion of Training. It is well known, however, that significantly more new consultants will be expected to perform flexible and direct laryngoscopy on appointment, which are the absolute core skills, as opposed to septorhinoplasty or neck dissections, which are considered by many to be subspecialist procedures. Regardless of the subspecialty that a given trainee may pursue, diagnostic competency in laryngology is a prerequisite for both the Certificate of Completion of Training and clinical practice as a consultant.
The European Working Time Directive placed limitations on the working hours of surgical trainees.Reference Morris-Stiff, Sarasin, Edwards, Lewis and Lewis1–Reference Tami4 Trainees and trainers are now required to come up with innovative ways to provide coverage of the wide range of clinical skills and experience, whilst working within the new time constraints. There is a conflict for trainees between this undertaking and the necessity for service provision.Reference Vasani, Whittaker, Sharma, Wong, Patel and Choa3, Reference Tami4, Reference Pothier, Toll, Grant and Giddings7, Reference Varley, Keir and Fagg8 As a result, time spent with a trainer in laryngology and voice clinics will often be compromised.
Online testing has become a useful tool for assessing a wide range of trainees across a large geographic region. The use of video-based questions allows for a dynamic and realistic assessment of diagnostic skills based on what trainees will regularly see during clinical examination. An advantage of online curricula and assessment is that a large geographical area can be reached and potentially examined. Another advantage is the inclusion of less common pathologies, which may not present themselves to the trainee during a given period of their training.
This study aimed to ascertain the adequacy of laryngology training in UK Higher Surgical Training programmes for otolaryngology. An additional aim was to assess the correlation between years of clinical experience, specialty-specific experience and clinical diagnostic ability in laryngology.
Materials and methods
An online test was designed using the html-based SurveyGizmo survey software tool.9 The test was targeted at all otolaryngologists, with a particular focus on training grade registrars. Demographic data were collected during the test, which included: responder deanery; grade; level of experience in a Specialist Advisory Committee approved post in both laryngology and head and neck surgery; and overall experience in the specialty.
The test featured 20 embedded digital videos that had been filmed during laryngoscopy procedures carried out for a wide range of benign and malignant laryngeal pathologies. All pathologies were present in the UK training syllabus and none were rare.
The videos contained no patient details or data. The patients involved had given consent for their videos to be used for teaching purposes. The answer format was free text, in order to avoid any guessing. The first question in each section was diagnostic, and the second question related to an aspect of management for the condition demonstrated in the video.
The test was circulated via several forums including the Association of Otolaryngologists in Training and the Doctors.net.uk online user community. Participants were able to complete the test at any computer, in their own time. Examination conditions were advised but not enforced. Participants were provided with e-mail support for website use.
A marking scheme and schedule was devised by the senior investigator of this project, who is recognised as an expert within the field of laryngology. Results were collected electronically for analysis using the ‘R’ software package, and run with the Statistical Package for the Social Sciences software. Where multiple t-tests were used on the same dataset, Bonferroni corrections were applied (this is reflected in the quoted p values shown below).
Results and analysis
There were a total of 126 responses. Fifty-five responders answered all questions, 51 of whom regarded otolaryngology to be their primary field of expertise. The remaining 74 responses (partial, incomplete and non-otolaryngology responses) were excluded from further analysis for this study. Cronbach's alpha for inter-item reliability of the test scores was 0.91, indicating that the inter-item (question) score variance was congruent.
Responders represented 13 deaneries. The grades of responders ranged from core surgical trainee year two to consultant; 81 per cent of responders were training grade registrars (Figure 1). The mean test score was 54 per cent; the highest and lowest scores were 95 per cent and 18 per cent respectively. The ranked scores failed a normality test (the D'Agostino–Pearson test), indicating that the results were non-parametric without normalisation. Only time in Specialist Advisory Committee approved posts counted towards experience levels.
Fig. 1 Pie chart showing the levels of all included participants. CT = core trainee; ST = specialist trainee; SpR = specialist registrar
When the average scores of each deanery were compared, there was a trend towards a difference, although this difference was not significant (p > 0.05, Kruskal–Wallis test (non-parametric analysis of variance)). The number of participants varied between deaneries, with London providing the most responses. There was, however, a significant difference in scores between the highest and lowest scoring deaneries (p < 0.05, Mann–Whitney U test (all Mann–Whitney U tests were carried out with a Bonferroni correction if applicable)) (Figure 2).
Fig. 2 Deanery scores anonymised and ranked according to mean scores and standard error.
A graph (Figure 3) was compiled to compare scores with experience. This demonstrated a weak positive correlation between months of experience and test scores (p = 0.02, Spearman's correlation). However, when the consultants and senior house officers (SHOs) were removed, there was no correlation between scores and overall ENT experience (p > 0.05, Spearman's correlation). This indicates no significant improvement in scores for the registrar level trainees, despite their level of overall experience (Figure 4).
Fig. 3 All participant scores plotted against number of months of ENT experience.
Fig. 4 Registrar grade scores plotted against number of months of ENT experience.
Consultants scored higher than both SHO level and junior registrars (specialty trainees, years 3–5) (p < 0.05, Mann–Whitney U test). However, there was no difference between the scores of junior registrars and senior registrars (specialty trainees, years 6–8), or senior registrars and consultants (Mann–Whitney U test) (Figure 5). Three out of eight consultants identified laryngology as their primary field of expertise by indicating they had five or more years of laryngology experience.
Fig. 5 Mean test scores for all grades of respondents. ST = specialist trainee
There was a significant improvement in test scores for trainees who had completed six or more months of approved laryngology training (p < 0.001, Mann–Whitney U test). Although there was an apparent positive correlation with laryngology experience, the difference was not significant beyond 12 months of experience (Mann–Whitney U test). These findings demonstrate a significant improvement in scores after 6 months of laryngology experience, followed by a smaller increase after 12 and 24 months of experience, at which point the scores began to plateau (Figure 6).
Fig. 6 Trainee scores plotted against number of months of Specialist Advisory Committee approved laryngology experience.
Trainees with 12 months of head and neck experience demonstrated significantly better scores than those with only 6 months of head and neck experience (p < 0.05, Mann–Whitney U test). There was no difference in the scores of participants with 12 months versus 24 months of head and neck experience, or between those with 6 months of head and neck experience and those with no experience (p > 0.05, Mann–Whitney U test) (Figure 7). However, when trainees with six months or more of laryngology training were removed from the analysis of head and neck experience, there was no correlation or demonstrable improvement in score (Mann–Whitney U test). Trainees with equivalent head and neck experience who had not completed at least six months of laryngology training in a Specialist Advisory Committee approved post had significantly lower scores than those who had this experience (Figure 8).
Fig. 7 Trainee scores plotted against number of months of Specialist Advisory Committee approved head and neck experience. H&N = head and neck
Fig. 8 Graph showing scores of trainees with equivalent head and neck experience, who had at least six months of laryngology experience (purple) or no laryngology experience (blue). H&N = head and neck
Consultant scores were then used as a control to determine the factors that give trainees comparable scores to consultants, using a Mann–Whitney U test. The experience factors that led trainees to obtain scores that were statistically similar (p > 0.05, Mann–Whitney U test) to their consultant trainers were: 12 months of laryngology, or a combination of at least 6 months of laryngology and 12 months of head and neck surgery experience (Figure 9).
Fig. 9 Graph demonstrating trainee groups with scores comparable to the consultant group (red), and groups with significantly lower scores than the consultant group (blue). M = months; H&N = head and neck; laryn = laryngology
Trainees who had 24 months' experience of head and neck surgery also showed comparable results to the consultant group (p < 0.05, Mann–Whitney U test). However, when participants with six months or more of laryngology experience were excluded, the trainee scores were significantly lower than those of the consultant group (p < 0.05, Mann–Whitney U test) (Figure 9).
Trainees who had any combination of experience that amounted to less than 12 months of laryngology training, or less than a combination of 6 months of laryngology experience and 12 months of head and neck experience, demonstrated significantly lower scores than the consultant group (p < 0.05, Mann–Whitney U test) (Figure 9).
Limitations
This study assessed trainees with a wide range of experience, and was represented by trainees from all deaneries within England and Wales. There was a higher representation from the London and the south east UK deaneries. Twenty of the 51 (39 per cent) responders included in the study were from the London Deanery, which is an over-representation when one considers the number of new appointments last year (7 of 32 appointments, 22 per cent).10
The test was mainly distributed via national online forums. The Association of Laryngologists in Training forum has 700 members, although the association themselves have pointed out that there are some duplicate and inactive e-mail addresses on their mailing list. Doctors.net.uk has over 190 000 members, but only a fraction of these read the ENT forum and no official data are available. The Association of Laryngologists in Training forum generated 116 separate responses (prior to exclusions), giving a response rate of 16 per cent. The response rate from Doctors.net.uk was less than 1 per cent. Postal invitations or individual e-mail invitations may yield higher response rates.11
There is evidence to suggest that response rates as low as 20 per cent can show data as accurately as higher response rates.Reference Holbrook, Krosnick, Pfent, Lepkowski, Tucker, Brick, de Leeuw, Japec and Lavrakas12 A larger follow-up study, with invitation via the programme directors of each deanery, may provide a more representative outcome.
The control group or standard in this study was consultant otolaryngologists. Clearly assumptions have to be made about this group's level of ability and the adequacy of their diagnostic knowledge. Although this study demonstrates that as a group the consultants scored the highest when compared with participants of more junior levels, it could be that they themselves are below or above the desirable standard. Once again, a more encompassing study could provide answers as well as discriminators to ascertain whether this group is at an ideal standard.
Discussion
Despite best efforts, Higher Surgical Training programmes are on the whole heterogeneous throughout the UK when it comes to laryngology experience. This is reflective of a system that needs flexibility to ensure a balanced consultant workforce. It is essential, however, that consultant level otolaryngologists possess adequate core competency both for appointments and for participation in the general on-call rota.
Our data indicate a significant difference between the highest and lowest scoring deaneries. This suggests that experience throughout the UK may not be homogeneous. A sample of the lowest scoring deanery did not appear to have completed any specific voice or laryngology training, despite having similar levels of overall ENT experience. Further investigative work may reveal whether any specific laryngology or voice training is on offer in the lower scoring deaneries. The response rates from the largest and smallest deaneries were in proportion to the number of new appointments in those deaneries. There is still, however, a possibility that some deaneries may have a non-representative sample due to their size. A larger sampling of both UK trainees and consultant otolaryngologists would provide clearer indications as to the standards within each deanery, as well as the opportunities for experience.
The results of this study demonstrate a weak positive correlation between overall ENT experience and the diagnostic ability of otolaryngologists. However, when only registrar level participants were sampled, no correlation was demonstrated between overall experience and test score. It seems natural that experience plays a role at the lower and higher ends of training, but during the training years, more specific experience is required to perform well within this field and (more importantly) to match the expertise of their consultant trainers.
The Specialist Advisory Committee has made recommendations regarding experience in subspecialty clinics, and suggests a rotation through all of the main subjects within the ENT curriculum.5 The list of specialties includes voice, but no quantitative figure has been placed on the length of experience required. It might therefore be assumed that some trainees are required to learn this specialty as part of their skills training in head and neck surgery, which suggests a degree of overlap. The results in this paper indicate that this assumption is false. The findings suggest that head and neck experience alone does not provide trainees with adequate laryngology-related diagnostic skills compared with consultants. Even trainees with 24 months of head and neck experience did not achieve equivalent scores to consultants in the absence of 6 months of laryngology experience.
There was a clear correlation between specific experience in laryngology and test score. This was highlighted by the fact that scores of trainees with 12 months of laryngology experience were comparable to those of consultant trainers. A combination of 12 months of head and neck surgery and 6 months of laryngology experience also provided scores that were statistically similar to consultants' scores.
The realisation of six months' laryngology training may be difficult for many trainees to achieve given the reduction in hours and training opportunities associated with the many reforms that have taken place within surgical training over the last decade.Reference Vasani, Whittaker, Sharma, Wong, Patel and Choa3, 6, Reference Chikwe, de Souza and Pepper13
• There was a difference in laryngology-related diagnostic skills of trainees nationwide
• There is no quantitative recommendation in the current curriculum regarding voice and laryngology experience
• Head and neck experience alone does not seem to provide trainees with adequate laryngology-related diagnostic skills
• There was a clear correlation between specific experience in laryngology and test score
• Trainees should undergo at least six months of laryngology training within their programme
The authors of this paper recommend that trainees undergo at least six months of laryngology training within their programme. We suggest that this includes experience in at least one specialist voice clinic, a microlaryngoscopy operating theatre schedule, and attendance at a head and neck cancer multidisciplinary team meeting which deals with transoral resection. This experience could potentially be combined with a general, or head and neck placement. Trainees that wish to further their training can be offered more exposure as can be allowed by their respective deanery.
Conclusion
Laryngology-related diagnostic ability is an essential requirement for the Certificate of Completion of Training. Higher Surgical Training otolaryngology programmes in the UK are heterogeneous in terms of the time that is devoted specifically to laryngology, and many programmes seem to be incorporating this aspect into their head and neck training. The results demonstrate no correlation between overall experience and laryngology-related diagnostic ability within the registrar level trainee group. We recommend at least six months of specialist laryngology experience as part of the curriculum in order to provide trainees with adequate grounding in laryngology diagnostics.
Acknowledgement
The authors would like to thank Mr Suki Aluwhalia, the Association of Laryngologists in Training president, for his help regarding the distribution of the study test.
