Introduction
Adverse events in surgery are more likely to originate from failures in the non-technical aspects of performance. Fifteen per cent of hospital in-patients experience one or more adverse events, though approximately 50 per cent of these events are avoidable.Reference Vincent, Neale and Woloshynowych1 The Scottish Audit of Surgical Mortality reported that only 4.3 per cent of errors in surgery resulted from poor technical skills.2 Poor communication is a contributing factor in over 40 per cent of errors made during surgery, and over a quarter of legal claims result from cognitive errors occurring in operating theatres.Reference Gawande, Zinner, Studdert and Brennan3,Reference Wilson4
The importance of non-technical skills for improving safety within surgery is increasingly recognised, having developed from the airline and nuclear industries as well as other professions with a strong culture of safety within their workplace. The assessment and critique of surgeons’ and surgical teams’ non-technical skills and behaviours are becoming more common, and have been facilitated through the development of reliable and reproducible rating systems and the establishment of practical courses and online materials.Reference Yule, Flin, Paterson-Brown, Maran and Rowley5–Reference Crossley, Marriott, Purdie and Beard10
Most of the research regarding non-technical skills in surgery has occurred within general surgery, as this is the largest surgical specialty, and because the development and initial validation work of the Non-Technical Skills for Surgeons (‘NOTSS’) behaviour assessment tool was undertaken within that field.Reference Yule, Flin, Paterson-Brown, Maran and Rowley5,Reference Yule and Paterson-Brown11,Reference Yule, Gupta, Gazarian, Geraghty, Smink and Beard12 Research into the assessment and training of non-technical skills has developed in other surgical specialties, including orthopaedics, urology, cardiothoracic and neurosurgery.Reference Heaton, Little, Akhtar, Ramachandran and Lee13–Reference Hénaux, Jannin and Riffaud16
We performed a systematic review of the published literature to examine the research undertaken to date on the assessment of non-technical skills and behaviours within otolaryngology. The study aimed to assess the applicability of published studies to current practice, explore how these studies could guide future research and determine the areas where further development could be undertaken.
Materials and methods
A systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (‘PRISMA’) guidelines.Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche and Ioannidis17 Publications of every level of evidence and type of study were included. Literature searches were performed on 27th December 2019 by the authors using the PubMed, Medline, Embase and Cochrane Collaboration databases. The search terms used included: ‘otolaryngology’, ‘otorhinolaryngology’, ‘ENT’, ‘ENT surgery’, ‘ear, nose and throat surgery’, ‘head and neck surgery’, ‘thyroid surgery’, ‘parathyroid surgery’, ‘rhinology’, ‘otology’, ‘laryngology’, ‘skull base surgery’, ‘airway surgery’, ‘non-technical skills’, ‘non technical skills for surgeons’, ‘NOTSS’, ‘behavioural markers’ and ‘behavioural assessment tool’. Articles of all languages were included in the review.
Results
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart for the systematic review is summarised in Figure 1. Three publications were included in the review: one randomised, controlled trial and two cohort studies.Reference Chang, Chou, Lin, Huang, Shih and Wu18–Reference Wu, Kim, Fung and Roth20 All were simulation-based studies, involving 78 participants in total. The results of the studies are summarised in Table 1.
Fig. 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (‘PRISMA’) flow chart.
Table 1. Results from articles included in systematic review
NOTSS = Non-Technical Skills for Surgeons behaviour assessment tool; RCT = randomised, controlled trial; ANTS = Anaesthetists’ Non-Technical Skills behaviour assessment tool
Wu et al. discussed the application of the Non-Technical Skills for Surgeons tool for evaluating otolaryngology residents’ non-technical skills during simulated otolaryngology emergency scenarios, including facial trauma, penetrating neck injury, post-thyroidectomy haematoma development and paediatric foreign body aspiration.Reference Wu, Kim, Fung and Roth20 Independent raters used the tool whilst viewing video recordings of the participants during the scenarios, generating mean scores for each domain and element of the Non-Technical Skills for Surgeons tool. The domains and the elements were then ranked in order of their recorded scores.
Chang et al. reported on the use of the Anaesthetists’ Non-Technical Skills (‘ANTS’) behaviour assessment tool in assessing various professionals’ skills involved in the intra-hospital transfer of critically ill patients.Reference Chang, Chou, Lin, Huang, Shih and Wu18 The participants included emergency medicine physicians, respiratory therapists, otolaryngology residents and nurses. Groups of these professionals all received a 2-hour teaching session on the subject of patient transfer within hospital and underwent a pre-test simulated scenario. Groups were then randomised into subgroups that either received or did not receive additional in situ inter-professional, simulation-based training prior to participating in a further scenario, where scores for the Anaesthetists’ Non-Technical Skills tool were used as markers of effectiveness of the additional training.
Sahovaler et al. used both the Non-Technical Skills for Surgeons tool and a scenario-specific Medical Expert Checklist tool to assess video recordings of otolaryngology residents’ management of post-operative haemorrhage from the tongue base area following transoral surgery.Reference Sahovaler, Eibling, Bruni, Duvvuri, MacNeil and Nichols19 The mean scores for the Non-Technical Skills for Surgeons tool were recorded for the domains and individual elements and then ordered according to the recorded scores as an outcome measure. The overall mean scores of the five individual components of the Medical Expert Checklist were also used as an outcome measure for participants’ non-technical skills.
Discussion
Although otolaryngology is one of the largest surgical specialties globally, comparatively little research has been undertaken on the non-technical skills of doctors working in this field, or on the applicability and potential benefits to the specialty. The work that has been undertaken has been performed in simulated environments, in either single institutions or on national otolaryngology emergency courses for trainee surgeons.
However, useful conclusions can be made from this initial work. Otolaryngology trainees appear to have good skills relating to ‘situation awareness’, and can gather and understand information quickly and thoroughly. Whilst this is important for all emergency presentations, it is especially important for those involving patients’ airways, given their time-critical nature, particularly in children. The good skills demonstrated in anticipating how situations will develop and preparing for these are important in resolving emergency situations promptly, and can reassure other health professionals and trainees’ senior colleagues. The mean scores reported by Wu et al.Reference Wu, Kim, Fung and Roth20 and Sahovaler et al.Reference Sahovaler, Eibling, Bruni, Duvvuri, MacNeil and Nichols19 show that trainees’ leadership skills were marginally better than decision-making, communication and team-working (2.5 vs 2.3). Although determinations are difficult to make based on only two studies, these may be areas where further training and encouragement are required for this group of otolaryngologists. However, these are skills that do improve as progress is made through training programmes and as doctors’ general experience grows.Reference Gostlow, Marlow, Thomas, Hewett, Kiermeier and Babidge21
The review also suggests that behaviour assessment tools other than the Non-Technical Skills for Surgeons tool may have a role in ascertaining otolaryngologists’ non-technical skills. The use of the Anaesthetists’ Non-Technical Skills behaviour assessment tool, as reported by Chang et al.,Reference Chang, Chou, Lin, Huang, Shih and Wu18 is arguably more suitable for assessing skills during those airway emergencies or intra-operative airway complications that contribute to an otolaryngologist's workload. This would be as relevant for those who encounter such issues both regularly and only on occasion, based on the nature and extent of individuals’ specialisation. It could also be relevant for those working in centres with a heavy burden of craniocervical trauma.
Assessing non-technical skills appears to be straightforward, although arranging additional training for multiple raters, and providing video-recording facilities to improve score validity and eliminate observer bias, is costly and time-consuming. These factors may dissuade others from performing further research in this area. However, the importance and growing awareness of non-technical skills does offer a convincing argument for resources to be provided for this work, especially given the comparative lack of evidence in otolaryngology.
A number of articles that were rejected during the Preferred Reporting Items for Systematic Reviews and Meta-Analyses selection process do discuss the assessment of non-technical skills during simulation-based training courses aimed at otolaryngology trainees. These articles were excluded because of insufficient information on assessment outcomes; however, their content suggests that the observation of surgeons’ non-technical skills in otolaryngology – head and neck surgery is possibly more widespread than the results of this systematic review suggest. Post-scenario debriefings, where participants’ demonstration of non-technical skills are discussed and their preparation for emergency airway interventions are assessed, were reported by Amin and FriedmannReference Amin and Friedmann22 and by Hogg et al.Reference Hogg, Kinshuck, Littley, Lau, Tandon and Lancaster23
Nguyen et al. describe 20-minute debriefing sessions following each of 19 different airway simulation scenarios, where trainee surgeons reflect on their reasoning for particular courses of action and their decision-making.Reference Nguyen, Bank, Fisher, Mascarella and Young24 These debriefing sessions are enhanced by the viewing of video recordings made of participants during the scenario.
Hall et al. report on the training and assessment of trainee surgeons’ non-technical skills during a simulation-based laser surgery and laser safety course.Reference Hall, Okhovat, Milner, Montgomery, Hitchings and Kunanandam25 Participants’ knowledge, use and correct completion of a pre-operative laser safety checklist were assessed, as was their management of an airway fire secondary to laser use. These articles do provide a useful example of how to assess non-technical skills during a practical course, and demonstrate how otolaryngologists can design and construct scenarios where the aim is to exclusively assess such skills.
Further work in two aspects of non-technical skills in otolaryngology would be useful. First, assessing senior surgeons’ skills would offer this group the opportunity to reinforce areas of strength and provide training in areas where improvements may be required. Yule et al. identified that surgical consultants from five different specialties had higher scores on the Non-Technical Skills for Surgeons tool (and thereby greater non-technical skills) than reference ratings from trainees for six different scenarios.Reference Yule, Rowley, Flin, Maran, Youngson and Duncan8 It is important to determine whether senior otolaryngologists’ skills reach the same level as those in other specialties and to establish points of learning from other fields of surgery. Non-technical skills assessment can also contribute to surgeons’ lifelong learning and appraisal processes.
Second, the ‘boot camp’ model of otolaryngology training, whereby established trainers and more junior trainees interact in a simulated emergencies training environment, has the potential to be adapted for non-technical skills assessment for both groups. Such boot camps are well-established in the UK and throughout the world. These models have proven to be adaptable enough to include new techniques of training and assessment for different levels of experience.Reference Chin, Chin, Roth, Rotenberg and Fung26–Reference Kovatch, Wertz, Carle, Harvey, Bohm and Thorne28
Assessing non-technical skills during actual, non-simulated surgery has demonstrated benefits in hepato-biliary and pancreatic surgery, and in transplant surgery, as it can identify issues that would not have manifested during testing in a simulated environment.Reference Siu, Maran and Paterson-Brown29 This has particular potential during the longer and more demanding operations undertaken in otolaryngology; for example, skull base surgery, or head and neck oncological resections with free tissue transfer reconstruction, particularly as the skills of multiple operators could be assessed and constructive feedback provided to improve overall surgical performance.
Conclusion
Modern surgical practice requires technical and non-technical skills. The work already performed within otolaryngology provides an indication on the level of non-technical skills, as well as guidance on how to perform further research in this and other areas that would benefit from further study.
Competing interests
None declared
