Introduction
The anterior ethmoidal artery passes from the orbit to the anterior fossa of the skull through a canal within the ethmoid fossa.Reference White, Sincoff and Abdulrauf1,Reference Pernas, Coughlin, Hughes, Riascos and Maeso2 Injury to the anterior ethmoidal artery within this canal is a complication that can occur during functional endoscopic sinus surgery (FESS), and may result in significant morbidity, including intra-operative haemorrhage, vasospasm, retraction into the orbit and subsequent sight-threatening retro-orbital haematoma.Reference Pernas, Coughlin, Hughes, Riascos and Maeso2
The anterior ethmoidal artery canal may be contained within the roof of the ethmoid fossa (fovea ethmoidalis) in direct contact with the skull base,Reference Poteet, Cox, Wang, Fitzgerald and Kannan3,Reference Yang, Lu, Liao and Dang4 or it may be found below the skull base within a ‘mesentery’ of bone. The risk of anterior ethmoidal artery injury during FESS is significantly increased if the artery is exposed within a mesentery. Therefore, in rhinological practice, thorough pre-operative examination of a patient's paranasal sinus computed tomography (CT) images is routinely performed, with specific assessment of the anterior ethmoidal artery location.Reference Error, Ashby, Orlandi and Alt5 Indeed, locating the anterior ethmoidal artery is often part of checklists for pre-operative CT assessments, such as in the well-known mnemonic ‘CLOSE’, which stands for cribriform plate, lamina papyracea, Onodi cell, sphenoid sinus pneumatisation and (anterior) ethmoidal artery.Reference O'Brien, Hamelin and Weitzel6 Once the location of the anterior ethmoidal artery is identified, care is taken by the surgeon when addressing these areas surgically.
A number of techniques are used to assist in identifying the anterior ethmoidal artery on coronal CT images. One of the most popular methods is identification of the ‘nipple sign’, a triangular evagination of the lamina papyracea between the superior oblique and medial rectus extraocular muscles, corresponding to the anterior ethmoidal foramen, the entry point of the artery from the orbit into the nasal cavity.
Convolutional neural networks are a method of deep learning that represents a form of artificial intelligence, allowing computers to learn from large volumes of data by recognising patterns and making predictions. Convolutional neural network structure resembles the biological organisation of the visual cortex, where each convolutional layer applies an input and passes this onto the next layer. As a result, convolutional neural networks are well suited to, and have predominantly been used in, visual image recognition algorithms.Reference Yamashita, Nishio, Do and Togashi7
Recent advances in computational speed and power, and the availability of open source software, have facilitated artificial intelligence such as convolutional neural networks to grow in popularity and be applied to many healthcare fields. In radiology, learned convolutional neural networks have been demonstrated to have excellent diagnostic accuracy in identifying tuberculosis in plain chest radiographs, with an area under the curve of 0.99.Reference Lakhani and Sundaram8 In ophthalmology, deep learning tools have aided the assessment of images from digital photographs, optical coherence tomography and visual fields in various diagnoses such as cataracts, glaucoma and macular degeneration.Reference Du, Li and Hu9 In dermatology, convolutional neural networks have been found to identify malignant skin lesions from photographs, with similar accuracy to dermatologists.Reference Esteva, Kuprel, Novoa, Ko, Swetter and Blau10
However, the literature on the use of artificial intelligence such as convolutional neural networks in otolaryngology remains sparse, particularly in rhinology. This study aimed to identify the utility and accuracy of a trained convolutional neural network in differentiating an anterior ethmoidal artery that is either adhered to the skull base or hanging on a mesentery on pre-operative sinus CT scans using an image classification algorithm. Such an algorithm would have clinical application in automatically stratifying risk of injury to the anterior ethmoidal artery during FESS.
Materials and methods
Consecutive pre-operative fine-slice (0.6–1 mm) paranasal sinus CT scans, requested by a single tertiary rhinologist, and performed between 1st January 2015 and 31st December 2018 at two independent radiology practices, were retrieved from online practice databases (InteleViewer; Intelerad Medical Systems, Montreal Canada). Image request information and radiologist reports were also examined. A predefined proportion of the collected CT images were used as testing data rather than training data. Allocation of images to either training or testing was performed randomly using a computer random number generator.
Data recorded from each scan included patient age and sex, date of scan, and indication for imaging. Scans were excluded if: the patient was under the age of 18 years, there was evidence of previous sinonasal surgery, or surgery was performed for tumour resection or endoscopic skull base surgery.
A single coronal slice at the anterior ethmoidal foramen (Kennedy's nipple) was then extracted from each scan, captured in the ‘JPEG’ format (Adobe Photoshop; Adobe, San Jose, California, USA). The images were then cropped into an approximate 200 × 200 pixel snapshot, which included the lamina papyracea, anterior skull base and middle turbinate. Both left- and right-sided areas of interest were included. The right hemi-slices were flipped on the vertical axis to maintain consistency in the training data.
Two blinded ENT residents then examined each image to determine whether the anterior ethmoidal artery was adhered to the skull base or ‘suspended’ in a mesentery. Any disagreement in assessment was referred to the senior author (NS), a fellowship-trained rhinologist, for a final decision.
Convolutional neural network training and testing
The artificial intelligence algorithm was developed using the Inception-V3 convolutional neural network base model (Google, Mountain View, California, USA), trained with 1.28 million images. The classification layer was retrained in Python™ (programming language software) using a transfer learning method to interpret the coronal CT slices.
Test images were used to evaluate the performance of the algorithm. Images were randomly selected to be used for validation and test purposes. As reported previously, the neural network is designed to be trained for 100 000 steps at a learning rate of 0.05.Reference Chowdhury, Smith, Chandra and Turner11
Misclassifications of the convolutional neural network were of particular interest in this study. These images were identified and matched with the algorithm confidence levels. Subgroup analysis was performed to determine which image factors most predisposed images to be misclassified. Each misclassified image was reviewed by the two otolaryngology residents to assess for potential reasons for error.
Data analysis
Statistical analysis was performed using SPSS software, version 25 (SPSS, Chicago, Illinois, USA). Descriptive data were displayed as percentages for categorical data (sex, indication for scan, location of anterior ethmoidal artery), and means with standard deviations for continuous variables (age). Diagnostic accuracy data were presented as percentages with 95 per cent confidence intervals (CIs), and as receiver operating characteristic curves and area under the curve calculations. Assessment of agreement between the residents and the convolutional neural network was reported using the kappa statistic.
Results
A total of 675 images from 388 patients were used to train the convolutional neural network. Of these, 63.7 per cent (n = 430) had the anterior ethmoidal artery located at the skull base according to resident assessment. Data are summarised in Table 1.
Table 1. Summary of images classified by registrar or trained neural network
AI = artificial intelligence
A total of 197 images were used to test the convolutional neural network algorithm. Of these images, 67.5 per cent (n = 133) were assessed to have the anterior ethmoidal artery adhered to the skull base according to resident assessment, compared to 62.4 per cent (n = 123) by the convolutional neural network algorithm.
The overall diagnostic accuracy was 82.7 per cent (95 per cent CI = 77.7–87.8 per cent), which represented an area under the curve of 0.86 (Figure 1). Assessment of agreement between resident and convolutional neural network yielded a kappa statistic of 0.62 (95 per cent CI = 0.50–0.73), representing a moderately strong correlation.
Fig. 1. Receiver operating characteristic curve demonstrating the diagnostic accuracy of the trained algorithm in identifying the anterior ethmoid artery location. (Diagonal segments are produced by ties.)
Subgroup analysis of misclassified images
Twenty-three images were incorrectly classified as a suspended mesenteric anterior ethmoidal artery by the convolutional neural network. On review of these images, a large proportion of scans (n = 12) appeared to demonstrate a thickened supraorbital ridge (Figure 2), which the authors suggest may be the point of confusion for the artificial intelligence. Of the remaining scans, other potential causes of misclassification include dehiscence of the lamina papyracea (n = 2) or gross opacification of the surrounding ethmoid air cells (n = 2).
Fig. 2. Examples of cropped coronal computed tomography images for which the algorithm misclassified the artery as lying in a ‘mesentery’ that also demonstrated a thickened supraorbital ridge.
Eleven images were incorrectly classified as a skull base anterior ethmoidal artery by the convolutional neural network. Review of these images demonstrated no obvious potential causes of misclassification, although one image demonstrated gross opacification of the surrounding ethmoid air cells and another demonstrated lamina papyracea dehiscence.
Discussion
Deep learning artificial intelligence algorithms such as convolutional neural networks show great potential in multiple applications throughout healthcare. While many studies have previously demonstrated that convolutional neural networks may replace clinical roles with similar efficacy, particularly in diagnosis, the ability to recognise the potential pitfalls of surgery, such as the identification of the anterior ethmoidal artery in this study, presents a way for artificial intelligence and clinicians to work together to optimise patient outcomes.
• Deep learning convolutional neural networks are a form of artificial intelligence well suited for image classification
• Convolutional neural networks have been utilised with promising results in many medical fields; however, there is limited research in ENT
• The algorithm developed here demonstrated good diagnostic accuracy in differentiating anterior ethmoidal artery location at the skull base or hanging on a bone ‘mesentery’
• Anterior ethmoidal artery location is clinically important and often underreported in radiologist reports
• Deep learning algorithms may assist surgeons in operative planning, and in reducing complications and maximising outcomes
This study demonstrated that a trained convolutional neural network can locate the anterior ethmoidal artery with very good accuracy without prior knowledge, despite being given only a small amount of training data. The diagnostic accuracy of 82.7 per cent and area under the curve of 0.86 represent a good to excellent result according to the traditional academic point system for diagnostic tests. Nevertheless, with greater amounts of testing data, the algorithm would be reasonably expected to yield increased diagnostic accuracy.
Specific application of convolutional neural networks in determining the location of the anterior ethmoidal artery, while simple and often easily assessable by a rhinologist, still provides a useful additional source of diagnostic data for both reporting radiologists and surgeons. Deutschmann et al.Reference Deutschmann, Yeung, Bosch, Lysack, Kingstone and Kilty12 noted that the majority of otolaryngologists would like more clinically useful information in sinus CT radiological reporting and many were dissatisfied with current reporting standards. Furthermore, they found that anatomy of the anterior ethmoidal artery was reported in only 2 per cent of reports by a subspecialised neuroradiologist, and in 0 per cent of reports by head and neck radiologists and other radiologists. Implementation of the aforementioned convolutional neural network may therefore provide a safety net in case the surgeon makes an error in interpretation or if imaging scans are not available when the patient arrives at the operating theatre.
There are few reports of the use of artificial intelligence algorithms in rhinology in the literature. Chowdhury et al.Reference Chowdhury, Smith, Chandra and Turner11 trained and examined an image classification convolutional neural network to determine whether the ostiomeatal complex was obstructed, based on coronal CT slices. Using 956 images for training, their convolutional neural network achieved an accuracy of 85 per cent, and an area under the curve of 0.87 for ostiomeatal complex occlusion when compared to assessment by a fellowship-trained rhinologist. The slice chosen to represent the ostiomeatal complex was defined as the first slice through the natural maxillary ostium immediately posterior to the nasolacrimal duct. On examination of the misclassified images, the authors noted that in many cases classification could reasonably be argued in either direction.
It is the authors’ opinion that otolaryngology – head and neck surgery has great potential to benefit from developments in artificial intelligence, and presents several unique challenges from a data science perspective. One of the main reasons is the multiple modalities required in the diagnosis and management of otolaryngology pathology and presentation. For example, in the context of a patient who complains of nasal obstruction, clinicians are tasked with interpreting a number of subjective and objective measures to determine a clinically appropriate course of action, including a patient's subjective symptoms, patient-reported outcome measures, rhinomanometry, acoustic rhinometry, allergy testing and CT imaging. A patient with sudden sensorineural hearing loss on the other hand, requires assessment of the external auditory meatus and tympanic membrane, pure tone audiometry, tympanography, and CT imaging. The need to deal with these extremely heterogeneous data sources and fuse this information to form an accurate prediction certainly posits a trial for artificial intelligence algorithms. However, if successful, this has the potential to augment the often difficult diagnostic investigation of various otolaryngology diagnoses in collaboration with the clinician.Reference Baltrusaitis, Ahuja and Morency13
This study has several limitations. Firstly, the number of training images examined is small compared to the number used in other image classification algorithms in the published literature. However, when compared to image classification systems specifically in healthcare fields, the number of images assessed is comparable. Secondly, only images from a single coronal slice were used. Cadaveric studies have demonstrated that the morphological characteristics of the anterior ethmoidal artery course across the ethmoid roof vary. Therefore, an artery found to be adhered to the skull base on this particular slice does not necessarily mean that it does not appear to ‘hang’ elsewhere on the complete scan.
Conclusion
Trained convolutional neural networks appear to reliably classify the location of the anterior ethmoidal artery in two-dimensional coronal CT sinus images. This algorithm may assist surgeons by providing a safety net for surgeon interpretation, particularly in the context of deficient radiologist reporting. Future directions of artificial intelligence in otolaryngology include effective diagnosis based on data from multiple modalities and the potential to augment clinician-based assessment.
Competing interests
None declared
