Introduction
There were more than 22 000 emergency admissions for epistaxis in England in 2014–2015,Reference Hall, Blanchford, Chatrath and Hopkins 1 accounting for 39 000 bed-days at a cost of more than £3.5 million.Reference Lowe, van der Meulen, Cromwell, Lewsey, Copley and Browne 2 Whilst admissions for epistaxis are common (accounting for more than a third of all emergency ENT admissions during the year), a recent audit of six hospital sites showed considerable variation in management of the condition.Reference Hall, Blanchford, Chatrath and Hopkins 1 However, given the small size of that audit, meaningful conclusions about regional variations in practice could not be drawn. It is essential to reduce unwarranted practice variation to ensure that patients with epistaxis receive the same high-quality and evidence-based management, regardless of which hospital they attend or doctor they consult.
As a first step to improving the management of epistaxis, a large national audit of in-hospital epistaxis management would allow quantification, and a greater understanding of the regional disparities in care provided. This method of quality improvement has been demonstrated by the National Prospective Tonsillectomy Audit in England and Northern Ireland, and led to the issuance of national guidance and a subsequent reduction in complication rates.Reference Lowe, van der Meulen, Cromwell, Lewsey, Copley and Browne 2 , Reference Audit 3
However, there are several barriers to delivering a successful nationwide audit. There are few bridging structures between the senior researchers, who have traditionally orchestrated projects on this scale, and the junior doctors, who provide the mainstay of local treatment for epistaxis. In addition, variations in regional patient management pathways mean that patients attending one hospital may be transferred and managed at a second site. This may lead to lower case-ascertainment, as was shown in the national tonsillectomy audit. 4 Unlike elective procedures, such as tonsillectomy, epistaxis patients cannot be pre-identified, as admissions are unplanned. This requires a robust system for prospective case identification and reporting, as data collected retrospectively are prone to recall bias and selective reporting because of poor documentation.
These issues are amplified by the need for national involvement. Traditionally, multicentre audits have relied on collecting data with paper case report forms. Information from these forms is then transferred onto a central database, either by sending the original forms to the co-ordinating centreReference Hall, Blanchford, Chatrath and Hopkins 1 or locally uploading them using a secured internet connection. 4 This method increases data transfer points, which reduce data quality.Reference Nahm, Pieper and Cunningham 5 In addition, the process is time-consuming, expensive and has low user ratings.Reference Le Jeannic, Quelen, Alberti and Durand-Zaleski 6 Whilst electronic case record forms circumvent some of these issues, they have their own drawbacks, including: the need for on-site technology; assistance by information technology (IT) staff or software providers; the complexity of the installation and maintenance of the software; and the high upfront investment cost.Reference Welker 7 For all of the above reasons, the final barrier to co-ordinating a national study is high cost, with the National Prospective Tonsillectomy Audit costing over £250 000. As medical research funding has decreased over the last six years, 8 funding for quality improvement projects is now extremely scarce.Reference Ham, Berwick and Dixon 9
To overcome the barriers to delivering this project, we investigated the feasibility of a web-based interface to capture patient data, led and delivered by a multi-region trainee collaborative.
Materials and methods
Site lead recruitment
In order to ensure national representation, site leads were recruited from established ENT trainee research collaboratives across England to collect and submit data from a single hospital. The induction of site leads included compulsory good clinical practice training to ensure that local study conduct met the high research standards.
Site initiation
Site leads were responsible for ensuring the audit was locally approved and appropriately registered, and that the clinical team were aware and engaged in the audit. Local clinicians were encouraged to familiarise themselves with the audit tool to ensure that relevant data were captured during the course of the clinical encounter.
Data collection tool development
As a standard of care for epistaxis management has yet to be established, the preliminary case report form was designed around the framework of the most recent review on epistaxis management.Reference Nahm, Pieper and Cunningham 5 Amendments were made following review from the executive board of the British Rhinological Society. The questionnaire that resulted from these changes was piloted on 11 specialist trainees, to ensure it balanced depth of information gathered with efficiency of use. The final questionnaire was based on branching logic so that follow-up questions appeared depending on the answers given to preliminary items.
Data capture and storage
We created a system that used a simple questionnaire-style, secure web interface, obviating the need for local software installation. Data were uploaded onto an encrypted central server housed and maintained within a university IT system; all the required infrastructure was in place to maintain the integrity of the website and data quality.
Audit items were captured locally using the Research Electronic Data Capture (‘REDCap’) tool. This tool was designed to comply with the Health Insurance Probability and Accountability Act of 1996. It allows data input from anywhere in the world, with secure web authentication, data logging and Secure Sockets Layer encryption. The Research Electronic Data Capture tool provides user-friendly web-based case report forms, whilst creating data audit trails.
Data collated from the study were sent to the University College London secure server. This server is certified to ISO27001 information security standard and has level 2 compliance for the National Health Service information governance toolkit. The network is built using a ‘walled garden’ approach, which allows the data to be stored, processed and managed within the security of the system. This system avoids the complexity of assured end-point encryption, and prevents unauthorised handling, manipulating or transferring of data. A file transfer mechanism enables information to be sent into the system simply and securely. The Research Electronic Data Capture tool installation within the University College London secure server ensures data safety from point of entry onwards.
The use of a secure web-based interface and the University College London secure server greatly reduced the running cost of data capture, whilst maintaining the highest standards of information governance.
Pilot design
In-hospital ENT management of epistaxis was audited prospectively across six hospital sites (Guy's and St Thomas’ Hospitals, London; Nottingham University, Torbay and South Devon NHS Foundation Trust, Torquay; North Cumbria University Hospitals NHS Trust, Carlisle; Great Western Hospital NHS Trust, Swindon; and Shrewsbury and Telford Hospital NHS Trust, Shrewsbury).
The audit included all patients admitted during a two-week period starting on the 1st May 2016. Site leads liaised daily with on-call teams to identify current in-patients with epistaxis and to remind the department of the ongoing audit. Site leads were asked to upload data on the day the clinical encounter ended (when the patient was either discharged or transferred). A site-specific patient code key was developed, so that patient identifiable information could be held locally, and a unique patient identification code for the audit was uploaded with each case report form. This allowed readmission and mortality data, collected 28 days after the close of the audit, to be assigned to the relevant patient.
The International Classification of Diseases 10th revision (‘ICD-10’) codes for epistaxis (R04.0, R04.8, R04.9) were used to identify all patients admitted with epistaxis during the audit window. This was cross-checked against admitted patients to ensure patients were not missed.
For the purposes of this pilot, the lead author monitored the database to detect any issues with data entry or quality.
Site-lead feedback
Once the pilot study had ended, site leads were sent a structured feedback form. The form included the following subheadings: audit registering, audit advertising, ease of website access, patient identification, time between patient discharge and data upload, time taken to upload patient data, fields felt to be ambiguous, fields felt to be irrelevant, fields difficult to complete, study ownership and other suggestions. The site leads were also asked to rate their satisfaction with the pilot audit on a five-point Likert scale and to list the parts they enjoyed the most.
Results
Recorded cases
Sixty-three patients requiring ENT management of epistaxis were admitted across the six pilot sites over the two-week period (Table I). The mean patient age was 64.7 years (range, 4–90 years) and 53 per cent were male. Mean Modified Early Warning Scores and haemoglobin levels on admission were within the commonly used normal ranges, with little regional variation (score of 1.02 and 12.48 g/dl, respectively). Forty-six per cent of patients admitted were on anticoagulants and 55 per cent of these patients had their anticoagulant agents reduced or stopped.
Table I Patient data for each pilot site
SD = standard deviation; MEWS = Modified Early Warning Scores
Forty-nine per cent of patients were managed without nasal packing, with large variation from centre to centre (23–100 per cent). Sixty-two per cent of patients were cauterised, with large variation in rates between centres (18–100 per cent). The mean duration of hospital stay was 1.65 days (standard deviation = 1.49), with a range of 0.81–6.33 days. There were no re-admissions or mortality within 28 days of discharge. A search of International Classification of Diseases 10th revision codes in the audit period did not identify any new patients that had been missed.
Collaborator feedback
Site leads generally found local registration of the audit easy. However, one site lead reported a delay in registration due to infrequent scheduling of audit approval meetings. They recommended at least a one-month lead to register the national audit prior to the start of the data collection period.
The majority of site leads found advertising the pilot to the clinical team straightforward and very helpful. Two site leads reported difficulty as the pilot spanned a doctor changeover period, and they recommended avoiding these times. One site lead suggested that standardised posters advertising the project would be helpful.
Five of the six site leads had no issues accessing the data capture website; one centre reported restrictions from the local trust firewall, blocking access to the website. The local IT department was able to lift the restriction, but this resulted in a 48-hour delay in uploads from this centre. This centre recommended that the local IT department be sent the website address in advance to ensure it is not blocked by firewalls.
Site leads reported that the case report forms were intuitive, and took between 4 and 10 minutes to complete per patient. Whilst all site leads found data entry easy, they all reported that poor clinical documentation led to certain fields being left unanswered. This was most abundant in admission observations, which had 22 per cent missing data. One site lead suggested advertising variables to be measured prior to the audit start to improve documentation, whilst another suggested having a proforma that can be attached to the medical notes.
Nearly all site leads uploaded information within 24 hours of discharges that occurred during weekdays, and within 72 hours for those that occurred at weekends. However, one site lead had a one-week delay in data upload for more than half of their patients; they reported taking leave in the middle of the pilot recruitment period. Two site leads suggested that ample notice of the audit period should be given to prevent absence and delayed upload.
The lead author monitored data submission for quality, and noted errors in unique patient identifiers, with either replication or non-sequential entry. When site leads were probed about this, they reported difficulty monitoring audit-specific patient identifiers on the paper code key provided. One suggested that having access to previously uploaded patient records might prevent duplication of audit-specific patient codes, whilst another suggested that developing a unique code based on pre-existing patient identifiers may help.
When site leads were asked about their satisfaction with the questions, two mentioned that reporting on blood results was time-consuming and they questioned its relevance to the management of epistaxis.
Site leads suggested being able to review submissions daily to ensure an extra layer of data quality assurance. All site leads reported high satisfaction with the pilot audit (mean, 4.83 out of 5). When asked which parts in particular they enjoyed the most they reported, ‘local leadership’, feeling of ‘ownership for the project’ and ‘being part of a meaningful project’.
Discussion
A pilot study was devised and completed to investigate the feasibility of a national audit of epistaxis management using a web-based interface and secure server, and involving multiple local trainee collaboratives. The results suggest that whilst the patients presenting to departments are similar in terms of age, sex, and Modified Early Warning Scores and haemoglobin levels at admission, there is wide variation between departments in the treatments offered and the length of hospital stay. This will be investigated in greater detail in the full national audit.
We have shown that a study led and delivered by trainees results in high local investigators’ ownership of the study. Furthermore, as epistaxis management is provided by trainees, there is the additional benefit of data extraction being closely tied to the clinical contact. Both of these factors may have contributed to the high data quality seen in this pilot.
There were no issues with registering the pilot with local audit departments or publicising it within the ENT teams locally. Progress of these steps prior to the start of the pilot was monitored through e-mail contact between the author and site leads. However, as the number of sites is increased for the national audit, a more scalable, transparent and robust system will have to be implemented to ensure timely progress through these key steps.
All site leads reported that the case report forms were simple and easy to complete because they followed a logical and chronological order. Reporting of issues was made by the site leads to the lead author via e-mail or telephone. However, as the audit is scaled nationally, a more robust mechanism for problem reporting will have to be developed.
The site leads reported problems with the unique patient codes that were distributed centrally, as they did not always have the code key to hand when uploading patient data. As reporting was being monitored live, the lead author was able to liaise with site leads, and, in real time, validate patient codes that appeared to be missed, duplicated or inappropriate. The central database was then amended to include the correct patient codes. However, an alternative system will have to be developed that allows site leads to accurately insert patient identifiers without fear of breaching patient confidentiality. A potential solution could be a transformation of specific patient identifiers into an audit-specific patient code. This would allow the site lead to create the code at the time of data entry, but also enable the site lead to trace back the patient if they are only supplied with the code.
Whilst there were minimal missing data within key audit fields, there were considerable missing data for blood test results. Site leads reported that these sections were missed because they were not relevant, not available or time-consuming. The importance of these questions in the final audit will need to be weighed against user fatigue.
The current case report forms did not allow for tracing of patients across sites, in case of re-admissions or complications. The national audit will need to identify these patients to prevent under-reporting of adverse events. This could be achieved through additional questions or by developing an audit-specific patient code key that is related to the unique patient identifiers, as discussed above.
This pilot brought together six trainee collaboratives to work towards a common goal. The collaboration capitalised on individual expertise, such as web-design, network security configuration, study design, statistics and trainee collaborative management, to design and deploy a successful multicentre audit. This model allowed a greater sense of project ownership, and increased the engagement from trainees who gave up their personal time to deliver an audit to the highest of standards, with no setup or running costs.
-
• There is variation in epistaxis management
-
• A national audit of epistaxis management has been prioritised by ENT-UK and the British Rhinological Society
-
• National audits are expensive or have poor data quality
-
• Regional trainee collaboratives can be mobilised towards a national project
-
• A web-based interface on a secure server allows high data quality, without compromising on cost or data safety
However, the national audit will be on a much grander scale. Collaborative research and auditing has seen a recent resurgence within ENT at the trainee level. 10 The success of such projects, and the widespread support of both junior and senior clinicians within the specialty, has led to the genesis of The National ENT Trainee Research Network (‘Integrate’). Integrate will be fundamental to engaging regional trainees and co-ordinating a centralised data collection infrastructure, with the aim of increasing participation and reducing costs.
Going forward, it is essential that a consensus-driven standard for the management of epistaxis is developed. The national audit should aim to map variations in practice and to identify areas where care is suboptimal. The case report form of the national audit should be built around a consensus standard and should also implement practical lessons learnt from this pilot. For a national audit of epistaxis to be delivered effectively, a system that allows robust, cost-effective oversight and co-ordination between sites will need to be developed and implemented. A web-based system that integrates the data collection tool could serve this purpose.
We have shown that a multicentre audit using a web interface, delivered through co-ordination between multiple trainee collaboratives, can produce results with high data quality efficiently and cost-effectively. Feedback from site leads and results from the pilot have resulted in several changes to the case record forms for the final audit. However, the scaling of this pilot audit to a national level will require robust systems of central co-ordination to be developed and implemented.
Acknowledgements
We would like to thank the British Rhinological Society and ENT-UK for all of their support in this project.
