Introduction
Neonates in the neonatal intensive care unit (NICU) are vulnerable to natural and man-made disasters because of their persistent demand for medical resources that require electricity, medical gases, clean water, and appropriate shelter. 1–Reference Cohen, Murphy, Ahern and Hackel5 When the medical resources supply is unreliable after disasters, neonates in the NICU need to be evacuated to safe locations. Reference Espiritu, Patil and Cruz6,Reference Ma, Cohen and Lee7 Considering the limited medical and transport resources available after disasters, the order of evacuation is likely to affect the safety and subsequent survival of neonates. To perform NICU evacuation within a limited time period, priority needs to be determined using a triage system that accounts for the severity of illness and demand for medical and human resources of the neonates during transition and transportation. Reference Femino, Young and Smith8,Reference Lin, Taylor and Cohen9
The Simple Triage and Rapid Treatment (START) system has been widely used to identify adult and pediatric patients who need urgent medical care and transportation after large-scale collisions, earthquakes, floods, terrorist attacks, and explosions. Reference Benson, Koenig and Schultz10 In Japan, a modified START system (START-Neo) was developed for hospitalized neonates. Using the START-Neo system, NICU nurses, neonatologists, and even trainees can easily classify neonates into four categories, grey, red, yellow, and green (ranging from high-demand to low-demand neonates), based on the clinical conditions of and medical care provided for the neonates. 1,Reference Kawase, Iwata and Kondo11 When the 2016 Kumamoto Earthquake occurred in Japan, the START-Neo system helped determine the evacuation order of 38 hospitalized neonates from a tertiary NICU that was at risk for collapse. Reference Kawase12 However, because START-Neo subsequently classified 95% of the neonates into either the red category or the yellow category, NICU staff had difficulties assigning the evacuation order of the neonates within the same categories. Reference Kawase12 Therefore, in addition to being quick and reproducible, an efficient triage system for neonates ideally needs to provide semi-quantitative information regarding the clinical conditions, demand for medical resources, and subsequent transportation priority of the neonates. Reference Barfield and Krug4–Reference Espiritu, Patil and Cruz6,Reference Kawase, Iwata and Kondo11
This study developed and tested the validity and reproducibility of a novel triage system for neonates, compared to a conventional triage system, and compared it with a standardized prognostic scoring system for hospitalized neonates.
Methods
A retrospective observational study was conducted. Routinely collected triage data were used from a tertiary NICU (Nagoya City University Hospital - Nagoya, Aichi, Japan) that has 12 intensive care beds, 15 special care beds, and approximately 300 admissions per year.
Patient Triage
In the NICU, patient triage was implemented as a routine practice of the ward in September 2018 based on recommendations created after the 2016 Kumamoto Earthquake. Reference Kawase, Iwata and Kondo11 Every Monday and Friday, at the time of the daily morning ward round, one physician and one nurse who were responsible for the round record independently made assessments on each neonate using two triage systems. The first was a revised version of the START-Neo system called START-Neo-R. 1 The START-Neo system classified neonates into four levels: green (evacuation possible); yellow (evacuation possible with continued medical treatment); red (evacuate only if necessary); and grey (difficult to evacuate). These four levels were based on the severity of disease and dependence on medical care (Supplemental Material 1 shows details regarding the triage items; available online only). After considering the NICU evacuation during the 2016 Kumamoto Earthquake, during which 95% of the neonates were subsequently classified into the yellow and red categories, the task force of the Japanese Society for Prematurity and Neonatal Medicine (Osaka, Japan) and the Liaison Committee on Neonatal Care (Japan) revised this system and created a five-level scale for START-Neo-R by dividing the red category into Red-I (dependent on oxygen, high-flow nasal cannula, or noninvasive positive pressure ventilation, using vasopressors/dilators, and/or in a closed incubator) and Red-II (dependent on mechanical ventilation, use of arterial and/or umbilical catheters, and/or use of drainage; Supplemental Material 1 shows the details of the triage items [available online only]). 1 The second triage system was the Neonatal Extrication Triage (NEXT) system, which was originally developed in 2017. Reference Yamamoto, Karasawa and Nishio13 The NEXT system is a simple system that was designed to provide semi-quantitative composite scores to avoid the accumulation of cases at particular levels. To perform objective and reproducible assessments, six questionnaires regarding the neonates’ immaturity, requirements for respiratory support, residing canulae, monitoring, incubator, and total risks were used to weigh the demands for medical resources and care during transition and transportation, rather than the severity of illness or clinical conditions. Scores of zero to two were assigned to each questionnaire to obtain a composite score ranging from zero to twelve (Table 1). A preliminary report of the development of the NEXT system was previously published in Japanese. Reference Yamamoto, Karasawa and Nishio13
Table 1. Scoring Sheet for Neonatal Extrication Triage
Note: Score of 0-2 was assigned for each of six questionnaires to give a composite score of 0-12.
Abbreviations: ECG, electrocardiogram; HFNC, high-flow nasal cannula; NIPPV, noninvasive positive-pressure ventilation.
Data Collection
Patients admitted to the NICU of Nagoya City University Hospital from August 1, 2019 through February 28, 2020 were enrolled. Of these, 162 patients for whom triage was assessed at least once at the twice-weekly triage round were included in the analysis. Patients who were hospitalized longer than one week had multiple data points; the scores on different days were treated as separate data. Clinical variables were collected from the electronic medical records, including gestational age, birth weight, diagnosis at admission, postnatal age, and body weight at the time of assessment. The Neonatal Therapeutic Intervention Scoring System (NTISS) scores were retrospectively calculated by referring to the patient data at 9:00am on the day of triage. Reference Gray, Richardson, McCormick, Workman-Daniels and Goldmann14 The NTISS is a therapy-based severity assessment tool that evaluates the illness of neonates receiving intensive care. It comprises 64 questionnaires and provides composite scores ranging from zero to 100.
Data Analysis
Values are shown as mean and standard deviation (SD) or median and interquartile range (IQR) unless otherwise specified. Findings of START-Neo-R were regarded as ordinal data ranked from one to five, whereas composite scores obtained using NEXT and the NTISS were regarded as numerical data because these values were subsequently confirmed to be normally distributed. First, violin plots were used to visually observe the distributions of the START-Neo-R and NEXT scores for each measurement day. The inter-rater reliability of START-Neo-R and NEXT when used by nurses and physicians were assessed using Cohen’s kappa coefficient. Currently, there is no established evacuation triage system for neonates that can be used as a reliable gold standard. Therefore, for the current study, the criterion validity of START-Neo-R and that of NEXT were assessed using NTISS scores based on the comprehensive clinical information incorporated and a previous finding that NTISS scores more than 20 during the first 24 hours of admission were associated with more severe illness and increased mortality for neonates. Reference Gray, Richardson, McCormick, Workman-Daniels and Goldmann14,Reference Wu, Lee, Lee and Chen15 The associations of the START-Neo-R and NEXT scores with NTISS scores were examined using Spearman’s correlation coefficient. A receiver-operating characteristic curve was used to assess the predictive values of START-Neo-R and NEXT for NTISS scores of 20 or more. The area under the curve and 95% confidence interval (CI) were also calculated. The optimal cut-off value was determined using the Youden index.
Ethical Approval
This study was approved by the Institutional Review Board of Nagoya City University Hospital (60-20-0110). The authors applied the opt-out method to obtain consent for this study via the website of Nagoya City University Hospital. According to the guidelines, the institutional review board waived the need for informed consent and approved the consent procedure because this study was retrospective and no patient identifier was used in the analysis.
Results
During the study period, 162 neonates were admitted to the NICU, all of whom were assessed at least once on 49 days of the triage round. All required data were collected without any missing data. Nurses and physicians assessed the NTISS, START-Neo-R, and NEXT scores of a total of 1,079 patients. The mean weight of the neonates comprising the study population was 2,224g (SD = 1,101g) at the time of evaluation. The gestational age at birth for the neonates was 35.4 weeks (SD = 4.1 weeks). The mean postnatal age of the neonates was 48 days (SD = 58 days) at the time of triage (Table 2).
Table 2. Patient Characteristics
Abbreviation: LBWI, low birth weight infant.
The median NTISS and NEXT scores among physicians were six (IQR, 3-13) and six (IQR, 3-8), respectively, whereas the START-Neo-R scores among physicians categorized 30.2%, 19.0%, 23.0%, 25.3%, and 2.5% of the neonates into the green, yellow, Red-I, Red-II, and grey categories, respectively. The START-Neo-R and NEXT scores were well-dispersed across different levels without excessive accumulation in specific categories, regardless of the average severity of illness of all neonates admitted on the measurement day (Figure 1; Supplemental Material 2 and 3 show the distribution of scores across all measurement dates [available online only]). Inter-rater reliability between physicians and nurses was convincingly high for both START-Neo-R and NEXT with kappa coefficients of 0.952 (95% confidence interval [CI], 0.946-0.957) and 0.973 (95% CI, 0.969-0.976), respectively (Figure 2).
Figure 1. Violin Plot of Distributions of NEXT and START-Neo-R Scores by Evaluation Date.Note: The days with the highest mean NTISS scores were recorded during the study period. The distribution of the NEXT (a) and START-Neo-R (b) scores for the five days with the highest (1-5) and lowest (44-49) mean NTISS values are shown in the violin plot. Scores of both the START-Neo-R and NEXT were well-dispersed across different levels without excessive accumulation in specific categories, regardless of the average severity of illness of all newborns admitted on the measurement day. Abbreviations: NEXT, Neonatal Extrication Triage; START-Neo-R, Simple Triage and Rapid Treatment for Neonates, Revised; NTISS, Neonatal Therapeutic Intervention Scoring System.
Figure 2. Bubble Chart Showing the Correlations of NEXT, START-Neo-R, and NTISS Scores Evaluated by Physicians.Note: (a) NEXT versus NTISS; (b) START-Neo-R versus NTISS. The straight lines in the graphs show the regression lines. The NEXT and START-Neo-R scores were closely correlated with the NTISS scores.Abbreviations: NEXT, Neonatal Extrication Triage; START-Neo-R, Simple Triage and Rapid Treatment for Neonates, Revised; NTISS, Neonatal Therapeutic Intervention Scoring System.
Both the START-Neo-R (correlation coefficient, 0.850; P <.001) and NEXT (correlation coefficient, 0.889; P <.001) scores were strongly correlated with the NTISS scores determined by physicians (Supplemental Material 4 shows the correlations of START-Neo-R and NEXT as assessed by nurses using the NTISS [available online only]). The receiver-operating characteristic curves of the predictive value of START-Neo-R and NEXT for NTISS scores of ≥20 are shown in Figure 3. The area under the curve values of the predictive value of START-Neo-R and NEXT for NTISS scores of 20 or more were 0.934 (95% CI, 0.923-0.944; cut-off, Red-I) and 0.973 (95% CI, 0.962-0.975; cut-off, 8), respectively. For START-Neo-R, the cut-off level of Red-I had a sensitivity of 0.986 and specificity of 0.859, whereas the cut-off value of eight for NEXT had a sensitivity of 0.971 and specificity of 0.884 for predicting NTISS scores of 20 or more.
Figure 3. ROC Curves with Predictive Values of START-Neo-R and NEXT for NTISS Scores ≥20.Note: The area under the curve to predict NTISS scores ≥20 was 0.934 (95% CI, 0.923-0.944; cut-off Red-I) for START-Neo-R and 0.973 (95% CI, 0.962-0.975; cut-off 8) for NEXT. Thick dashed line, NEXT; thin dashed line, START-Neo-R; straight line, diagonal reference line.Abbreviations: NEXT, Neonatal Extrication Triage; START-Neo-R, Simple Triage and Rapid Treatment for Neonates, Revised; NTISS, Neonatal Therapeutic Intervention Scoring System; ROC, receiver-operating characteristic.
Discussion
To the best of the authors’ knowledge, this is the first study to validate the inter-rater reliability and criterion validity of a novel NICU evacuation triage. The novel triage system, NEXT, could instantly and objectively rank neonates in the NICU using twelve levels based on time and medical resources required for transition and transport. Both START-Neo-R and NEXT had good reproducibility and correlation with the severity of neonates indicated by the NTISS, which was used as the gold standard. These findings suggest that NEXT might be suitable for determining the priority of evacuation for neonates hospitalized in the NICU.
To assist with post-disaster evacuation from the NICU in resource-limited settings, the order of transportation needs to be determined by an objective, reproducible, and easy-to-apply triage system. Reference Barfield and Krug4 In contrast to the requirement for the brevity of the triage system, assigned scores should ideally provide sufficient information regarding the severity of disease and dependence on medical resources and care required during transition and transportation. Reference Cohen, Murphy, Ahern and Hackel5 Additionally, to conveniently decide the evacuation order, the scores are expected to disperse into a wide range. Reference Cohen, Murphy, Ahern and Hackel5
The NTISS, which was used as the gold standard to examine criterion-related validity, collects detailed clinical information regarding the administration of 64 therapies provided for neonates. Reference Gray, Richardson, McCormick, Workman-Daniels and Goldmann14 Designed to predict short-term outcomes of hospitalized neonates, each NTISS questionnaire can be assigned objectively. However, because of the large number of survey items involved in the NTISS, its assignment typically requires approximately ten minutes per patient, even when used by experienced physicians and nurses. Additionally, assessors need to refer to patient records for detailed information. These features render this scoring system unsuitable for the purpose of evacuation triage after disasters unless an electronic system is developed that allows for survey items to be automatically completed and continuously updated by referring to the medical records.
The START-Neo-R system is a simple alternative to the complex scoring process of the NTISS. Because the original START system was designed to identify pediatric and adult evacuees who need urgent medical treatment after disasters, its revised version, the START-Neo-R system, consistently focused on the severity of illness. 1 The START-Neo-R system is simple and easy to apply. When used by physicians and nurses, scores are assigned within approximately 30 seconds. In contrast, the simple five-level ranking classification system of START-Neo-R inevitably accumulates several neonates within the same category. Reference Kawase12,Reference Iwata, Kawase, Iwai and Wada16 The scores appear well-distributed at first glance; however, during the current study, when used by physicians, 67.3% of the study cohort was classified into the red or yellow category, suggesting that an additional evaluation is required to determine the evacuation order. Moreover, neonates admitted to the NICU can easily become hypothermic because of environmental temperatures or experience hypoglycemia because of feeding interruptions, making them prone to clinical instability even if they are classified as green by START-Neo-R. Nevertheless, because the original START system has been well-accepted as a tool for pediatric and adult evacuees, the green category may be immediately perceived as “low-risk evacuees.” Reference Johnson, Diekema, Lewis-Newby and King17 This is especially important because multiple professionals with both medical and nonmedical backgrounds need to share patients’ information before a NICU evacuation plan is developed. Reference Cohen, Murphy, Ahern and Hackel5,Reference Janvier, Leblanc and Barrington18 Finally, START-Neo-R has the possible negative effect of automatically determining categories based on a few keywords. If neonates require drainage or use closed incubators, then they will be classified into the red category regardless of their general condition and will be placed in the same category as other neonates with unstable specific conditions. This results in a significant dissociation between the triage assessment and the actual clinical situation. Nonetheless, the START-Neo-R scores showed excellent correlation with NTISS scores and considerably high inter-rater reliability. Reference Janvier, Leblanc and Barrington18
The NEXT system has been developed to enable semi-quantitative evaluation of the evacuation order by weighting the medical resources and care required during transition and transportation. This scoring system comprises six questionnaires that provide composite scores ranging from zero to twelve for hospitalized neonates and have excellent inter-rater reliability. The NEXT scores showed a strong linear relationship with NTISS scores, suggesting that the NEXT scores represent both the dependence on medical resources and severity of illness. The NEXT scores were well-dispersed into a wide score range, which may be the greatest advantage of the NEXT system because the priority of evacuation can be provided instantly by the composite score. Unlike the NTISS system, the evaluation by the six simple questionnaires of NEXT typically requires less than 30 seconds. Because the survey items are exclusively on-going treatments, assessment can be completed at the bedside or using the medical records. Considering the simple and objective evaluation algorithm of NEXT, its scores are expected to be reproducible even when they are assigned by different professionals with medical and non-medical backgrounds, potentially leading to improved information sharing among physicians, nurses, and transport coordination teams after disasters.
Limitations
The current study had several limitations that need to be addressed. First, this study was conducted at a single NICU; therefore, the findings might be different for neonates with various backgrounds at multiple centers. However, a prospective study is being performed to collect and examine triage data from multiple NICUs of different types and scales. Second, although convincingly high inter-rater reliability was demonstrated between physicians and nurses, this study involved only professionals with NICU experience. Further studies involving multi-disciplinary health care professionals with or without neonatal and pediatric medicine expertise are needed to assess inter-rater reliability.
Conclusions
A novel evacuation triage system, NEXT, for neonates in the NICU has been developed. Composite scores obtained from this six-questionnaire triage system showed a close relationship with scores obtained using the standardized prognostic NTISS. With its well-dispersed scores across different levels, the NEXT system might be a powerful tool for objectively determining the evacuation priority and securing the safety of vulnerable neonates after disasters. Further validation studies are required to determine the impact of NEXT on time required to complete the evacuation and ensure the safety of hospitalized neonates. A large-scale simulation study is being performed to identify the optimal evacuation order for NICUs and to investigate whether the use of NEXT improves the safety and survival of neonates, using various assumptions regarding the patient background, disaster type, and available human, medical, and transportation resources.
Conflicts of interest/Funding
The authors declare none.
Acknowledgments
The authors thank Ms. Takako Nishio, Ms. Miyu Koike, and Ms. Ayaka Kashima for their considerable help with data collection, and the staff of the NICU for their continuous support during the study.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X22000553
