Simulation-Based Training for Disaster Preparedness

Simulation-based education is an effective tool to enhance knowledge, skills, and behavioral outcomes in healthcare education.^{Reference Cook, Hatala and Brydges3,Reference Patterson, Geis and LeMaster4} It has been increasingly used and is effective in disaster preparedness because high-risk low-frequency events can be practiced in a safe environment.^{Reference Patterson, Geis and LeMaster4,Reference Gardner, DeMoya and Tinkoff5} Hospital disaster plans can be tested and personnel trained using a variety of modalities, each with inherently different levels of fidelity, technology, and associated costs. Low-fidelity and low-technology exercises include seminars, workshops, and tabletop exercises.^{Reference Sarin, Cattamanchi and Alqahtani6} These discussion-based exercises are designed to familiarize participants with plans and procedures through facilitated discussion in an informal setting (“learning by discussing”). While discussion-based exercises require less organizational maturity and are less resource and cost intensive than full-scale exercises, they are still time consuming while offering lower physiologic and environmental fidelity and limited scope. On the contrary, operation-based exercises, including drills, functional and full-scale exercises, allow participants to actively engage in high-fidelity exercises (“learning by doing”) and allow for more realistic simulation and better test interactional and operational capabilities.⁷ However, such exercises require extensive planning and often necessitate a substantial investment of both time and capital.^{Reference Skryabina, Reedy and Amlôt8,Reference Lin, Cheng and Hecker9} Moreover, due to the inherent shift-based staffing structure of the ED, not all personnel will be present and able to participate in exercises at any given time, thereby making it difficult to recruit all staff members for 1 single exercise.

In Situ Simulation for Disaster Preparedness

Disaster preparedness needs to include the entire health-care staff when responding to a disaster. There still is a lack of competencies and tools to deliver and evaluate disaster preparedness appropriately.^{Reference Gowing, Walker and Elmer10} In situ simulations allow for increased fidelity connecting the simulation with the work environment where work is actually performed.^{Reference Scerbo and Dawson11} Department-based in situ simulations can lead to improved organizational learning, making this setting a promising choice for in-hospital disaster preparedness.^{Reference Sørensen, Østergaard and LeBlanc12,Reference Rüter, Kurland and Gryth13} They help improve knowledge and communication during a sudden influx of patients in the emergency department, allowing for the entire department to be better prepared for disaster scenarios.^{Reference Jung, Carman and Aga14}

Huddles

Huddles are brief staff meetings among key shareholders and have been used in various sectors of health care to improve communication and situational awareness.^{Reference Goldenhar, Brady and Sutcliffe15,Reference Chan and Vadera16} In 1 domain, they may exist as a planned daily management meeting and serve as a quality improvement initiative.^{Reference Donnelly17} A huddle may also be implemented as an ad hoc meeting between key medical staff members. In this scenario, key personnel hold a meeting in real time to identify and discuss particular management dilemmas in a patient-centered and collaborative effort.^{Reference Edbrooke-Childs, Hayes and Sharples18,Reference Stapley, Sharples and Lachman19} We adopted the concept of huddles and applied it in a novel way toward disaster preparedness (Figure 1).

FIGURE 1 Conceptual Framework of Fidelity Versus Cost in Mass Casualty Incident (MCI) Training Types.

We propose the novel concept of “disaster huddles” to build administrative disaster preparedness. We hypothesized that brief, task-specific, inexpensive, targeted disaster huddles at frequent intervals in a single pediatric ED may improve administrative disaster preparedness.

Disaster Huddle Development

Three MCI scenarios were developed and modified by a multi-disciplinary team consistent of physicians, nurses, technicians, and simulation experts (Online Supplemental Material 1). Five researchers were oriented to an evaluation tool that was used for all scenarios to record the key measures of interest in staff performance during the disaster huddle.

Disaster Huddle Implementation

Over a period of 26 continuous weeks from January 2018 to June 2018, the 3 disaster huddle scenarios were implemented at Yale-New Haven Children’s Hospital Pediatric Emergency Department (PED). Institutional Review Board approval was waived by Yale University. The participants were nurses, physicians, technicians, and business staff who worked in the PED at the time of the intervention. Disaster huddle participants were evaluated as a team on shift rather than individually. Over the course of the study, 1 disaster huddle per week was conducted in the PED, facilitated by a member of the study team. The huddles were assigned to rotating days and shifts to ensure an appropriate representation of real patient surge, staffing composition, and other factors that might influence the ability of the emergency staff to participate in the disaster huddle. Each huddle consisted of a brief, in situ simulation of disaster readiness. Three scenarios were alternatingly executed throughout the study period and included: (1) an active shooter with injured victims, (2) an acute patient surge from internal evacuation of inpatients, and (3) an MCI from a bus crash (Online Supplemental Material 1). An evaluator filling out the scoring sheet and an instructor facilitating the disaster huddle was assigned to each huddle. The scenario began with a radio announcement that was audible in the same manner a real-world radio transmission from local emergency medical services. Staff was expected to perform as they would in a real-world scenario, following the PED’s emergency response checklist. The scenario ended with the distribution of department-specific disaster cards, a predefined “end exercise” point in the checklist. During the drill, outcomes of interest were recorded. Following each drill, a short debriefing was held to enhance learning.

Disaster Huddle Evaluation

The success of the implementation of disaster huddles was defined as the timeliness and correct performance of critical actions. Time to performance of critical actions included time to: (1) page for the huddle, (2) all staff members present, (3) code D (disaster response) activated, and (4) page operator call. Critical actions included the following: (1) obtain appropriate information from Central Medical Emergency Dispatch; (2) overhead page for staff huddle and briefing; (3) assess staffing resources; (4) determine need for code D activation; (5) call STAT line/activate code D; (6) assign staff roles; and (7) distribute disaster cards and brief PED. Lastly, staff participants and their function (eg, charge nurse, attending physician, technician) were recorded (Online Supplemental Material 2). The data were entered into an electronic survey (Qualtrics, Qualtrics LLC, Provo, UT) for analysis.

The primary outcome was staff performance of critical actions, defined as the sum of the critical actions performed by the staff, with each action receiving a score of 0 (did not complete), 1 (completed partially), or 2 (completed). The total possible score for an individual drill was 16. The 8 items included (1) obtain appropriate information from Central Medical Emergency Dispatch; (2) overhead page; (3) staff huddle and briefing; (4) assess staffing resources; (5) determine need for code D activation; (6) call STAT line/activate code D; (7) assign staff roles; and (8) distribute disaster cards and brief PED. Secondary outcome measures were times to performance of critical action items.

An experience factor (EF), calculated by dividing the combined experience level of all participants by the total number of participants in the drill, was included in the linear regression model to control for experience.

Disaster Huddle Outcomes

Over the course of the 26-week study, we conducted a total of 25 disaster huddles with 1-9 participants during each disaster huddle. Huddles lasted between 1 and 10 min. A total of 115 people participated in the drills, and there were 78 unique PED staff members who participated in at least 1 disaster huddle out of the total 81 PED staff in the PED. The EFs for the staff ranged from 1.0 to 2.33. The mean duration of each huddle was 7.00 min (SEM = 0.57).

The primary outcome was the disaster huddle score, which was a sum of the critical actions each ranked 0 to 2 for a total score of up to 16. A linear regression was used to fit the disaster huddle score, demonstrating a positive trend over time (r ² = 0.373) with a slope of 0.04 (SE = 0.011; P = 0.0012) (Figure 2). An adjusted linear regression was calculated using EFs. This revealed a positive effect of time (in days) on the total disaster huddle scores (β = 0.032; 95% confidence interval = 0.001, 0.063; P = 0.045), while EF had no impact on the disaster huddle score (β = 1.37; 95% confidence interval = -2.24, 4.80; P = 0.439). Additions of additional variables into the model (ie, shift time [am or pm] and number of participants) did not reveal significant predictors of a higher score.

A linear regression was used to fit the data (r ² = 0.3725) and the slope of the regression was 0.04 (SE = 0.011, p = 0.0012). Dashed grey lines represent the 95% confidence interval of the fit.

FIGURE 2 Disaster Huddle Scores Over Time.

Overall, staff successfully completed more items on the checklist over time (Figure 3). Mean scores during the first half of the study (n = 13 huddles) were significantly higher than the last half (n = 12 huddles) of the study (P = 0.001) (Figure 4). However, using linear regressions to assess trends over time, time to critical actions did not change over time, including calling page for huddle (r ² = 0.059; P = 0.349), gathering staff (r ² = 0.018; P = 0.537), code D activation (r ² = 0.048; P = 0.316), and page operator call (r ² = 0.060; P = 0.311). Nine of twenty-five disaster huddles were assessed in minutes rather than in seconds, leading to exclusion of these data points in the calculation of time to action. No scenario took more than 16 min, including the debriefing.

Each cell represents a score from 0 (red) to 2 (green), showing an increasing percentage of green cells over time. We took the mean scores from the first half of the study to compare against the second half of the study using independent two-sided T-tests and obtained a p-value of 0.001 (**).

FIGURE 3 Heat Map Showing the Disaster Huddle Components (Rows) Over Time (Columns).

(p = 0.001)

FIGURE 4 Disaster Huddle Mean Scores During the First Half of the Study (n = 13 Huddles) Compared to the Last Half of the Study (n = 12 Huddles).

LIMITATIONS

This study is subject to several limitations. First, the huddle evaluators were part of the investigation team and were un-blinded as to study hypothesis. Additionally, although the team was comprised of 5 evaluators, all of whom had reached consensus about the use and application of the evaluation tool, there may have been variation in the facilitation and prompting of huddle participants, as well as the use of the evaluation tool. Video recording of the huddles and asynchronous evaluation of huddle performance would have mitigated this limitation. Next, there was no head-to-head comparison of the disaster huddles to a live drill or exercise, limiting the ability to determine relative performance of these training strategies. Additionally, there was high variability in staff participation (ranging from 1 to 9 participants) and in staff roles at each huddle. While we attempted to mitigate this variation through the use of the weighted EF, it nevertheless introduces the potential for skewed data. We see this variability as a strength in some regards, as there was no trend toward more or fewer participants over the course of the study, and there was improvement in disaster task performance. Finally, the exclusion of 1 of 3 of our time-level data may have resulted in the inability to identify otherwise statistically significant trends in this data.

FUTURE DIRECTIONS

An important next step in this work is measuring a potential impact on lasting behavior and real patient outcomes in actual disasters. Although it would be difficult to measure correlation with disaster huddle participation and effectiveness of responses to administrative challenges in MCIs, this would be the most compelling argument to institute huddles widely in emergency departments. One approach would be to evaluate blended exercises over time incorporating a per-post exercise assessment. This would help evaluate impact on care outcomes and improvement.

A more achievable surrogate for this would be to bracket a several-month period of weekly disaster huddles, with a full-scale simulated disaster drill or exercise immediately before and after the huddles, to measure improvements that correlate with or are attributable to the use of huddles in preparedness training. Furthermore, a cost-effectiveness analysis comparing disaster huddles with the use of full-scale exercises or task-specific drills would potentially make another argument for the relatively low-cost training strategy examined in this study. Finally, a multi-center study conducted at pediatric and general emergency departments would strengthen the generalizability of huddles in a range of clinical settings.