Introduction
The modern world is affected by many types of disasters that result in large numbers of civilian mass casualties.Reference Barthel, Pierce and Goodhue 1 - Reference Shirm, Liggin, Dick and Graham 4 These catastrophic events include natural disasters, such as tsunamis, earthquakes, and hurricanes, as well as acts of terrorism and violence, such as the 2013 Boston (Massachusetts USA) Marathon bombing.Reference Barthel, Pierce and Goodhue 1 - Reference Gold, Montano and Shields 3 , Reference Hupp 5 Recent disasters have prompted health care providers and policy makers to emphasize the importance of preparedness exercises for hospitals and other health care facilities. These exercises have focused increasingly on the vulnerability of pediatric victims.Reference Barthel, Pierce and Goodhue 1 - Reference Gold, Montano and Shields 3 , Reference Bar-On, Lebel, Blumberg, Sagi, Kreiss and Israel Defense Forces Medical 6 - Reference Ferrer, Ramirez, Sauser, Iverson and Upperman 10 Children have important physiological and developmental needs that are distinct from those of adults, and in the event of a mass-casualty situation, they need specialized care involving customized supervision, staffing, and resources.Reference Kaji and Lewis 7 , Reference Allen, Parrillo, Will and Mohr 9 , Reference Luby, Belden and Botteron 11 - Reference Thompson, Lyle, Mullins, Dick and Graham 13 Despite the special needs of children, pediatric care, including clinical specialists and customized resources, remains limited at many hospitals.Reference Allen, Parrillo, Will and Mohr 9 , Reference Ballow, Behar, Claudius, Stevenson, Neches and Upperman 14 - Reference Petrosyan, Guner, Emami and Ford 18 In a mass-casualty situation, this limited availability of resources may leave pediatric victims vulnerable to mismanagement, avoidable complications, and death.Reference Burke, Berg and Vee 15
Several methods have been demonstrated to improve pediatric disaster preparedness and response. For instance, employing evidence-based, pediatric-specific disaster triage systems can lead to life-saving outcomes during a mass-casualty event.Reference Ballow, Behar, Claudius, Stevenson, Neches and Upperman 14 , Reference Burke, Berg and Vee 15 Disaster exercises focusing on pediatric victim surges can also strengthen the readiness of hospital staff to respond to pediatric disaster victims by allowing them to utilize and practice relevant pediatric emergency response skills and protocols, including triage. In addition, conducting regular disaster exercises may help improve communication between hospitals, physician participation, and knowledge of pediatric-specific protocols.Reference Chokshi, Behar, Nager, Dorey and Upperman 2 , Reference Cicero and Baum 19 , Reference Eppich, Adler and McGaghie 20
The purpose of this study was to use mixed methods to comprehensively assess the current state of disaster preparedness in three local hospitals focusing on pediatric disaster victims. Through the implementation and evaluation of a full-functional disaster exercise involving pediatric casualties, multiple elements of emergency response were assessed, including communication, equipment and supplies, security, staffing, Incident Command (IC), pediatric safety, and training. Quantitative and qualitative feedback from exercise participants and observers was expected to reveal opportunities for improvement in multiple aspects of pediatric emergency response and aid in the development of strategies to improve hospital preparedness for pediatric disaster victims.
Methods
Setting
On May 14, 2011, a full-functional exercise involving the simulation of a 7.8-magnitude earthquake was conducted at three Los Angeles (California USA) hospitals, one of which was a designated Level I Pediatric Trauma Center. The other two facilities were nonpediatric hospitals offering comprehensive medical services to patients of all ages. The exercise lasted three hours and 40 minutes and entailed a prolonged surge of mock pediatric victims requiring triage and treatment at each facility. Details of the drill scenario are delineated in the Master Scenario Events List (Appendix; available online only). Institutional review board approval was obtained from all participating sites.
Participants
Exercise participants included staff from the three participating hospitals. Staff members at all three facilities were informed by the administration at their respective hospitals about the exercise and were invited to participate. Participation was voluntary, and no incentives were provided. At each facility, staff who elected to participate included physicians, nurses, and nonclinical workers. All participants were invited to complete an electronic survey following participation in the exercise. A subset of participants who played leadership roles also provided qualitative feedback in interviews immediately following the exercise.
Observers
Exercise observers included nine researchers and staff from the pediatric hospital with expertise in pediatric disaster preparedness. Because the present investigation was concerned with assessing aspects of pediatric disaster preparedness, no observers from the nonpediatric facilities were selected. To ensure that observers were adequately prepared to carry out proper observation and interview protocols, they participated in a 2-hour training session. The session entailed careful review of pediatric issues in disaster readiness, review of the constructs being assessed during the exercise, and specific instructions for carrying out comprehensive exercise observations and semi-structured interviews.
Prior to the exercise, each observer was assigned to a specific station—IC, Triage, or Treatment—within one of the three hospital facilities. Observers were responsible for carefully observing exercise performance at their assigned facilities and reporting their qualitative feedback after completion of the exercise. In addition, they were asked to provide recommendations for improving pediatric disaster preparedness based on their observations of exercise performance. Immediately following the exercise, observers were also responsible for conducting one-on-one interviews with exercise participants at their assigned stations.
Data Collection
Quantitative Participant Feedback
After the exercise, participants were asked to complete an anonymous, online survey that was created using Qualtrics software, Version 52934 (Provo, Utah, USA). Consisting of 13 closed-ended questions, the survey was designed to gather quantitative feedback regarding participants’ perceptions of pediatric disaster response during the exercise. Participants were asked to report demographic data, employment information, previous disaster training experience, satisfaction with various aspects of emergency response, and perceptions of the exercise's value for enhancing disaster preparedness.
Qualitative Participant Feedback
Immediately after completion of the exercise, observers conducted semi-structured, one-on-one interviews with exercise participants at their respective facilities. A total of nine interviews were conducted across the three hospitals, with each observer interviewing a participant who headed their assigned point station. The purpose of these interviews was to obtain a more detailed understanding of challenges during the exercise from the perspective of participants and to comprehensively examine vulnerabilities that would benefit from additional planning and future exercises. Observers asked participants who held leadership roles six open-ended questions and transcribed their responses on a personal notebook computer or by hand.
Qualitative Observer Feedback
Observers evaluated exercise performance at their assigned point stations within their assigned facilities and recorded their feedback on an observation form. This form prompted observers to provide open-ended feedback regarding their perceptions of communication, equipment and supplies, security, staffing, training, IC, triage, and treatment during the exercise. In addition, the form also prompted observers to provide recommendations regarding communication, staff roles and responsibilities, resources and assets, and triage based on their observations during the exercise. The purpose of these recommendations was to aid in the development of evidence-based strategies for improving pediatric disaster preparedness and response.
Data Analysis
Quantitative Participant Feedback
Participants’ survey responses were computed using Qualtrics software, Version 52934 (Provo, Utah, USA). For analysis of responses provided on a Likert scale, “Strongly Agree” and “Agree” responses were combined into an “Agree/Strongly Agree” category, and “Strongly Disagree” and “Disagree” responses were combined into a “Disagree/Strongly Disagree” category.
Qualitative Participant and Observer Feedback
Qualitative analysis was based on a thematic analysis approach using ATLAS.ti qualitative analytical software (Version 5; Berlin, Germany), which identifies recurring themes in qualitative responses. The authors reviewed thematic patterns that emerged from participant interviews and observers’ qualitative feedback and subsequently created codes that reflected those themes until saturation of themes was reached. Themes were catalogued and, along with the previously mentioned aspects of drill performance, collectively became the basis for evaluation of the disaster exercise.
Results
Quantitative Participant Feedback
Although all exercise participants were asked to complete an electronic survey following the disaster exercise, only 20 participants from the pediatric hospital completed the survey. Ten (50%) of these respondents were nurses, and the remaining 10 (50%) identified themselves as nonclinical staff members.
Participants’ survey responses are presented in Tables 1 and 2. A majority of participants viewed the exercise as valuable for enhancing preparedness, with 17 (85%) reporting that they received appropriate just-in-time training during the exercise and 19 (95%) claiming that the exercise enhanced their knowledge of disaster response. With regard to their evaluation of roles during the exercise, 18 (90%) participants agreed that their team successfully fulfilled all responsibilities. Thirteen (65%) participants agreed that staffing was sufficient in their areas, and 11 (55%) indicated that they knew their identified role in their facility's disaster plan. Finally, 15 (75%) participants agreed security for both adult and pediatric patients was adequate and that overall communication during the exercise was satisfactory.
Table 1 Participant Feedback Regarding Exercise Performance
Table 2 Participant Satisfaction with Emergency Response Constructs and Stations
Qualitative Participant Feedback
Participants from all three facilities provided qualitative feedback during semi-structured interviews with observers; their feedback is summarized in Table 3. A number of participants from all three hospitals reported feeling under-prepared for pediatric disaster victims, noting that they would have performed better with more comprehensive pediatric disaster training. Additionally, participants from one of the nonpediatric facilities noted limitations in the availability and functionality of radios and a telephone. With regard to communication and staffing, many participants noted that communication with IC could have been improved, Job Action Sheets needed to be updated, and confusion regarding roles and responsibilities was evident. They suggested that encouraging open communication in future exercises would help ameliorate staff confusion regarding their responsibilities. Finally, participants at all facilities noted that the Triage stations were overloaded with patients and had insufficient physician staffing.
Table 3 Participant Interview Responses/Feedback Regarding Exercise Performance
Abbreviation: IC, Incident Command.
Qualitative Observer Feedback
With regard to equipment and supplies, at one of the nonpediatric facilities, observers noted that many participants were not familiar with how to use radios despite having received training prior to the start of the exercise. Similarly, at the same facility, the radio at IC was not functional, and observers reported that important telephone calls could not be made because the phones were initially connected to incorrect outlets.
Observers also reported that participants at all three facilities varied in their understanding of roles and responsibilities. Individuals reporting to command centers seemed unsure of where and to whom they should report. Staff members were unclear regarding their extent of participation in the exercise. For instance, in one of the nonpediatric hospitals, observers reported that section chiefs were not aware that Job Action Sheets, which outlined specific disaster response roles and responsibilities, were available at their facility. Further, observers at all three facilities reported that participants often identified themselves by name as opposed to assigned role. Additionally, they noted that many participants seemed unclear regarding the structure of the Hospital IC System. Furthermore, observers at all facilities reported that few physicians participated in the exercise, with a majority of participants being nonclinical staff members.
Observers noted several areas needing improvement related to medical management and triage of mock victims. During the exercise, nurses did not switch from individual triage to disaster triage, leading patients to be prioritized according to severity of injury as opposed to survivability and resource availability. Additionally, at one triage area at a nonpediatric facility, a nurse appeared to be overwhelmed by her responsibilities but did not ask for assistance. Observers also noted that at the nonpediatric facilities, children and adults were treated similarly, with no specific recognition of clinical care of children. Observers also reported that pediatric safety was an issue for the nonpediatric hospitals, where children were left unattended in waiting rooms and treatment and triage areas. Further, one observer reported that one of the mock victims at a nonpediatric facility did not return to the meeting place after completion of the exercise despite being instructed to do so.
Observers’ Recommendations for Improving Preparedness and Response
Table 4 presents a summary of observers’ recommendations for improving pediatric disaster preparedness and response. Based on their evaluations of exercise performance, observers recommended informing IC and section chiefs about the importance of each hospital conducting regular exercises, assigning advocates among physicians, and designating check-in personnel at the IC. In terms of managing resources, observers recommended that hospitals regularly check and update their contact information and inform and remind chiefs and personnel about the Job Action Sheets. Observers also highlighted the importance of using radios during disaster scenarios and checking their functionality on a regular basis. In addition, observers suggested assigning at least one individual to track pediatric victims and another to supervise unaccompanied minors. Finally, observers reported that triage staff would benefit from training in pediatric-specific triage, such as the JumpSTART Pediatric Multicasualty Triage Tool developed by Dr. Lou Romig (Miami, Florida USA) in 1995.
Table 4 Observer Recommendations for Improving Pediatric Disaster Preparedness and Response
Discussion
Consistent with previous investigations of pediatric disaster preparedness, evaluation of the exercise revealed opportunities for improvement in the readiness of all three hospitals, including the pediatric institution, to respond to pediatric victims.Reference Ferrer, Balasuriya, Iverson and Upperman 17 , Reference Burke, Iverson, Goodhue, Neches and Upperman 21 - Reference Stamell, Foltin and Nadler 23 However, the exercise was also a valuable exercise for enhancing pediatric disaster preparedness among health care professionals and yielded important findings about how to improve the efficiency of pediatric disaster response. Quantitative and qualitative feedback from both participants and observers was useful for assessing the exercise's successes and obstacles, as well as identifying areas for improvement and developing recommendations for improving pediatric disaster preparedness.
Quantitative Participant Feedback
A majority of participants reported that the exercise enhanced their preparedness for managing pediatric victims in a disaster. This is consistent with previous disaster evaluations which found that simulation and training activities developed skills in teamwork and specific aspects of disaster management. However, only a small majority of participants rated performance during the exercise as “good,” reiterating the need for ongoing pediatric disaster response training to ensure continuous improvement.Reference Ferrer, Balasuriya, Iverson and Upperman 17
Based on their survey responses, the majority of participants also agreed with the adequacy of staffing, security, equipment, and supplies during the exercise. This finding may be explained by the fact that participants completing the follow-up survey came only from the pediatric facility. Participants at all three facilities were asked to complete the survey, but despite several reminders, only a subset of those at the pediatric facility participated. Participants from the pediatric facility likely received more education and training regarding issues in pediatric disaster response compared to those at the nonpediatric facilities, leading their perceptions and satisfaction with exercise performance to be more positive. While presence of selection bias must be taken into account when interpreting these results, qualitative feedback from participants at all three facilities revealed many opportunities for improving exercise performance, corroborating the quantitative results.
Qualitative Participant and Observer Feedback
Compared with their survey responses, qualitative feedback provided by participants during interviews revealed additional areas for enhancing exercise performance. As previously mentioned, the disparity between quantitative and qualitative feedback is likely explained by the fact that the quantitative was only completed by participants at the pediatric facility, while interviews were conducted with participants from all three facilities.
Both exercise participants and observers reported problems with communication during the exercise. Poor inter- and intra-hospital communication has been a previously reported problem in assessments of disaster preparedness at other institutions. As an example, Ferrer et al demonstrated that many hospitals expect to transfer pediatric victims to pediatric hospitals in the event of a disaster without having negotiated or delineated agreements with those pediatric hospitals.Reference Ferrer, Balasuriya, Iverson and Upperman 17 The events of September 11, 2001 (New York USA), also revealed major deficits in communication between hospitals regarding the care of pediatric victims. There was a lack of plans in place to transfer children to specialized pediatric centers or deploy pediatric providers to centers with mass pediatric victims.Reference Stamell, Foltin and Nadler 23
Observers and participants agreed on significant variability in the availability and functionality of equipment and supplies. Although most participants felt that resources within their respective facilities were sufficient for response to pediatric disaster victims, a majority of observers highlighted difficulty with disaster operations equipment, including the functionality and utilization of radios and telephones. Unavailability and inadequacy of appropriate supplies and protocols for the care of pediatric victims has been reported in previous studies. As an example, one investigation found that only six percent of all emergency departments in the United States had all the equipment identified by the American Academy of Pediatrics (Elk Grove Village, Illinois USA) as important for the emergency care of children.Reference Middleton and Burt 24 Another study reported that only 59% of emergency departments were aware that national guidelines for the preparedness of emergency departments exist.Reference Gausche-Hill, Schmitz and Lewis 25
With regard to staffing, observers noted that some participants from all facilities seemed unclear of their roles and responsibilities. In accordance with participants, observers reported that few physicians participated in the exercise, with mostly nurses and support staff participating instead. Importantly, observers and participants at all three facilities reported that disaster triage of pediatric victims could have been improved, especially at the nonpediatric centers. These findings are consistent with other studies noting that hospitals are inadequately prepared to respond to mass pediatric casualties.Reference Burke, Iverson, Goodhue, Neches and Upperman 21 - Reference Middleton and Burt 24 Many hospitals have comprehensive disaster-preparedness plans but have not fully accounted for the specialized needs of pediatric victims, as evidenced by their lack of pediatric-specific disaster plans and training.Reference Chokshi, Behar, Nager, Dorey and Upperman 2 , Reference Ferrer, Balasuriya, Iverson and Upperman 17 , Reference Gausche-Hill 22 , Reference Fox and Timm 26 , Reference Freyberg, Arquilla and Fertel 27
Recommendations for Enhancing Pediatric Disaster Preparedness
Based on their evaluations of the disaster exercise, observers and participants provided valuable suggestions for improving pediatric disaster training exercises and readiness. Many of these recommendations align with those of previous investigators and public health experts, and their implementation will be critical for improving the state of pediatric disaster preparedness. First, observers recommended that both pediatric and nonpediatric hospitals conduct pediatric disaster exercises on a regular basis to practice the management of pediatric victims. The ongoing process of exercise planning, implementation, evaluation, and quality improvement is critical for optimizing pediatric disaster preparedness.Reference Ballow, Behar, Claudius, Stevenson, Neches and Upperman 14 , Reference Fox and Timm 26 , Reference Gebbie, Valas, Merrill and Morse 28 Additionally, both exercise participants and observers of the exercise suggested that hospitals implement regular assessments of equipment to ensure the presence and functionality of all resources necessary for the management of pediatric victims.Reference Middleton and Burt 24 , Reference Gausche-Hill, Schmitz and Lewis 25 Participants and observers highlighted the need for clarity of roles and responsibilities of each player in disaster scenarios, including designation of leadership roles and who to access with questions or emerging problems. Further, the need for increased physician participation was apparent; one physician who participated recommended clearly communicating prior to the drill what the physician role should be and guiding them through that role during hands-on training activities. Ensuring that clinicians receive adequate disaster-response training and preparation is critical for maximizing hospitals’ response capacities.Reference Chokshi, Behar, Nager, Dorey and Upperman 2 , Reference Ballow, Behar, Claudius, Stevenson, Neches and Upperman 14 , Reference Gausche-Hill, Schmitz and Lewis 25
Finally, observers emphasized the need for enhanced and updated pediatric disaster preparedness education and training tools, echoing suggestions made by previous researchers and experts in disaster preparedness.Reference Gausche-Hill 22 , Reference Severin 29 Many pediatric disaster preparedness resources, including national guidelines, free preparedness toolkits, disaster Job Action Sheets, and computer-based disaster simulation exercises, are publically available and may enhance hospital-based preparedness education and training. 30 , 31
A Novel Approach to Evaluating Pediatric Disaster Preparedness
Mixed methods approaches have not been used widely in the study of pediatric disaster preparedness. To the authors’ knowledge, this study is the first to utilize such an approach for the evaluation of pediatric disaster preparedness within varied hospital settings. Past investigations have used mixed methods approaches to evaluate disaster exercises in schools, as well as to assess the use of telecommunications for pediatric disaster response, but not to assess pediatric preparedness in the hospital setting.Reference Burke, Berg and Vee 15 , Reference Ramirez, Kubicek, Peek-Asa and Wong 32 In the present study, gathering both quantitative and qualitative feedback from exercise participants and observers permitted a more comprehensive understanding of current deficits in pediatric disaster preparedness. New insights were also gained by analyzing feedback from exercise participants and observers in conjunction, allowing for the development of evidence-based recommendations for future pediatric disaster exercises and preparedness efforts. Findings of the present study may be valuable for improving future disaster exercises, enhancing pediatric-specific training tools, especially for facilities unaccustomed to providing care to children, and strengthening the ability of all health care facilities to manage pediatric victims in the event of a disaster.
The present study is also novel in its examination of differences in pediatric disaster response at pediatric versus nonpediatric centers. To the authors’ knowledge, a mixed methods approach has not been utilized to compare disaster preparedness at these two types of centers. Important aspects of preparedness at nonpediatric centers that remain to be improved were identified; ongoing monitoring of these areas may be valuable for ensuring that all hospitals, regardless of designation, are equipped to deal with the specialized needs of pediatric disaster victims.
Limitations
In interpreting findings from the present study, several limitations should be noted. Since exercise participants and observers were self-selected volunteers, selection bias is a concern, and the included sample may over-represent individuals who have strong opinions about, or previous experience in, disaster response. As previously noted, another source of voluntary response bias arose from the fact that participants completing the quantitative survey included only a subset of those from the pediatric facility. This is likely due to the fact that surveys were distributed by each facility's administrators for optional completion by those who participated in the exercise. If survey responders had included participants from the nonpediatric facilities, participant feedback would have been significantly less positive, with responders noting reduced satisfaction with exercise performance and reduced agreement with the statements in Table 1. Capturing quantitative responses of participants at the nonpediatric facilities may have revealed additional gaps in hospitals’ pediatric preparedness; inclusion of these individuals in future pediatric disaster exercise evaluations is warranted. Finally, all feedback from observers and participant interviews was self-reported on observation forms. Inter-rater reliability was not established in the analysis of observer feedback.
Conclusion
The present study has identified several opportunities to enhance hospitals’ readiness to manage and care for pediatric disaster victims. Analysis of quantitative and qualitative feedback from exercise participants and observers revealed the need to expand the participating hospitals’ communication, designation of responsibilities, pediatric safety, disaster triage, and emergency operations center equipment and supplies. Based on the observed areas for improving pediatric preparedness, observers provided valuable recommendations for improving pediatric disaster preparedness at both pediatric and nonpediatric health care facilities. Implementation of these recommendations may enhance the effectiveness of disaster training programs and exercises, thereby strengthening the readiness of hospitals to respond to pediatric disaster victims during a mass-casualty incident.
Supplementary Materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1049023X14001137
