Introduction
Evidence suggests that most prehospital and hospital providers are inadequately prepared to manage a multiple-casualty incident (MCI).Reference Hsu, Jenckes and Catlett1 Establishing and maintaining optimal communication is often one of the most difficult aspects of the response. An MCI may present many obstacles in the absence of preparedness. Responders must not only triage, treat and transport victims, but may also be required to implement an (developed) incident action plan, determine and obtain needed resources, establish and implement an incident command system, and assign tasks to supporting personnel. A report by FurbeeReference Furbee, Coben and Smyth2 revealed 38% of emergency management service organizations indicated that a recent MCI (within the past two years) had overwhelmed them in terms of appropriate response and resources. Additionally, this report noted that many prehospital service organizations do not frequently conduct exercises to determine their preparedness status.Reference Furbee, Coben and Smyth2
Hospitals accredited by The Joint Commission are required to exercise their disaster plans twice yearly.3 However, less attention is paid to prehospital and first responder training. There are few objective data in the literature evaluating the effectiveness of hospital disaster drills. One report of a two-day public health preparedness training suggested improved participant pre- vs post- test scores and improved confidence regarding participants’ likely roles in an actual event.Reference Rottman, Shoaf and Dorian4 An Australian report of an audiovisual presentation of the hospital disaster plan and simulated exercise improved participants’ confidence in disaster preparedness and their pre- vs post-test scores, although the post-test pass rate was only 50%.Reference Bartley, Stella and Walsh5 Idrose et al found that the combination of lecture, simulation, and video training effectively improved participants’ knowledge of disaster plans significantly. The participant's pre- vs post-test scores increased by at least 40% in this study.Reference Idrose, Adullah, Adnan and Villa6 This mixed modality instruction method combines low cost training, diverse instructional methods, is relatively easy to conduct, and is more enjoyable for participants. Lastly, a 16-hour course using a mixed modality approach was found to significantly improve hospital providers’ test scores and comfort levels for disaster preparedness.Reference Collander, Green, Millo, Shamloo, Donnellan and DeAtley7
No studies were found that evaluated the effectiveness of a course designed to train prehospital and hospital personnel together. “Managing Multiple-Casualty Incidents: The Hospital/Prehospital Interface” course was developed to educate hospital- and prehospital- based personnel in the basic principles of MCI response and to evaluate the effectiveness of this training method.
Methods
Population and Setting
Through the Montana Bioterrorism Training Project, a component of the US Assistant Secretary for Preparedness Response's Bioterrorism Training and Curricular Development Program: CFDA 93.996, a statewide effort was made to offer MCI training to hospitals, communities, and individuals. After consultation with members of the Montana Department of Health and Human Services, Emergency Medical Services, and the Montana Regional Trauma Advisory Council, E-mail and phone recruitment for MCI training was made to communities throughout western Montana. Recruitment presentations were made at state and regional trauma conferences. Course announcements were placed in the Montana State Fire Service Training School daily newsletter. Once a training date was set, a volunteer local coordinator assisted in determining the best training site for the class and in recruitment of other key first responders and hospital personnel. Members from local fire, law enforcement, EMS, and hospital emergency departments were present at all course offerings. The institutional review board for The University of Montana granted approval of the study design and protocol and exempted the study from a full human subject review. However, all participants signed informed consent documents.
Course Development Process
Two experienced trainers developed the course using the overarching “cross-cutting” critical- event competencies for disaster training for health care workers articulated by Hsu, et alReference Hsu, Thomas, Bass, Whyne, Kelen and Green8 (Table 1). Didactic instruction was divided into four modules: communications, incident command, roles and responsibilities of the medical branch, and triage using the Simple Triage and Rapid Transport (START) method. Course content was reviewed by a panel of experts prior to actual course offerings. Dispatch audio tapes of actual emergencies and other media presentations of simulated emergencies were carefully selected and used during instruction to highlight critical prehospital and hospital emergency department roles.
Table 1 “Cross-cutting” Critical-event Competencies. Adapted from Hsu, et al.Reference Hsu, Thomas, Bass, Whyne, Kelen and Green8
Course Delivery
The Managing Multiple-Casualty Incidents course consisted of a short introduction, a written pre-test (30 min), pre-course “functional” exercise (one h) and didactic training consisting of four separate one-hour reviews and discussions on communications, National Incident Management Systems (NIMS)/Incident Command Systems (ICS), roles and responsibilities in NIMS, triage, and essential elements of an Incident Action Plan. Post-instruction activities included a functional exercise (one h) and post-test (30 min).
Pre- and Post-test Items
The written pre- and post-tests each consisted of 18 items that assessed the participants’ general knowledge of communications, ICS, and ability to accurately determine triage categories using the START method. The number of questions per test devoted to the didactic lecture material of communications, NIMS/ICS, roles and responsibilities, and triage were comparable. For testing triage competency, the physical description of the patients’ injuries in each test was different but the identical number of specific START category patients remained the same. Upon completion of the pre-test assessment, the group participated in the pre-instruction functional exercise.
Pre- and Post-Instruction Functional Exercise Tools
A computer simulation using Microsoft Visio (Microsoft Corporation, Redmond, Washington USA) was developed and used to provide a visual focus for the exercise. Google Earth (Googleplex, Mountain View, California USA) satellite imagery was used to create a visual background of the “actual” disaster site that would be recognizable to the participants. The prehospital group saw an image of the disaster scene. The hospital group was in a separate room and saw a floor plan image of the local hospital's emergency department. Easily recognizable, on-screen icons were used and superimposed on the Google Earth satellite image. The icons identified numbers and types of first responders, resources and equipment available at the scene and in the community, disaster images specific to the scenario, and injured patients. A drop-down box attached to the individual patient gave basic vital statistics that were used by “responding” triage members to calculate START scores and to determine appropriate transport priority. After the patient received their START triage “tag” designation, the patient's icon was changed to the appropriate color.
Pre- and Post-Instruction Functional Exercise Description
The MCI scenarios for the pre- and post-training functional exercises were specific to each training site community. The number of critical patients in the MCI scenario was determined based on community capacity, which the authors defined as twice the number of available emergency room beds in each receiving facility. The pre- and post training exercise scenarios differed in description (eg, bleacher collapse vs bus crash) but the number of causalities, injury severity according to proper START classification and available personnel and resources were comparable. Prior to the actual exercises, trainees volunteered or were assigned roles consistent with their professional designation or current job descriptions. During the exercises, the prehospital responders communicated with “hospital” emergency department personnel via radio or phone, thus simulating the conditions used during real emergencies.
The MCI simulation was scripted using a Master Sequence of Events list. The exercises were initiated by a call to the participants and the response communications were facilitated by an experienced 911 dispatcher. Prehospital responders were expected to organize the scene, give assignments, triage the patients, and prioritize transport to the hospital. Hospital personnel were expected to react to the information being provided from the scene and to develop and implement their own plans for receiving and re-triaging patients.
The participants were asked to manage the situation using their existing community and hospital disaster plans. Events and actions taken occurred in as “real time” as possible and the scenario ran for approximately one hour from the time the incident was reported. Use of the satellite image for the specific community allowed the instructors to judge issues such as selection of staging areas, ingress and egress from the scene, travel routes to hospital, or other factors specific to the actual geography of the community.
Data Analysis
All data were analyzed using SAS v9.0 (Cary, North Carolina USA). Baseline, pre-instruction scores were calculated for participants and summarized by gender, age, position, size of community, agency, and licensing. Differences in baseline scores across these variables were evaluated by analysis of variance. Changes in score from pre- to post-test were evaluated according to these variables by paired t-test or analysis of variance, as appropriate.
Results
Among those trained, 193 individuals completed the pre-test with an overall mean score of 9.7 (SD = 2.5). Table 2 shows the pre-test results according to various subject or community characteristics. There was no difference in pre-test scores by sex or age. Individuals who worked in fire-related positions had higher baselines scores compared with all others (P = .016). Baseline test differences were also observed for different size communities. The small and very small communities had a mean baseline test score of 9.2 (SD = 2.4) compared with a mean score of 10.2 (SD = 2.4) for moderate to large communities (P = .005). Differences in baselines scores were also observed among participants according to the type of agency they worked for and their licensing credentials (Table 2).
Table 2 Pre-training Scores on 18-question Multiple-choice Test for Selected Characteristics of Subjects or Training Sites. P value: Analysis of variance, one-way
Abbreviations: EMS, Emergency Medical Service; EMT, Emergency Medical Technician; EMT-B, EMT-Basic; EMT-FR, EMT First Responder; EMT-I, EMT-Intermediate; EMT-P, EMT-Paramedic; FFI, Firefighter I; FFII, Firefighter II; LPN,Licensed Practical Nurse; MD, medical doctor; RN, Registered Nurse
aMissing characteristic data for sex (n = 26), age (n = 24), position (n = 19) and agency (n = 17); percentages and analysis of variance based on known data for a given characteristic.
bFire = chief, assistant chief, captain, lieutenant, fire fighter, prehospital EMS; Management = administrator, director, manager; Nurse = charge nurse, staff nurse, LPN.
cTraining site as described by community capacity of training site. The number of critical patients in the training scenarios were scaled according to the community capacity by multiplying the number of available ED beds by 2.
dSubjects reporting both fire department and EMS were listed as EMS only.
eFirefighting = FFI or FFII; EMT basic = EMTFR or EMTB; EMT advanced = EMTI or EMTP; Nursing = RN or LPN; Firefighting and EMT = those with both firefighting and any EMT training, not included in other firefighting or EMT categories.
Of the 193 participants who completed the pre-test, 175 also completed the post-test. The post-test completion rate was lower because some participants had to leave training for actual emergency response purposes. Overall, participants improved by almost four points following the training, and improvements were observed in all three subject areas (Table 3). Score improvements varied according to position, size of community, agency, and licensing (Table 4). Improvements were observed among participants regardless of the size of the community. Despite the lower baseline scores observed among smaller communities, post-test scores were similar for large and small communities (Figure 1). Improvements in scores were also observed for all occupational groups (Figure 2). Firefighters improved the least, but started with the highest baseline, pre-instruction scores. The strongest overall improvement was observed for nurses (+5.3 points, P < .001).
Table 3 Overall Results for 18-question Test and Topic-specific Components Before and After Training (n = 175). P value: paired t-test
Abbreviation: ICS, incident command system.
Table 4 Improvement in Score on Pre- vs Post-training 18-question Test and Sub-scales for Selected Characteristics of Subjects or Training Sites. P value: Analysis of variance, one-way
Abbreviations: EMS, Emergency Medical Service; EMT, Emergency Medical Technician; EMT-B, EMT-Basic; EMT-FR, EMT First Responder; EMT-I, EMT-Intermediate; EMT-P, EMT-Paramedic; FFI, Firefighter I; FFII, Firefighter II; LPN, Licensed Practical Nurse; MD, medical doctor; RN, Registered Nurse
aMissing characteristic data for sex (n = 23), age (n = 21), position (n = 13) and agency (n = 11); analysis of variance based on known data for a given characteristic.
bIncludes chief, assistant chief, captain, lieutenant, firefighter, prehospital EMS.
cIncludes administrator, director, manager.
dIncludes charge nurse, staff nurse, LPN.
eTraining site as described by community capacity of training site. The number of critical patients in the training scenarios were scaled according to the community capacity by multiplying the number of available ED beds by 2.
fSubjects reporting both fire department and EMS were listed as EMS only.
gFirefighting = FFI or FFII; EMT basic = EMTFR or EMTB; EMT advanced = EMTI or EMTP; Nursing = RN or LPN; Firefighting and EMT = those with both firefighting and any EMT training, not included in other firefighting or EMT categories.
Figure 1 Comparison of Pre-test vs Post-test Scores for Participants in Small Communities (n = 89) Compared with Larger Communities (n = 86). Small communities are defined as community capacity ≤6 Emergency Department (ED) beds, and large communities are defined as ≥7 ED beds in the receiving hospital(s).
Figure 2 Comparison of Pre-test vs Post-test Scores According to Position. The “Fire” category includes chief, assistant chief, captain, lieutenant, fire fighter, and prehospital EMS; “Nurse” includes charge nurse, staff nurse, and LPN; “Management” includes administrator, director, and manager.
Discussion
In the two- to three-year period prior to the training described here, significant training in NIMS and ICS had been offered throughout western Montana communities. The primary aim of the training described in this report was to enhance the communications, ICS planning, and triage skills of prehospital and emergency room personnel.
The Managing Multiple-Casualty Incidents course offers several advantages. First, building training scenarios that are specific to a community helps bring a sense of reality to the exercises. In addition, the computer simulation using satellite imaging from Google Earth and resource icons reflected the true capacity of the communities’ physical and manpower resources, thus ensuring that the scenarios would create realistic logistic challenges and a more authentic emergency response simulation. Another unique feature included training and exercises in “real time” on the same scenario that incorporated representatives from each of the key emergency response disciplines, actual dispatchers, and ED personnel. This format allowed all participants to understand the individual roles and responsibilities of the other disciplines as defined in the didactic setting, and the scenarios allowed for the identification of deficiencies and ineffective response at a systems level.
This study was designed to test the competency and preparedness of prehospital and hospital workers with regard to MCI management in rural and small community hospital settings. The pre-test scores from this small study suggest that the preparedness levels are below an optimal level and the post-test scores indicate training in MCI management can significantly improve competencies, at least in the short-term. The low baseline scores might reflect inadequate training opportunities, ineffective training methods, higher personnel turn-over rates for key first-responder or emergency department staff, or other unidentified factors.
The authors observed a difference between firefighter's pre- and post-test scores that was smaller in magnitude than test scores with nursing personnel or other first-responders. It may be that firefighters’ higher baseline scores were attributed to their discipline's more consistent use of MCI practices in their daily functions or that they train more frequently in communications, ICS, or triage.
Limitations
Most of the literature that describes the methods employed for teaching content for disaster-related response lacks standardization, utilizes subjective assessments, and lacks scientific rigor.Reference Williams, Nocera and Casteel9–Reference Macintyre, Barbera and Brewster11 This study suffers from some of the same limitations common to previous studies. The participants volunteered for training, so selection bias cannot be ruled out. This study used a weaker study design, a pre- and post-test format with no “control group.” Finally, even though the post-test scores demonstrated significant numerical improvement it is not possible to state if the numerical improvement would translate into enhanced performance during a real emergency. Unfortunately, the authors were unable to secure needed funds or adequately trained observers to rigorously evaluate the functional exercise components or to reevaluate learning retention over time. Despite the limitations, this study does demonstrate that training can lead to significant numerical improvement for a diverse group of first-responders and health care workers. Anecdotally, the authors also believe the pre- and post-didactic functional exercises allowed participants to identify important aspects of their individual organization's MCI disaster plans and the needs of the greater community's response capabilities. Participants were encouraged to revise their plans based on the information revealed during the exercises.
Conclusions
Experience suggests hospital personnel and first responders seldom, if ever train jointly. The economic burden of consistently testing, training, and exercising represents a critical barrier to optimal preparedness and response, particularly in small hospitals and critical access facilities. Consequently, mistakes that could be identified and corrected with proper training are made manifest when a real emergency presents itself and these same mistakes may even be repeated over time. This small study has shown that the types of training and functional exercising the authors developed and offered may be very effective in improving outcomes. The authors also believe that this study reflects a realistic training approach that is sensible, effective, and cost-efficient for small, rural hospitals or critical access facilities.
The authors have demonstrated that: (1) existing preparedness competencies are less than optimal; and (2) this site-specific, multi-discipline, computer-modeled MCI training course, consisting of interdisciplinary didactic and functional exercise training, results in improved competencies. The authors would intuit that to sustain optimal competency, continued and periodic training will be required. There may be different levels of competency between professional disciplines requiring specific, focused training vs a one-size-fits-all training.
