Introduction
A crucial component of a hospital’s disaster plan is an efficient staff recall communication method to ensure adequate staffing in the face of a sudden influx of patients. After the November 2015 terrorist attacks in Paris (France), the Level-1 trauma center Groupe Hospitalier Pitié-Salpêtrière had resuscitation capacity for 19 patients and received 53 casualties, including 28 immediate emergencies in less than six hours.1 In situations such as this, extra hospital staff must be promptly called as back-up.
Many hospitals and health care services use a “calling tree” type protocol to call personnel back to work. In such protocols, a small number of people, usually department heads, begin by calling several staff members each, who then call several more, and the number of staff contacted expands as the “tree” branches out.2 This method is simple and allows direct contact with each individual, thereby avoiding uncertainty about whether staff received the desired message. Drawbacks include that it is slow and tedious,3 and that both landline and cellular phone services frequently become compromised in disaster situations.4 Lastly, staff contact lists may not be updated consistently, and this is often only noted during a disaster.
Improvements to this traditional method of calling individual staff members have been proposed and explored in various drills and real-life situations. The department of anesthesiology at the Jefferson Medical College in Philadelphia, Pennsylvania (USA) conducted a staff recall drill utilizing Short Message Service (SMS) texts sent to staff members’ phones and recorded a response rate (50%) as well as staff answers estimating how long they would take to arrive at the hospital given their location.5 The emergency medicine chief residents of New York University (New York City, New York USA) spontaneously used the group texting app GroupMe (GroupMe; New York USA) to communicate with all residents and successfully recalled 15 residents to work within 45 minutes during Hurricane Sandy in October 2009.6 The literature demonstrates that important issues to consider when designing a new staff recall method include using technology that people are already familiar with,6 conducting drills multiple times (including at a time when lines of communication are likely to be busy, such as a holiday or weekend),5 and when possible, utilizing various modes of communication to ensure the highest possible rate of reception.7
At the McGill University Health Centre (MUHC; Montreal, Quebec, Canada), the emergency department (ED) has been using the phone tree method and conducting an annual staff recall drill to ensure its staff contact list is up-to-date. The MUHC Call Centre has developed a custom-made SMS messaging system allowing for rapid disaster notification of department heads, directors of key services, and hospital administrators simultaneously. Each department is responsible for their own staff recall procedures. The hospital SMS messaging system does not currently assist with departmental staff recall. In addition, text messages can only be sent by MUHC Call Centre personnel with the current hospital system.
Alternate staff recall methods can potentially allow more rapid communication with a larger number of people simultaneously, and possibly easier tracking of the responses, leading to a more efficient recruitment of back-up personnel.8,9,10 Instant messaging applications present several advantages: they are already used by many people, they are free to download, and they incur no additional cost to the user or hospital. Some applications allow the creation of groups of up to 250 members and can operate using a wireless internet access without requiring a functional cell phone network, which often becomes overwhelmed in disasters.
With the objective of identifying the most reliable and efficient staff recall method, an unannounced, randomized, ED staff recall drill was conducted using three different communication tools: manual phone tree, SMS, and IMA. The drill was performed at night - when there is the greatest need for back-up personnel and staff is most difficult to reach.11
The primary objective of this study was to measure the proportion of staff who respond within 45 minutes in each of the three groups. The null hypothesis of no difference between proportions of respondents was tested against the alternative hypothesis that at least one group was different from the others. The secondary objective was to measure the average response time for each of the three groups. The null hypothesis of no difference in response time between the three groups was compared to the alternative hypothesis that response time for at least one of the groups was different.
Methods
Study Design
The study design was a randomized experiment. Participants were randomized to one of three communication tools: manual phone tree, WhatsApp IMA (WhatsApp Inc; Menlo Park, California USA), and custom-made hospital SMS messaging system developed jointly by GES Technologies (Las Vegas, Nevada USA) and MUHC Information Services. Participants did not know which tool they were randomized to until the moment they received the drill communication on the data collection date. Only the co-investigators and data collectors were aware of the exact data collection date and time. A rehearsal staff recall drill (SMS and IMA) was conducted two months prior to the drill date, during the daytime, and using only the phone numbers of the co-investigators and data collectors to fine tune the data collection process. Study participants were added into the “All ED Staff Recall” IMA group as the last step of their enrolment process. The day prior to the unannounced drill, a new IMA group was created and named “This Is It.” The participants randomized to the IMA group were added onto the “This Is It” group seconds prior to the drill start time at 4:00am on the data collection date.
Ethics Review
The study obtained ethics approval for research involving human subjects from the McGill University Research Ethics Board (#2018-3698).
Recruitment
Recruitment was done through a study web site where participants could read a brief description of the study and its requirements. A first advertisement email, as well as a reminder several weeks later, were sent to the ED staff members to stimulate enrollment. In addition, posters explaining the study and the need for participants were placed in the staff lounges and bulletin boards in each ED. No reward was offered in exchange for enrollment. Participants were asked to enter their phone numbers (cell phone and landline, as well as IMA phone number if different than their cell number). They also had to fill out a brief online questionnaire about their phone habits.
Inclusion and Exclusion Criteria
Registration in the study was open to emergency physicians, emergency medicine residents, nurses, patient attendants, pharmacists, clerks, and administrative personnel working in the departments of emergency medicine of the following MUHC hospitals at the time of participant enrollment: the Montreal General Hospital, a Level-1 trauma center; the Royal-Victoria Hospital, a tertiary care center with expertise in cardiology, oncology, and organ transplant; and the Montreal Children’s Hospital, a pediatric Level-1 trauma center. Participants consented to save the MUHC Call Centre numbers in their contacts and authorize visual and sound effects for messages sent by the MUHC Call Centre and the IMA. In addition, they were required to let a member of the study team arrange the settings on their cell phone to ensure that SMS messages from the MUHC Call Centre would produce the same type of notification at night as they would during the daytime. There were six trained study team members responsible for completing the cell phone verification process for all the study participants. This process also required to check that the proper phone number was entered in the registration web site. Staff members not owning a smart phone with SMS messaging capabilities and those on leave of absence or maternity leave during the study period were excluded. To complete their enrollment, all participants had to download the IMA on their phone and join the “All ED Staff Recall” group prior to randomization.
Randomization
Simple randomization was performed at the end of the enrollment period. Using a true random number sequence generator,12 a sequence of N integers (range 0 to N-1) was generated. Each participant was assigned a number (x) from the sequence by matching the participant list rank to the sequence rank (1:1). For each participant, the group was determined by calculating in which tercile of the range (N/3) x corresponded to. This created a 1:1:1 randomization between groups.
Measures
Data regarding age, gender, profession, number of mobile apps used weekly, prior use of the IMA, and presence of a landline phone at home were collected for all participants through the study web site at time of enrollment. In addition, cell phone habits at night were also assessed through the study web site at time of enrollment. Specifically, participants were asked if they turn the power off on their cell phone when going to bed at night, if they leave sound and visual notifications on, and if they use the airplane or do not disturb modes.
The communication drill message was considered to have been received once the participant was reached over the phone or when the participant replied to the staff recall message by sending an SMS or a message on the IMA, as per the instructions received. Four data collectors performed 11 calls each in the phone tree arm and recorded the following information manually or electronically on a standardized data collection form: whether or not the participant was reached, how that person was reached (cell phone or landline), and at what time. Participants in the SMS arm were directed to respond by text message to a specific cell phone number, whereas those randomized to the IMA group had to respond directly on the “This Is It” group. Participant responses and associated times in the SMS and IMA groups were noted in a Microsoft Excel (Microsoft Corp.; Redmond, Washington USA) spreadsheet. The discussions on the “All ED Staff Recall” and “This Is It” groups were exported and saved on the principal investigator’s computer on the day of data collection to facilitate future reference. An official SMS drill report was produced by the MUHC Call Centre who was responsible for launching the SMS communication drill at 4:00am. This report allowed verification to which participants the SMS drill message was sent.
Statistical Methods
For the purpose of sample size calculation prior to recruitment, the average response rate to night time staff recall communications was expected to be around 50%. To capture one group more effective by 30% (which means that group would have an 80% response rate), and using a power of 80%, 87 people were estimated to be needed in total (29 in each group). To allow for possible participant drop-out, 40 participants were recruited to each group.
Data analysis was performed using “R: A language and environment for statistical computing” version 3.4.3 (R Foundation for Statistical Computing; Vienna, Austria).
For the primary outcome, differences between the proportions of responders in the three groups were evaluated using the Pearson Chi Square Test. If the overall chi-square statistic was found to be significant, analysis was to proceed using the proportion test to evaluate the difference between the best group and the others. Differences between groups were expressed in 95% confidence intervals for effect size. For the secondary outcome, differences in time to response between the three groups was tested using the Kruskal-Wallis test. P values of less than .05 were considered significant for all statistical tests.
The statistical methodology and research hypotheses were fully specified prior to data acquisition. No additional unreported statistical tests were performed.
Results
Recruitment and Randomization
One-hundred forty-four participants registered through the study web site. Twelve entries were excluded either because of duplicate registration, participant withdrawal before randomization, or exclusion criteria were met. One-hundred thirty-two participants completed the cell phone verification process and were added onto the “All ED Staff Recall” IMA group. Of note, there were 11 discrepancies between the phone numbers entered on the web site by participants and their actual correct phone number obtained through the verification process. All identified errors were corrected prior to randomization. Forty-four participants were randomized to each arm. There were no significant differences in baseline characteristics between the three groups (Table 1) other than the randomization of more attending emergency physicians to the SMS group. Table 2 summarizes the night time cell phone habits of the study participants.
Table 1 Baseline Characteristics of Participants
Abbreviation: SMS, Short Message Service.
Table 2 Night Time Cell Phone Habits of the Study Participants
Abbreviation: SMS, Short Message Service.
The communication drill was performed on December 14, 2017 at 4:00am. Both the SMS and the phone tree arms were launched at 4:00am, while the IMA arm was launched at 4:02am due to logistical difficulties. Two participants in the SMS group did not receive the SMS communication drill message due to a programming error. One participant in the IMA group did not receive the communication drill message due to failure to correct an erroneous phone number, despite its identification during the cell phone verification process.
Primary Outcome
In the telephone group, 18 (41%) responded within 45 minutes. In the IMA group, 11 participants (25%) responded in the first 45 minutes. In the SMS group, seven participants responded in the first 45 minutes (16%; Figure 1). There was a significant difference in the proportion of respondents at 45 minutes in the three groups (Chi-Square Statistic=7.1 on two degrees of freedom; P=.029). The manual phone tree was found to be significantly better than SMS with an effect size in this study of 25% (95% confidence interval for effect: 4.6% to 45.0%; P=.018). Conversely, there was no significant difference between manual phone tree and IMA with an effect size of 16% (95% confidence interval for effect: -5.7% to 38.0%; P=.17). In the manual phone tree group, seven of 18 (39%) responses were obtained through the secondary phone number (landline at home).
Figure 1 Proportion of Responses to the Staff Recall Communication Received at 45 Minutes. Abbreviation: SMS, Short Message Service.
Secondary Outcome
Median response time for the phone group was 8.5 minutes (range: 2.0 to 8.5 minutes). Median response time for the SMS group was 152 minutes (range 2.0 to 336 minutes). Median response time for the IMA group was 104 minutes (range 1.0 to 458 minutes; Figure 2).
Figure 2 Median Response Time. Abbreviation: SMS, Short Message Service.
Kaplan-Meier curves illustrating the response times for each arm are illustrated in Figure 3. They must be interpreted considering the following adverse event that occurred during data collection. At 6:27am, a participant randomized to the IMA arm responded to the drill message on the “All ED Staff Recall” group, probably by mistake or confusion because this person also responded a minute later at 06:28am on the “This Is It” group used for the communication drill. All the responses that came after 06:27am (145-minute response time) in all groups could potentially have been affected by the IMA message posted at 6:27am on the “All ED Staff Recall,” which all participants could read, regardless of the group they were randomized to. There was a significant difference between the three groups (Kruskal-Wallis chi-squared=19.5 on two degrees of freedom; P=.000006).
Figure 3 Kaplan-Meier Curves Representing the Distribution of Responses Over Time. Abbreviation: SMS, Short Message Service.
Discussion
To the authors’ knowledge, this study is the first randomized, controlled, multidisciplinary hospital staff recall communication drill conducted at night for members of a clinical department. Reaching personnel at night is much more challenging than during the daytime,11 which is why 4:00am was chosen as the drill start time. The reduced personnel on-site at night creates a greater need for back-up in case of a mass-casualty situation during that period.
With regards to the primary objective, the null hypothesis of no difference between the three groups with regards to proportion of responses at 45 minutes was rejected. The results suggest that manual phone tree and IMA notification were equally effective, and both were more effective than SMS. Regarding the secondary objective, the null hypothesis of no difference between groups in the response time was rejected. The study suggests a significant difference between response time with phone calls being faster than either IMA or SMS.
Only 57 (43%) participants declared having a landline phone at home. Despite that, the phone arm outperformed the other groups with statistical significance achieved when comparing the results with those of the SMS group. This may be because a phone ringing (or vibrating if it’s a cell phone) may awaken someone more effectively than a brief sound produced by an SMS or a message on the IMA. In addition, data collectors were aware of the drill date and time for logistical reasons. In real-life, callers may not be prepared to launch the phone tree at 4:00am. They may have to look for the document containing the phone numbers, they may need briefing on how to do it, and there may not be enough callers available at night to complete the calls in a timely fashion.
As for the difference in response time distribution between the IMA and SMS groups visible on the Kaplan-Meier curves in Figure 3, it is hypothesized to be due to the “ping festival” that occurs when several members of an IMA group post messages back to back. The repeated pings potentially triggered other participants to wake up and respond. Alternatively, the large number of messages visible on the phone upon waking up caught their attention more than one unread SMS message did. This is best illustrated between 70 minutes and 110 minutes (5:10am and 5:50am) after a long period without any responses, presumably because many people were sleeping. The participants that responded during that period most likely did not get woken up by the initial communication message and responses. It is likely that the message was seen by a participant who woke up early (shortly after 5:00am) for reasons not related to the drill. That participant responded on the “This Is It” IMA group, which triggered a ping festival in the early morning when people possibly have a lighter sleep or are awake but have not checked their phone yet. Alternatively, it could be the large number of messages on the IMA sent by colleagues with the associated visual notifications that caught the attention of participants who woke up by other means than the IMA sound notifications and checked their phone. This phenomenon does not exist with the hospital SMS system where individual responses are not heard nor seen by others receiving the same SMS message. Given that the participants were randomized, it appears unlikely that the difference in response time distribution would be solely due to more IMA group participants waking up earlier by other means than the drill messages compared to the SMS arm.
Response rates and response times are not the only variables to consider when choosing a communication method for staff recall, although these are the variables that were assessed in this study. The time required to launch the staff recall procedure at night needs to be considered, as well as the ease of documentation of responses received. The fact that all staff members must agree to learn and adopt new technology, such as a new phone app, may be an obstacle. Also, the app must not be too disruptive (referring to the ping festival described previously), or else staff members will likely stop using it. Furthermore, a very large group of coworkers may not all fit into one IMA group, which is limited to 250 participants for the tool studied. Alternatively, several groups could easily be created within a large department. Furthermore, it is best if the tool employed for staff recall during disasters is also used for regular communication during daily operations, as mentioned in the study by Laskowski, et al.6 Finally, various tools may perform differently depending on the time of the day, evening, or night, which would impact the choice of call-back tools and certainly merit further study.
Limitations
This study has several limitations. Although analysis of the primary objective showed that manual phone tree and IMA were equal, the confidence interval was large. Likewise, although the advantage of the manual phone tree over SMS was significant, it too had a large confidence interval. This suggests that the findings should be interpreted with caution and may require further study with a larger sample size.
Participants were recruited on a volunteer basis. It is possible that participants most likely to try and adopt new tools or technology were recruited. Therefore, the group studied may not adequately represent staff members of a typical ED. Furthermore, there were twice as many participants on night shift duty during data collection in the IMA arm (six) compared to three in the SMS messaging and phone tree arms. Finally, the study consisted of only one communication drill. It is possible that conducting several drills would have eliminated technical problems for participants not entirely familiar with the use of an IMA or unsure how to respond to the SMS drill message which required sending an SMS to a different phone number. These technical obstacles were non-existent in the phone group.
Conclusion
This unannounced, randomized, controlled, ED staff recall communication drill conducted at night compared three tools (manual phone tree, IMA, and custom-made hospital SMS system). Both the phone tree and IMA groups had a significantly higher response than the SMS group. There was no significant difference between the proportion of responses at 45 minutes in the phone tree group and the IMA group. The distribution of response times was different in the three groups. This study suggests that an IMA may be as effective as a manual phone tree in reaching ED staff members within 45 minutes. Further studies are needed to fully understand the performance differences between the different tools and to determine whether the findings obtained in this communication drill are generalizable to other centers.
Acknowledgements
This publication is the result of a thesis submitted in partial fulfilment of the requirements for the degree of Master of Science in Disaster Medicine (European Master in Disaster Medicine). The authors wish to express their sincere thanks and appreciation to Mrs. Lillia Glykis, Chief of Operations at the MUHC Call Centre, for her significant contribution to this study.
