Introduction
Disasters and outbreaks regularly have devastating effects on societies and populations. 1 According to the Center for Research on the Epidemiology of Disasters and Emergencies (CRED) and The United Nation’s Office for Disaster Risk Reduction (UNISDR), climate-related and geophysical disasters between 1998 and 2017 killed 1.3 million people and affected an additional 4.4 billion who were either injured, became homeless, displaced, or in need of emergency assistance. 1
To assist affected countries, an increasing number of national and international Emergency Medical Teams (EMTs) have been deployed. These are groups of health practitioners who provide healthcare to disaster-affected populations. However, these EMTs were relatively disorganized for many years. They had no standards to follow and it was not possible to ascertain the quality of the service they provided. Reference Brolin, Hawajri and von Schreeb2 Therefore, in an effort to harmonize, standardize and improve the quality of EMTs work in affected areas, the ‘EMTs initiative’ evolved in 2010 under the umbrella of the World Health Organization (WHO). As of October 2016, there were 59 registered EMTs, 14 of which completed the verification process and are completely classified according to the new WHO classification system for EMTs. 3
In February 2016, the European Union (EU) launched the European Medical Corps (EMCs) to help mobilize medical and public health teams and equipment for emergencies inside and outside the EU. According to the European Civil Protection and Humanitarian Aid Operations, the EMC will “significantly increase the availability of doctors and medical equipment in response to emergencies, and allow for better response planning and preparations.” 4
While there is potentially great value in deploying EMTs and EMCs to provide medical support in disaster stricken areas, studies have shown that such medical teams frequently lack the essential knowledge and competencies required to provide effective assistance. Reference Djalali, Ingrassia and Della Corte5,Reference Amat Camacho, Hughes and Burkle6
There have been a range of academic and non-academic inputs into the adaptation of professional competencies to low resource and disaster settings where EMTs will work, but till date no proposition has fully met their requirements. Reference Brolin, Hawajri and von Schreeb2 Both researchers and practitioners have emphasized the need to integrate appropriate training as an essential component of preparing EMTs to deploy. Reference Wenji, Turale, Stone and Petrini7,Reference Veenema, Griffin and Gable8
Significant shortcomings in leadership capacities and ineffective coordination within and between teams have been identified as problems requiring specific training to solve. Reference Djalali, Ingrassia and Della Corte5 Tzeng et al, Reference Tzeng, Feng and Cheng9 stressed the need to enhance levels of confidence in multidisciplinary collaboration with team members during disaster response, in order to improve the function of medical teams when deployed outside of their routine facilities. Moving forward, team work training for EMTs needs to be well defined in terms of scope, curriculum and teaching modalities.
To enhance the efficient provision of EMC/EMT field team work, the Training for Emergency Medical Teams and European Medical Corps (TEAMS) project was established. Its objective was to develop, pilot and assess a standardized, validated and cost effective training package which was adaptable to different types of EMCs/EMTs, focused on operational team training for EMCs/EMTs, and was sustainable within low income countries and resource poor settings. The outlines of the TEAMS Training Package are presented in Table 1.
Table 1. Outlines of the TEAMS training package
The purpose of this study was to assess the effectiveness and quality of the TEAMS training package in 2 pilot training programs implemented in Germany and Turkey, based on 2 aspects: (a) assessment of the effectiveness of the training package to increase self-efficacy and team work, and (b) assessment of the quality of the pilot training events organized.
Methods
Study Design
This study examined the change in 3 major constructs, i.e., team work, perceived self-efficacy, and perceived quality of training, among participants of the TEAMS Training (see following). The comparison was conducted for each participant responding to questionnaires administered before and immediately after the training.
The TEAMS Training Package
The TEAMS Training Package and Platform were designed to support the development and improvement of an EMTs’ team work. Through a series of 8 exercises, EMT personnel will be able to train for scenarios likely to be met on the field, while focusing on the importance of team work in achieving their goals.
The TEAMS Training Package is comprised of a set of 8 innovative blended-learning teaching materials and simulation-based exercises. Each exercise is a complete stand-alone module consisting of a concept note, learning objectives sheet, debriefing tool, and a variety of supplementary documents aimed at facilitating the exercise such as ‘injects’ (unscheduled injection into the simulated scenario), annexes, reading materials and gaming accessories.
The TEAMS Trainings
The training exercises’ components of the TEAMS Training Package were recently put to the test in Germany and Turkey in the context of the first and second pilot training exercises (respectively) within the TEAMS Project. The training in Germany took place in Irsee between September 3 and September 6, 2018 and was conducted by Humedica, a WHO-certified Type 1 Fixed EMT. The training in Turkey took place in Istanbul between October 22 and 25, 2018 and was conducted by Istanbul Medeniyet University, which overlooks the activities of a Type 2 EMT. During these pilot trainings, all 8 exercises comprising of the TEAMS Training Package were performed and evaluated.
Population and Sample
Overall, 19 German and 29 Turkish training participants completed the TEAMS training package. In each training exercise there were 3 trainers while the remainder of the participants were trainees (EMT physicians, nurses, logisticians, coordinators, etc.). Of the total, 14 participants (29.2%) were women (11 German and 3 Turkish). All participants in the training and subsequent evaluations were EMT employees/volunteers who are expected to be deployed to disaster-affected areas upon need. All participants were invited to be included in the evaluation’s sample and all chose to partake. Informed consent was obtained from all participants.
Variables and Tools
The evaluation of the TEAMS training program focused on 3 main constructs: (a) Self-efficacy—this index measures individual perceptions of the team’s capabilities to galvanize motivation, cognitive resources, and courses of action needed to meet given situational demands; (b) Team work skills—this index measures individual perceptions of leadership, team dynamics, situation awareness, and effective task management; and (c) Quality of Training—this index measures individual perceptions of the overall efficacy, appropriateness, and contribution to the team. A slightly different questionnaire was used to assess quality of training by trainees versus trainers.
Assessment of the selected variables was conducted using validated and/or original measurement tools created or adapted for the purpose of this evaluation: (a) Self-efficacy of the team was assessed using an adapted version of a scale developed by Chen, Gully, and Eden (2001). Reference Chen, Gully and Eden10 In the current evaluation data, this scale scored sufficiently high on the reliability scale (Cronbach’s alpha = 0.915 and 0.787 before and after the training, respectively); (b) Team work was assessed using the validated tool “Team Emergency Assessment Measure” developed by Cooper, et al. (2010). Reference Cooper, Cant and Porter11 This scale scored sufficiently high on the reliability scale (Cronbach’s alpha = 0.890 and 0.795 before and after the training, respectively); and (c) Quality of training was assessed using a questionnaire specifically designed for the purpose of this evaluation (Cronbach’s alpha = 0.816).
All assessment tools were based on Likert-scale measurement. Self-efficacy and Quality of training were assessed using a Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly Agree). Team work was assessed using a Likert scale ranging from 0 (Never/hardly ever) to 4 (Always/Nearly always). Tools are available upon request to the corresponding author.
Procedure
Participants were informed during the first day of the training week about the evaluation process and its purpose. Informed consent was requested from all participants willing to partake in the evaluation process. Subsequently, participants were asked to complete the first round of data collection by completing the self-efficacy and team work questionnaire. The information collected at this stage was considered as the ‘pre-training’ data. Upon the completion of the last day of training, participants were asked to re-take the self-efficacy and team work questionnaires, as well as to complete the Quality of Training questionnaire. The information collected at this stage was considered as the ‘post-training’ data. For the sake of cross referencing responses, participants were asked to indicate a short designated ID tag on their questionnaire in a manner that will allow matching of the data without compromising their anonymity.
In Germany, the original English versions of the questionnaires were used. In Turkey, all questionnaires were translated into Turkish and were administered in Turkish. Prior to that, validation of translation accuracy was conducted by translating the Turkish version back to English by an independent translator and comparing the result to the original. Minor adaptations in language were performed to ensure adherence to original intents. The complete questionnaires in English are available as supplementary materials.
Statistical Analysis
The statistical analysis of the results was performed using IBM’s SPSS Version 25 (IBM Corp, Armonk, NY). The analysis included both descriptive and analytical methods, and the statistical tests were chosen according to variables distribution. Prior to analysis, indices were generated and their reliability was assessed using Cronbach’s alpha.
Given the small sample size, non-parametric tests were used. Spearman correlation test (with multiple comparison correction) was used to examine correlations between continuous variables. Mann-Whitney U and Wilcoxon tests were used to compare means of independent and paired categorical variables, respectively. In all statistical analyses performed, a p-value of 0.05 or less was deemed as statistically significant.
Results
Team’s Self-Efficacy
In the overall sample, out of a maximum score of 5, the mean score (N = 42) of the self-efficacy scale was 3.912 ± 0.655SD (95% CI: 3.708, 4.116) prior to training and 4.580 ± 0.369SD (95% CI: 4.465, 4.695) following the training exercise. This difference is statistically significant according to Wilcoxon paired samples test (W = 713.00, Z = 4.511, p < 0.001). The median values were similar to the means (3.875 and 4.625 before and after the training, respectively). An increase in the selection of the top option of the Likert scale was also observed for all items following training. See Table 2.
Table 2. Comparison of means, percentage of top option selection and their change per item of the self-efficacy scale between countries (N = 42)
a Change in mean score was computed by subtracting the mean score prior to training from the one after the training;
b Change in percentage of top option was computed by subtracting the proportion of top option prior to training from the one after the training.
A residual variable of the difference in self-efficacy was computed by subtracting the mean score of self-efficacy before the training from the score afterwards. There was no difference demonstrated between men (0.69 ± 0.75SD) and women (0.61 ± 0.59SD) for this measurement according to the Mann-Whitney U-test (U = 200.00, Z = 0.314, p = 0.768). There was no difference observed also between German (0.42 ± 0.62SD) and Turkish (0.82 ± 0.71SD) participants (U = 271.50, Z = 1.648, p = 0.099).
There was no correlation observed between age and perception of a team’s self-efficacy neither before (r(42) = 0.296, p = 0.057) or after the training exercise (r(42) = 0.065, p = 0.684), according to the Spearman Correlation test.
Team Work
In the overall sample, out of a maximum score of 4, the mean score (N = 45) of the team work scale was 3.085 ± 0.591SD (95% CI: 2.890, 3.271) prior to training and 3.556 ± 0.339 SD (95% CI: 3.460, 3.672) following the training. This difference is statistically significant according to the Wilcoxon paired samples test (W = 890.00, Z = 4.209, p <0.001). The median values were similar to the means (3.182 and 3.636 before and after the training, respectively). An increase in the selection of the top option of the Likert scale was also observed for all items following training. See Table 3.
Table 3. Comparison of means, percentage of top option and their change per item of the team work scale between countries (N = 45)
a Change in mean score was computed by subtracting the mean score prior to training from the one after the training;
b Change in percentage of top option was computed by subtracting the proportion of top option prior to training from the one after the training.
A residual variable of the difference in team work was computed by subtracting the mean score of team work before the training from the score afterwards. There was no difference shown between men (0.55 ± 0.80SD) and women (0.43 ± 0.61SD) for this measurement according to Mann-Whitney U-test (U = 182.00, Z = −0.861, p = 0.389). There was no difference observed also between German (0.39 ± 0.37SD) and Turkish (0.53 ± 0.82SD) participants (U = 297.00, Z = 1.152, p = 0.249).
There was no correlation observed between age and the perception of a team’s self-efficacy either before (r(45) = 0.043, p = 0.780) or after the training exercise (r(45) = −0.005, p = 0.972), according to the Spearman Correlation test.
In addition, item 12 on the scale prompted participants to assess the global rating of the team’s non-technical performance on a scale of 1 to 10. The overall mean rating was 7.684 (± 1.612 SD) prior to training and 8.584 (± 0.805 SD) following the training. This difference was statistically significant according to the Wilcoxon Test (W = 269.000, Z = 2.900, p = 0.004).
A residual variable of the difference in responses to item 12 (global rating of the team’s non-technical performance) was computed by subtracting the mean score of this item before the training from the score afterwards. There were no differences observed between the countries (p = 0.126) and genders (p = 1.000) for this measurement.
Quality of Training
The quality of training was assessed once, at the end of each pilot training exercise, by all participants (N = 48). The overall mean score of the quality of training scale was 4.443 (± 0.671 SD). There were no differences observed between men (4.595 ± 0.308SD) and women (4.076 ± 1.087SD), according to the Mann-Whitney U-test (U = 320.00, Z = 1.867, p = 0.062) were observed. Turkish participants (4.651 ± 0.263SD) evaluated the training higher than German participants (4.127 ± 0.945SD), according to the same test (U = 416.50, Z = 2.984, p = 0.003). The quality of training scale is not correlated with age or the self-efficacy and team work scales (data not shown).
Both trainers and trainees assessed the quality of the training as equally high (4.431 ± 0.322SD for trainers and 4.445 ± 0.709SD for trainees). Overall, 75.9% of trainees and 100.0% of trainers thought that this training was effective and useful to the team. See Table 4.
Table 4. Means and percentage of top option selection per item of the Quality of Training questionnaire according to role (N = 48). Mutually exclusive items on the trainers versus trainees versions of the questionnaire are indicated with grey background
Participants were also prompted to provide open text feedback. The data here suggests that participants held a positive attitude towards the quality of the training. The verbal input of German participants indicated that the aspects that were to be preserved are the mix of tabletop and practical exercises, the engagement of team members, roleplaying, and realistic scenarios. The most pressing issue to improve was the provision of a more detailed explanation to trainees of the individual exercises and their goals prior to performing them. The verbal input of Turkish participants indicated that aspects to be preserved are the team work nature of the TEAMS package and the manner in which it endows knowledge, experience and confidence. The Turkish participants found some of the scenarios unrealistic and suggested that they should be reviewed and adjusted accordingly.
Discussion
Previous studies have shown that sometimes, EMTs deploy to communities struck by disasters, and are unprepared to provide the required services. Reference Djalali, Ingrassia and Della Corte5,Reference Amat Camacho, Hughes and Burkle6 Successful provision of medical services by EMTs is dependent on appropriate training of the staff prior to their deployment. Reference Walmsley and Blum12,Reference Albina, Archer and Boivin13
Most especially, there is a need to empower the EMTs to work as close-knitted teams before and during operation in unfamiliar environments, adapt professional competencies to limited-resource conditions, and support each other through multi-disciplinary skills and experience. Reference Albina, Archer and Boivin13,Reference Lam, Gundran, Lopez, Santamaria, Tuazon and Enabling Factors14
The importance of facilitating functions of the EMTs when operating under such challenging conditions prompted the development of designated training packages in the TEAMS project. The evaluation of the TEAMS pilot training in Germany and Turkey indicates overall positive attitudes of participants toward the TEAMS Training Package. The data indicates that participants improved their perception of self-efficacy and team work following the training, suggesting that the training has a positive effect over those perceptual constructs among participants. These findings are aligned with previous studies which presented that simulation training programs contribute to enhancement of self-efficacy, communication skills, team work and leadership competencies of healthcare teams. Reference Watters, Reedy, Ross, Morgan, Handslip and Jaye15,Reference Chiocchio, Rabbat and Lebel16 As it was shown that self-efficacy is related to the ability to successfully perform tasks during disasters, Reference Rahmati-Najarkolaei, Moeeni, Ebadi and Heidaranlu17,Reference Zagelbaum, Heslin and Stein18 utilization of the training package is expected to contribute to the preparedness of EMTs for deployment.
In contrast to previous studies, gender was not found to correlate with either team work or self-efficacy before or after the training programs. For example, Rosander and Johnson (2017), Reference Rosander and Jonson19 found that “gender is a moderator for professional confidence” and that men were more confident compared to women when acting as ambulance incident commanders. Ross et. al. (2018), Reference Ross, Redman and Mapp20 also found that following training programs, women perceived a lower capacity to apply a tourniquet compared to men, even when their actual competency did not differ.
In summary, the TEAMS training package appears to be a high quality product, which was considered by its users to be a useful and appropriate tool for addressing their perceived needs. The similarity of the findings following the implementation of the training packages in the 2 different systems—the German and the Turkish EMTs, despite the diversity of their cultural characteristics, type and size of EMT, mix of personnel, and experience in previous deployments to disasters, suggests that the training programs may benefit many different EMTs. The newly developed package is available online, free-of-charge, to any relevant stakeholder interested in implementing it in the context of their local EMTs. By creating a validated, cost-effective training tool for EMTs, TEAMS project further contributes to the global effort to promote a more high quality EMTs system, all in the benefit of affected populations and in an effort to save lives in emergencies.
Limitations
This study has 1 major limitation, i.e., the small sample size attributed to the number of participants in the training programs. This limitation resulted in fairly large standard deviation, rendering the conclusion making difficult. To overcome this obstacle, non-parametric tests were used; however, conclusions should be noted with caution. Given that this paper describes the outcomes of a pilot study, we would propose to explore further validity of the results with other EMTs choosing to train with the TEAMS Training Package.
Conclusion
The TEAMS Training Package for Emergency Medical Teams is has been demonstrated to be effective in promoting EMT team work capacity, and it is considered by its users to be a useful and appropriate tool for addressing their perceived needs.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2020.359
Acknowledgments
This publication reflects the authors’ views only; the European Commission is not responsible for any use that may be made of the information it contains. The authors acknowledge the paramount contribution of the TEAMS consortium members from HCRI, The University of Manchester, Karolinska Institute, Istanbul University, Humedica, and Novareckon Ltd. The authors also acknowledge the contribution of the WHO EMT Secretariat.
Conflict of interest
The authors report no conflict of interest.
Funding statement
This study was performed in the context of the European Commission funded project ‘TEAMS.’ The authors acknowledge the financial support from Directorate-General for European Civil Protection and Humanitarian Aid Operations (Agreement Number ECHO/SUB/2016/740145/PREP/16).
