Following the European Union’s (EU) response to the 2010 Haiti earthquake, it became evident that future efforts would require not only motivation and economic contribution but also a more coherent disaster response approach and management structure; one that was grounded in aggregating efforts across the multi-national union.Reference Brattberg and Rhinard 1
Disaster management (DM) includes the mitigation, planning/preparation, response, and recovery phases of a disaster and aims to counteract or minimize negative consequences. Successful DM entails leadership, cooperation, and coordination across professions, agencies, organization, and nations. Undefined roles, a lack of clarity regarding the chain of command, and poor leadership are the most frequently listed and major shortcomings in DMReference Abramson and Redlener 2 and can result in misunderstanding and chaos.
Disasters are characterized by fluctuating levels and layers of complexity and present significant challenges for responders now working in an increasingly international environment. One useful approach to adequately prepare responders is to provide a learning environment that emphasizes shared training, cooperation, and intercultural understanding. This training strategy underscores the commonality of the disaster context and the interdependence of responders rather than re-educating individuals in the skills they have already learned. Additional outcomes of this approach are the trust, sustained cooperation, and standardization of effort that will markedly improve response quality and effectiveness across the spectrum of responders, managers, and organizations.Reference Alexander 3 , Reference Subbarao, Lyznicki and Hsu 4 - Reference Galindo 7 Such are the tenets of the DITAC (Disaster Training Curriculum) project, which was designed to improve European training for international crisis management. Funded by the European Union’s Seventh Framework Programme, the DITAC project was characterized by multiple aims as follows:
-
∙ Analyze concepts, methods, and doctrines of crisis response and identify the relevant European competences of crisis management;
-
∙ Analyze existing initiatives to generate curricula for crisis management;
-
∙ Identify the requirements of local actors in crisis management education;
-
∙ Identify the needs of relevant actors and the resulting stakeholder requirements for significant improvement of training in international disaster response and crisis management;
-
∙ Develop a didactic concept to transmit common standards for crisis management education, using state-of-the-art methods for teaching and training;
-
∙ Develop a valid and reliable course evaluation tool.
Unremitting natural disasters, deliberate threats, pandemics, and humanitarian suffering resulting from conflict situations necessitate swift and effective response paradigms. The EU’s increasing visibility as a disaster response enterprise urges examination of and improvement to its response capabilities. The purpose of this article is to review the background and components of the DM approach offered by the DITAC project, to discuss the target groups best suitable for this training, to present a variety of training and evaluation methodologies, and to argue for the importance of standardizing DM training across Europe.
METHODS
The DITAC project was composed of over 35 experts from within and outside the EU countries. The experts’ specialities included emergency medicine and trauma surgery, public health, pre-hospital care, social sciences, emergency services (fire fighters and other providers), and technical expertise encompassing the breath of skills needed to inform the project and course components in areas that included planning and management, decision-making, ethics, and disaster management. Experts were also assigned to one of 5 groups or work packages (WP) as follows:
WP1: analyze current European concepts, methods, doctrines, education, and training programs in disaster management; study different curricula for crisis and disaster management; define a model of crisis management and the implied competencies; evaluate the ethical perspectives; and introduce policy recommendations.
WP2: define overall standards for crisis and disaster managers; describe suitable candidates and enrolment requirements for entry into the training course; develop a didactic concept for a curriculum on crisis management; create modules for a training curriculum; and develop an evaluation tool for a training course based on the curriculum.
WP3: implement and evaluate the pilot study course to “test” the curriculum´s concept (WP2); initiate curriculum modifications as needed; develop strategy for providing continuous external overview; and assess the ethical, societal, and security components of the course.
WP4: organize strategies for knowledge transfer and social learning within the project; disseminate WP results; and develop and implement a platform for networking existing training institutes.
WP5: ensure the scientific management of the project’s contractual responsibilities; facilitate an effective project communication infrastructure; manage the processes needed for registration and protection of intellectual properties; and ensure the dissemination of project results and knowledge.
The Consortium was assisted by 8 Senior Advisors. To ensure the robust nature of the work across each of the project’s WPs, comprehensive literature reviews were performed by using PubMed, Google Scholar, and different electronic databases at national and university libraries. Figure 1 shows the schematic approach to the survey and selection of the literature used in this article. Search paradigms incorporated the following words and topics: (1) disaster management courses using the key words “educational initiatives,” “disaster management,” “crisis management,” “courses,” and “multi-agency”; (2) standardization using the key words “standards,” “disaster management,” “crisis management,” “competency,” “multi-agency,” and “non-medical”; (3) didactic using the key words “didactic models,” “didactic concepts,” “adult education,” “vocational training,” “curriculum,” “curriculum design,” “competence,” “competence based training,” and “problem based training”; (4) teaching methods by use of the key words “teaching methods” with and without “disaster management,” “adult,” and “multiagency”; (5) evaluation methods by use of the key words “course evaluation,” “training evaluation,” “disaster management”; and (6) target groups by use of the key words “disaster management,” “lessons learned,” “stakeholders,” “organizations,” and “multiagency.”
FIGURE 1 The Literature Selection Process. DITAC, Disaster Training Curriculum.
Articles were excluded according to the following criteria: (1) case studies, abstracts, and citations; (2) articles not specifically related to the subjects; and (3) articles not dealing with disasters or humanitarian assistance.
Titles and abstracts of the identified journal articles were scanned. Literature not complying with the inclusion criteria was excluded. The full text was obtained for indeterminate articles and references were independently screened and selected by members of each group dealing with a specific topic. The selected literature for each topic was then evaluated by experts in each group. When disagreement occurred, the opinion of a third reviewer was sought. Finally, all authors reviewed around 100 relevant publications, in each group respectively, addressing both medical and nonmedical publications; the latter to avoid bias towards health-related methodologies. In some topics, such as “evaluation methods,” the number of “hits” was more than 1 million. To narrow the results to an acceptable number for further study, new key words were identified. The resulting data were studied and are presented in the Results.
WP members and project teams participated in 2 to 4 monthly conference calls across the duration of the DITAC project (2011-11-01 to 2014-01-01). General issues were selected for additional discussion at monthly meetings that included other DITAC project experts. If necessary, consultation with outside experts, in areas such as educational methods, was sought to support the project’s goals. Additionally, 7 meetings were held in various locations across Europe to facilitate discussion across all members of the project and ensure that each WP was in alignment with the other project components. The results obtained from the search of the 6 topics mentioned above are presented in 5 key themes (see below).
RESULTS: KEY THEMES
Overview of Existing Disaster Response Curricula
In a comprehensive review by Ingrassia et al (DITAC) in 2012, 140 European educational and training initiatives (ETIs) were identified.Reference Ingrassia, Foletti and Djalali 8 The majority of these initiatives were located in 3 countries (United Kingdom, France, and Germany), whereas 11 countries had no ETIs at all. Over 50% of these ETIs offered a master’s degree and were given as on-site education. Lectures were the most commonly used teaching method followed by discussion- or operations-based exercises. About two-thirds of the ETIs utilized a blending of theoretical content and exercise, but only 17% had both discussion- and operations-based exercises.
Seventy-one percent of ETIs offered a multidisciplinary program and in 20% the content was exclusively health-related. The most common topics presented in these programs were management, hazard and vulnerability analysis, logistics and transportation, law and ethics, and protection and safety. Most of the courses had a competency-based curriculum (61%) and lasted for 6 to 12 months. Courses were generally given in their native language; consequently, English was the most common language in 80% of ETIs. Speaking the official language of the course was mandatory for participation. Previous field experiences and technical education or training was a prerequisite in 18% and 26% of the ETIs, respectively.
Disaster Response Encompasses Many Disciplines and Multiple Skill Levels
Initial management of a major incident is executed by people who work in their area of competency. The European Qualification Framework defines competency as “the proven ability to use knowledge, skills and personal, social and/or methodological abilities in work or study situations, and in professional and personal development.”Reference Kane 9 - 11 In the context of the European Qualification Framework, competence is described in terms of responsibility and autonomy and is a structure composed of different elements in a hierarchy: the competency itself, learning objectives, and activities. Competence within these professional and personal knowledge and skills must be developed incrementally and include training that achieves the flexibility and adaptability usually demanded by a continually changing disaster environment.Reference Perry 12 - Reference Miles 18
Central to an integrated emergency management in many organizations is a command structure, which generally distinguishes two key levels: managerial (administrative, directorial, ministerial) and operational (functional, operational). 19 The managerial level is divided into strategic and tactical levels and has been criticized in many after-action reports for lack of good decision-making in command and control functions.Reference Brattberg and Rhinard 1 , Reference Abramson and Redlener 2 , Reference Perry 12 - Reference Miles 18 , Reference Djalai, Castren and Hosseinijenab 20 The strategic level functions distant from incident scene, whereas the tactical level is much closer to the operational level and the ground work. Both operational and tactical levels work under strategic command. Many successful disaster responses can be attributed to local knowledge and local coordination of tactical-level response elements.Reference Col 21 An improved command, control, and communication triad is central to managerial (strategic and tactical) system improvements to reduce the risks of negative outcomes.Reference Djalai, Castren and Hosseinijenab 20
Command infers the vertical leadership. The horizontal leadership, ie, “span of control,” deals with overall influence over a wide range of actions. Sound communication between these 2 levels results from effective, bidirectional information sharing that binds together the whole system.Reference Perry 12 , Reference Palttala, Boano, Lund and Vos 22 , Reference Khorram-Manesh, Angthong and Pangma 23 Not surprisingly, the most common problem associated with any disaster is a failure of communication.Reference Brattberg and Rhinard 1 , Reference Abramson and Redlener 2 , Reference Perry 12 , Reference Palttala, Boano, Lund and Vos 22 - Reference Hogan and Burstein 24
Most of the organizations directly involved in a disaster response are normally organized at strategic (gold), tactical (silver), and operational (bronze) levels. The hierarchical command in these entities, irrespective of the model, allows each organization to lead its own organization vertically but is also expected to interact with other parties horizontally to ensure better integration of activities and resources. A review of the literature revealed that around 29% of programs are aimed at the strategic or “gold” level, which includes managers and leaders or commanders in overall charge of each service and who are responsible for formulating disaster or crisis management strategy. Thirty-nine percent of the programs targeted the tactical or “silver” level, which focused on personnel responsible for formulating and adapting the tactics to their service. Operational or “bronze” staff, defined as the group who have immediate ‘‘hands-on’’ work at the crisis scene, were the target groups in 22% of programs. Three percent of ETIs covered both silver and bronze groups.Reference Abramson and Redlener 2
Social groups, economic groups, and political groups and their subgroups, respectively, constitute the key DM stakeholders.Reference Berke, Kartez and Wenger 25 Social groupsReference Ketti 26 consist of the organizational unit for emergency management, ie, the households, private sector groups such as religious organizations and other nongovernmental organizations (NGOs), nonprofit organizations, community-based organizations, and businesses. All of these groups vary widely in size, level of organizational complexity, and amount of resources available, but are all potential partners in formulating DM practices and policies.
Economic groupsReference Lindell and Prater 27 , Reference Lindell and Perry 28 are the fundamental units in the hierarchy of economic stakeholders. Destruction, damage, and even interruption of business activities can have significant adverse effects on the local economy and, in smaller countries, on the regional or even national economy. An especially important type of business, the public utility provider (private or public), consists of the providers of electricity, water, sewer services, solid waste management, and communications such as telephone, television, and internet access and are critical in the restoration of basic services to their customers and communities. The news media is especially important to the success of DM programs because their coverage of all phases of management can be used not only for information sharing but also for educating the public about potential hazards.
Political groupsReference Depoorter 29 are important and can represent varying levels of power and political systems from state to state. DM policy is set at different authority levels in different countries, and one of the tasks of political groups is to identify at which level this authority resides and what is the division of responsibility between different levels. There are also different agencies within each level of government that differ widely in terms of size, organizational complexity, and amount of human, financial, and technical resources. At the local level of government, the agencies most involved with DM are the fire and police departments, which are the first agencies to respond to most emergencies. Some communities have a separate emergency medical services agency, but often this function is provided by the fire department working together with local hospitals and ambulance companies. One important stakeholder is the state or national emergency management agency, which can vary widely in terms of expertise, staffing, budgets, and other organizational resources. Academics specializing in specific hazards (eg, seismologists, meteorologists, toxicologists) and mitigation measures (land use planners, architects) and hazard/disaster researchers (economists, geographers, political scientists, psychologists, and sociologists) form another important stakeholder group and provide the basic scientific knowledge base on which sound emergency management policies and practices are built.
To develop an effective emergency management system, the local emergency manager must involve the relevant stakeholders in all phases of DM. Facilitating relationships among the stakeholders will strengthen horizontal linkages within the community and vertical linkages of the community with outside resources in higher levels of government and will improve the flow of information, services, and supplies during a disaster.Reference Berke 30 This includes the fire department, law enforcement, emergency agencies (civil protection), emergency medical services, ministries of public health, environment, public affairs, energy, building and housing, service and maintenance, military, and NGOs.Reference Bagaria, Heggie, Abrahams and Murray 31 - Reference Lindell and Perry 39
A strong need exists for coordinating and harmonizing efforts across all responder groups and stakeholders. Familiarity with each other’s capabilities and limitations is essential and must take place BEFORE THE DISASTER. Deficiencies in DM are often the result of a lack of integration between multiple levels across the command structure and responding agencies. Note that one of the major factors of success in the 2005 London bombing was a result of having earlier adopted a standardized command and control structure.Footnote *
Moreover, the very nature of a large-scale regional or international disaster magnifies intercultural differences across multinational responder cohorts. Coordination between different levels of response nationally and internationally is needed and can be utilized by educational and training initiatives. Multidisciplinary DM training for all EU countries will ensure a high level of professionalism, interdependence, and cooperation, consistent with the DITAC project initiative.
Importance of Standardization in Disaster Management Training
The literature contains conflicting opinions about standardization. Many studies emphasize that standardization will ensure the quality and uniformity of training, facilitate international cooperation, provide transparency, guide people involved in DM, and facilitate evaluation and continual upgrades.Reference Wiclair 40 Detractors note the perceived inflexibility imposed by standards. Moreover, organizational and administrative differences between different countries and roles of the private and public sectors might seem to present an obstacle to establishing comprehensive European DM standards.
Attempts at standardization may initially pose difficulties, because the EU consists of different countries with unique systems of government and resources. However, standardization can be achieved by identifying the common reference points and similarities in any specific area. Alexander proposed that the following functions within the complex chain of DM would benefit from standards: (1) procedures and protocols concerning the communication and information circuit within a disaster; (2) procedures and plans for creating, testing, and activating emergency protocols; and (3) education and training for disaster.Reference Alexander 41 Standards specify certain requirements and qualities of management roles and can help ensure formality, good practice, and reliability as are reflected in key documents like FEMA (Federal Emergency Management Agency) Best Practices, UN-OCHA (United Nation Office of Coordination of Humanitarian Affairs) Guidelines, and ISO/PAS (International Standardization Organization Publicly Available Specification) 2007 Benchmarks.Reference Alexander 3
Examples of national or regional standards 42 - 53 include the United Kingdom Cabinet Office Standards for Civil Protection in England and Wales, 46 United States' National Fire Protection Association NFPA 1600 Standard on Disaster/Emergency Management and Business Continuity Programs, 47 SIPROCI Project Minimum Standards for a Local Plan of Civil Protection, 48 Sphere Project Humanitarian Charter and Minimum Standards in Disaster Response, 49 the Australia-New Zealand Standard on Risk Management AS/NZS 4360:2004, 51 and the US National Incident Management System. 52
A set of minimal standards can be achieved through consensus, education, and training and can be tested to evaluate its capabilities and possibilities.Reference Alexander 3 , Reference O´Neill 14 In a stepwise fashion, developing standardized training curricula can take the approach that follows.
The scope and objectives of training and education, the target group, and the qualifications of the educators should be identified. The curriculum guidelines should be compatible with the needs of the stakeholders, and the learning objectives should be evaluated against the minimal criteria required for certification. The details of curriculum (contents), required hours per unit of education, structure (modules), course methods, and both required and elective courses needed to achieve a minimum level of competencies in any of the levels of instruction designed must be rigorously studied before being adopted. The course should enhance not limit professional development and should be consistently revised as a result of student feedback, objective evaluation criteria, and the changing disaster climate, which will demand new skills and techniques.
Identifying Optimal Didactic and Teaching Methods
There are many types of teaching methods, from traditional lectures to tabletop and simulation methods, and all have pros and cons.Reference Gossman, Stewart and Jaspers 54 - Reference Savery 82 The term blended learning has been established in literature as its own teaching method. It is basically defined as a combination of 2 or more learning methods involving face-to-face environments and remote computer-mediated environments. Despite some negative remarks, sufficient publications exist to support this type of teaching, especially because it is evaluable by use of various assessment models.Reference Oliver and Trigwell 70 - Reference Graham 74
New technologies have revolutionized the whole teaching system. The main change has been brought about by the introduction of computer-based learning. Utilization of new media sources has made it possible to use the Internet as a communication source and natural education platform.Reference Gossman, Stewart and Jaspers 54 , Reference Persson, Fyrenius and Bergdahl 55 There are two intersecting dimensions of teaching. The first is passive/active dimension of student participation. At the passive pole (“lecture/document”), the teacher sets the agenda and the student absorbs information directly, with little contribution. At the active pole, the student actually has a hands-on experience, with the teacher facilitating the learning process. Cross-cutting this dimension is a second dimension of technology-mediation. At the technology-poor pole, the student and the teacher are engaged directly. At the technology-rich pole, the teacher-student interaction is largely mediated by technology such as IT and audiovisual media.Reference Campanale 63 , Reference Ricer, Filak and Short 64
The use of multimedia scenarios in problem-based learning has also been highlighted in the literature and has proven to be an excellent tool for curriculum development.Reference Gossman, Stewart and Jaspers 54 , Reference Persson, Fyrenius and Bergdahl 55 , 80 Students in problem-based learning are confronted with authentic concerns that demand their engagement in a process to solve a problem at hand. The learning process is an interactive group effort. Problem-based learning is a specific form of a constructivist approach to education and originated in North America in the early 1970s. It has often been used in medical education, but also in teacher education, life science engineering, and many other fields. 80 - Reference Savery 82
Published data from the Agency for Healthcare Research and Quality from 21 studies in 2004 suggested that simulated drills were the most effective method for disaster medicine. There was insufficient evidence to recommend computer-based simulation or tabletop exercises as the superior choices of teaching.Reference Hsu, Jenckes and Catlett 75 , Reference Ebell, Siwek and Weiss 76 In another study, didactic lectures were compared with combinations of lectures and tabletop exercise in 85 subjects in pediatric disaster management training. The result showed increased knowledge by both models on multiple choice tests; however, those completing the combination exercise reported a greater subjective sense of confidence in their knowledge.Reference Behar, Upperman and Ramirez 77 , Reference Ward and LaBranche 78 Overall results show that simulation and practical exercises are superior to other teaching methods. The definitive choice of teaching must be adjusted to the course duration and the necessity of having lectures onsite before any practical exercises.
Today´s continuously changing working environment results in high demands by learners for new skills, attitudes, and information acquisition. Efficiency and sustainability of education must be matched with new theories of teaching and learning.Reference Vander Stoep and Louw 79 Many teaching environments have shifted from lecture-based learning to adult-based learning. This environment does not put the “sage on the stage” but rather places the instructor as a “guide on the side.” Although the methods used in a training course are the tools for the trainers and provide suggestions about the best way of teaching or learning, it must be in agreement with the didactic concept, the curriculum, and the group of trainees. In a DM perspective, trainees or participants are the group that influences the didactic concept, the methodology for the training, and the entire curriculum. The fact that trainees are adults and experienced professionals from multiple agencies assembled in an interdisciplinary group with intercultural differences presents a specific set of demands to the curriculum and the didactic framework. Adults have different learning approaches. Seasoned professionals in an international scenario master a wide range of experience, knowledge, and competencies. However, they may need to address their competencies anew with regard to flexibility and communication in order to adjust certain structures of thinking and automatic paradigms of acting. At the same time, the training itself should profit from the knowledge available from the trainees. Different organizations may need to work in different modules; thus, the methods may vary from module to module and the best way to provide an optimal learning environment may depend on factors such as the trainer, the topic, the location, and preparatory steps in the course.
In a DM cycle, it is necessary that all involved multidisciplinary units perform their specific tasks when a situation, event, or problem is evident. Thus, the training should ensure better cooperation across all involved organizations. The theoretical framework of an interdisciplinary course, its didactic model, establishes the overall theory required to analyze and plan a given teaching procedure. To achieve the overall learning objective and its specific demands, the didactic concept establishes the framework for the teaching and learning process and provides the overall concept for the learning environment and the interaction between trainers and trainees. Different variables can influence such concepts, such as the staff´s background and age, learning materials and the course duration, use of media and web-based contents, etc. These variables will set the guidelines for the entire curriculum including the evaluation and the entire process of the training. Furthermore, the didactic concept should correlate the specific goals for the training to the appropriate learning process (certificate, private versus mandatory).
The curriculum consists of all parts of the training course, including the whole preparation phase. It assembles relevant issues for DM, sorts them into individual topics and specifies the learning objectives for learners in a specific level, and combines the entire body of knowledge and competencies. Knowledge is specific information needed for understanding the relevance and the concept and interrelation of a topic. Competencies are formed by an individual’s skills, experiences, and knowledge and enable a person to perform a specific task. The curriculum is the umbrella for all topics included in the course. Topics can be presented as theoretical knowledge or practical competencies, sorted into training units or modules designed for the target group. Some may be designed for individual learning before the training starts and others may be an essential part of an on-site training.
Different agencies need different competencies. Such competencies must be extracted from a common framework related to the specific concept of operations used and level of involved people (strategic, tactical, and operational). Core competencies for DM may be related to one or more of the items within such a common framework. By reviewing existing data from earlier publications, competency domains and common core competencies should be identified. Competency domains are the necessities of DM such as the 3 C´s ie, command, control, and communication, followed by safety and security, law and ethics, phase-related issues such as planning and preparation, and recovery. Common core competencies will be extracted from these domains and will include DM-related topics such as defining a disaster and its different phases, risk and vulnerability analysis, planning, command and control, communication issues, political and ethical and religious considerations, logistics, information sharing, etc. Furthermore, learning objectives for each topic can be defined and converted into activities that build up the course at different moments matched with the knowledge, skills, and competency needed for each group.Reference Brattberg and Rhinard 1 - Reference Abramson and Redlener 2 , Reference Subbarao, Lyznicki and Hsu 4 - Reference Walsh, Subbarao and Gebbie 6 , Reference Perry 12 , Reference Miles 18 , Reference Palttala, Boano, Lund and Vos 22 - Reference Berke, Kartez and Wenger 25 Subbarao et alReference Subbarao, Lyznicki and Hsu 4 published a consensus-based educational framework and competency set that shows a hierarchical learning framework of competency sets in disaster medicine and public health. Such a pyramid can demonstrate the path of development from core competency to highly specialized competency for each specific working group. The knowledge obtained should then be evaluated and awarded by certification.
On the basis of the above-mentioned factors and different working levels in DM, the overall scheme for the training should consist of 3 specific units. The core of the training initiative will be the unit that presents a realistic situation “scenario-based training.” Consequently, unit one will deal with the theory of intercultural cooperation, based on the participant’s level of activity (gold, silver, and bronze) and professional background. It opens the perspectives of how to work and behave in an international team. The second unit will raise the intercultural awareness and is designed for trainees to share and understand professional skills in the intercultural context specific to their professional environment. That is, for the proficiency of a European disaster response, it is not necessary for all professionals to act in the same manner but to have an understanding of the different approaches and variations to reach the common goal. In the third unit, the inter-agency approach is on focus. Dealing with interagency intercultural competencies, the unit offers a practical training specifically geared to optimize the proficiency of a heterogeneous workforce. The main competencies in focus are communication and flexibility. National standards in DM should be addressed and compared on this level. Finally, the last unit presents a scenario-based exercise (holistic training) that combines all aspects of the other units within a scenario-based simulation. The scenario should engage all agencies. The needed knowledge and competencies for trainees to overcome their tasks should be clearly defined in the setup of the scenario. Different issues and problems can be injected during scenario play to keep all groups involved in the same problem or an individual group with an issue specific to them. Evaluation tools should be available to monitor the process as a whole and for each individual.
Evaluation Methods
Literature review shows that different aspects of the evaluation process have led to different practical approaches, addressed to different aspects of training, and the more these aspects are reflected by the evaluation system, the better the performance of the evaluation.Reference Ahearn and Nalley 83 - Reference Nickols 93 Thus, evaluation represents an essential component of any training initiative owing to both its retrospective values (assessing the relevance, efficacy, and impact of learning upon trainees), and prospective utility. A complete picture of the training process needs a complex assessment from the needs perspective (pre-course assessment), duration (pre- and post-course assessment, as well as daily assessment), participants (evaluation sheets are to be completed by the trainers, the trainees, and the observers), and quality of learning (reflecting the efficacy, impact, progress in learning). In a competency-based training course, many methods can be used to evaluate learner´s competence.Reference Wakefield, Furber and Boqqis 81 , Reference Kirkpatrick 85 - Reference Nickols 93 However, the chosen methods should directly assess the competencies or learning outcomes and should also be examined to prove their value. Thus, a combination of different methods may be used to provide simplicity and flexibility in assessment of the competency that is being assessed. The following methods are some that may be used:
-
∙ Questioning. Oral/written questions include project/assignments, problem/case studies, and written tests. Methods can also include an interview, group assessment, written short answer, extended essay, or multiple choice.
-
∙ Simulation. Is an observation of a product or process. It can be used in case studies, simulated exercises, and simulated workplaces.
-
∙ Skill demonstration. Is a work sample or skill sample and can be used in a practical project, for structured problems and tasks using checklists, rating scores, research task, or assignments.
-
∙ Direct observation. Tests a product or process by using checklists, rating scores, research task, log books, skills book, or work experience and can be used in an interaction analysis, peer assessment, group assessment.
-
∙ Evidence of prior learning/pre-test. Examination of evidence. Used as portfolios, log books, qualifications, referees, reports.
It is important to ensure that the assessment instruments collect evidence that is representative, authentic, and sufficient to allow competence to be inferred. There are different approaches to training evaluation. The goal-based training evaluation method, proposed by Kirkpatrick in a series of articles in 1959, has been identified as the most used model in the literature. President of the American Society for Training and Development (ASTD) in 1975, Kirkpatrick subsequently included these articles in his book Evaluating Training Programs.Reference Kirkpatrick 85 - Reference Kirkpatrick 88 One major problem with the Kirkpatrick model is the fact that it presents vertical evaluation of a training course (trainees) and should thus be integrated with another approach that takes horizontal evaluations into consideration (everything else around trainees). The Nickols (Fred Nickols developed the US Navy´s first instructional systems development training course) approach (different points, different moments) as a horizontal (transversal) axisReference Nickols 93 and the Kirkpatrick levels as a vertical axis together include all the coordinates that need to be evaluated by different types of questions that address all aspects of a training course, such as the following: (1) pre-course evaluation sheets, which are designed to rate the candidates’ knowledge and skills before entry or to identify the right level for the participants, which will enhance the motivation and increase the willingness to work in a team; (2) post-course assessment, which is designed on the criteria of who, what, and at what level to evaluate. Thus, the evaluation sheets are designed for the following situations: (a) daily and module assessment, in which the trainees evaluate the course and the faculty on the basis of Kirkpatrick’s level; (b) observer assessment, in which the observer evaluates the performance of the trainees after each module and at the end of the course; and (c) individual trainee assessment and group activity assessment, in which the trainers evaluate the trainees at the end of the course or at the end of each module through direct questions referring to the curriculum (objective assessment) or in which the trainees are self-evaluating (subjective assessment), usually compared to the pre-course assessment. This comparison evaluation is used to assess the progress of the trainees and both the objective (the efficacy of the training) and the subjective components. The subjective component addresses improving the self-assessment system for the future, enlarging the approach of DM, providing that the level chosen for the trainee is correct, and finally raising the motivation of the trainees to recommend the course to other people involved in DM (ie, positive impact upon dissemination).
DISCUSSION AND CONCLUSIONS
Lessons learned from past events underscore the need for improved management of all parts of a disaster, especially across the command-control-communication triad.Reference Ingrassia, Foletti and Djalali 8 , Reference Bagaria, Heggie, Abrahams and Murray 31 - Reference Schultz, Koenig, Whiteside and Murray 38 , Reference Djalali, Ingrassia and Della Corte 94 Deficiencies identified in this triad were evident in the EU´s recent internatiomal engagements.Reference Brattberg and Rhinard 1 These deficiencies may be due to multiple factors such as the lack of a united European command and control system; a united training initiative that permits engagement of both national and international policies, rules, and leadership; standardized didactic concept and teaching methods; and an intercultural, competency-based, and multidisciplinary training course that trains all level of activities in different agencies by focusing on problem-based learning.
There is an urgent need for a new course in DM with a modular design to cover all phases of the disaster cycle, to bring different agencies into a common scenario play and to let each organization work within its vertical command based on its traditional or national organizational form while enabling horizontal coordination and communication. The outcome of such a course should be measurable and operational on several levels (eg, mortality and morbidity for health care, economic loss for administrators, societal disruption for government agencies, etc) to help each agency improve the competency of its participants. However, this approach is possible only if some common issues are standardized across organizations. This will facilitate concurrent training of operational, tactical, and strategic levels across the tiers of coordination, communication, and cooperation. From this foundation, responders will be able to successfully address the uncertainties and complicating factors inherent in a disaster situation. This will also allow participants to learn new and valuable lessons by performing various activities based on defined learning objectives to acquire desired competencies.
The goal of a European DM course should be to provide and enhance the European intercultural and inter-agency performance within the DM cycle and to develop a standardized training that will progress beyond the state of the art. This approach is complex and can be described as a complete educational process, because it (1) respects the globally accepted principles of education, (2) is based on needs and experience, (3) has a motivated target group, (4) has a clear outcome, (5) generates and can be improved by feedback, and (6) finally, produces considerable changes in and significantly improves DM.
Acknowledgments
The research leading to these results received funding from the European Community's Seventh Framework Program (FP7/2007-2013) under grant agreement n° 285036.
