Assessments

The basis for every intervention, no matter in which temporal phase of a disaster, consists of assessments. Levels of function(s) and the effects of an intervention are determined from assessments. An assessment is the product obtained from assessing;Reference Daily, Birnbaum, O’Rourke and Loretti ⁸ the collection of relevant information that may be relied on for making decisions.Reference Thompson ⁹ Assessments gather facts. Assessments are processes by which data are gathered and converted into information that reflects one or more elements of the current functional status (LOF) of the Societal System of the affected community. Conducting assessments requires the use of indicators of function, which must be determined before data collection begins and must be used consistently throughout subsequent evaluation(s).

Indicators of Function, Impact, and Benefits

Assessments of functions require identifying and using indicators that have good construct validity for reflecting the LOF of the Societal System in the affected population.Reference Pickett ¹⁰ An essential element of all assessments is the selection of common indicators so that all of the data collected will have identical content formats, and thus, will facilitate comparisons. The selection or development of appropriate indicators of LOFs for specific components of each of the Societal Systems is essential for the conduct of such assessments. Most Societal Systems have developed specific markers, or indicators of function, to which they adhere in day-to-day operations as well as in crisis situations.

Selecting appropriate indicators for each Societal System is a complicated process and ideally should involve multiple stakeholders and be undertaken as part of capacity building. Ultimately, a standardized set of indicators of function should evolve for each Societal System. Indicators of function are used to define the functional damage and to determine the impact/benefit of interventions. Indicators are characterized, in part, by their respective specificity and sensitivity.Reference Thompson ^{5 (p216)}

Indicators of function may be quantitative, qualitative, or a combination of both; some qualitative indicators may be scaled into semi-quantitative measures.Reference Birnbaum, O’Rourke and Daily ^{11 (pp118-122)} All indicators used must be understood, practical to assess and collect, and be reproducible.

During the last decade, many organizations have put forth sets of indicators for the Medical Care and Public Health Systems. Prior to 2002, protocols for conducting epidemiological assessments relative to disasters were provided by at least nine organizations, including: the World Health Organization (WHO; Geneva, Switzerland); United Nations (UN) High Commissioner for Refugees (UNHCR; Geneva, Switzerland); UN Agency for Refugees; UN Children’s Fund (UNICEF; New York USA); International Federation of the Red Cross (IFRC; Geneva, Switzerland); the Sphere Project (Geneva, Switzerland); Médecins sans Frontières (MSF; Geneva, Switzerland); Epicenter: US Office of Foreign Disaster Assistance (US-OFDA; Washington, DC USA); and the US Centers for Disease Control and Prevention (CDC; Atlanta, Georgia USA). ¹² In 2002, Bradt and Drummond synthesized the assessment protocols of these nine organizations into one simplified instrument to be used for the Rapid Epidemiological Assessment of Health in Displaced Populations. ¹² They proposed this instrument as the beginning of a standardized, minimum, essential data set with incorporated indicators that could be used for initial assessments as well as for monitoring progress. The indicators derived from their synthesis were grouped according to: (1) population; (2) security; (3) site management; (4) water; (5) sanitation; (6) food; (7) non-food; (8) shelter; and (9) medical. These groups correspond to some of the Societal Systems of the Societal Framework.Reference Pickett ¹⁰ Unfortunately, to date, there are no publications regarding the use of this exact set of indicators.

Since the earthquake and tsunami that devastated many areas in South East Asia in 2004, at least five additional attempts have been made to identify a universal set of health indicators for use in assessing/describing/evaluating the health aspects of disasters. These include, but are not limited to, the: (1) Initial Rapid Assessment Tool (IRA);Reference Bradt and Drummond ¹³ (2) the Tsunami Recovery Impact and Monitoring System (TRIAMS); ¹⁴ (3) the Health Resources Availability Mapping System (HeRAMS); ^{15 , 16} (4) the Multi-Cluster/Sector Initial Rapid Assessment (MIRA);Reference Zagaria ¹⁷ and (5) the Global Health Cluster Guide. ¹⁸ The Academy of Emergency Management and Disaster Medicine (EMDM; European Masters in Disaster Medicine) has developed a Template for Evaluation and Reporting of the Medical Aspects of Sudden-Onset Events. ¹⁹ The Foreign Medical Teams Working Group of the Global Health Cluster is developing a set of indicators to be used in the evaluation of the interventions provided by foreign surgical teams. ²⁰ In addition, the Sphere Project has developed a set of indicators in support of its standards. ²¹

In summary, there is an abundance of indicators and tools that may be used to assess the LOFs of health related to research/evaluations of disasters and emergencies (Medical Care and Public Health). However, most of these tools (excluding those generated by the EMDM) have been designed to meet operational issues, and it is not known at this time which of them will be useful in research. Regardless of the indicators selected, efforts should be made to assure, to the extent possible, that the indicators selected are not duplicative of other indicators. Those indicators that involve more than one Societal System (dependence) should be discussed and agreed upon between the Systems involved. Information on these indicators should not be collected by each of the stakeholders or Societal Systems. For the most part, these assessment tools are directed towards rapid health assessments and are a mix of assessments of structural damage, functional status, and process. They also are used to collect information relative to the public health aspects and the functional status of many of the other Societal Systems that bear directly or indirectly upon public health. However, these tools either have not been tested adequately and/or have not been accepted universally. Parts of existing assessment tools could be abstracted and amalgamated into a generic, universally endorsed tool. This not only would provide structure and promote appropriate and rapid responses, but also would facilitate future research/evaluations. All generic components in assessments, including endorsed indicators of function, should be integral parts of the pre-event inventory of a community.

Conducting Assessments

All assessments are best conducted by well-trained and experienced personnel using standardized data collection forms and techniques. Unfortunately, such personnel are difficult to identify and, for international disaster management, not universally credentialed. Currently, tools are being used and validated for use, accuracy, construct validity, and efficiency. Ideally, personnel who are expert in at least one of the Societal Systems should be charged with the responsibility for the collection of those data relative to their respective area(s) of expertise. A competent team must be multi-disciplinary; problems may require members knowledgeable about nutrition, logistics, environmental health, and/or engineering as well as experts from other Societal Systems required in providing health services (Medical Care and Public Health Systems). The composition of the teams may vary according to the type of hazard and characteristics of the event involved, the character of its onset, the scope of the structural and functional damages, the Temporal Phase of the disaster, and the setting in which the disaster has occurred (culture, religion, geography/topography climate, and so on). As a minimum, at least the leader of any assessment team should be experienced in conducting and processing assessments in disaster settings. The WHO has listed more specific demands that ideally should apply to all of the assessment team members; these include:

1. 1. Familiarity with the region, the culture, and language of the population affected;

2. 2. Knowledge of and experience with the type [nature] of the disaster for which the assessments are being conducted;

3. 3. Personal qualities, such as endurance, motivation, personal health, and the capacity for teamwork, as well as local acceptability for team members recruited abroad; and

4. 4. Analytical [interpretive] skills, particularly the ability to identify trends and patterns. ²²

Although these are stringent demands, the lack of these qualifications may contribute to incorrect interpretations of the observations. The responsibility for these problems reflects back to the selection of the team members by the respective organizations involved in disaster management. However, for most regions, countries, and especially districts, it currently is impossible to find all of these qualities, competencies, and proficiencies in each team member. Nonetheless, all requirements must at least be met by the group as one entity.

The outputs from the assessment process consist of data and information that describe the current LOFs of the Societal System(s) or subsystems of the community affected by an event. The collected data and information are combined with other information to identify the goods and/or services and other resources needed to maintain or reach a functional threshold.Reference Birnbaum, O’Rourke and Daily ^{11 (pp90-92)}

Pre-event Status (LOF)

The pre-event status (LOF) of the Societal System of the affected community establishes the baseline for the study. Since the ultimate goal is to return the LOF of the Societal System to its pre-event LOF, changes in LOF always are judged on variances from the baseline LOF. However, the current LOF also reflects the pre-intervention functional state, and thus, serves as the baseline for a specific intervention. The pre-event and pre-intervention assessment of the LOF for the Societal System (or component of the Societal System) must utilize the same indicators of function used for evaluation of the effects of the intervention.

When data/information of the pre-event LOFs for the Societal System and/or its components are not available, the pre-event status must be estimated given the data/information available. The error in such estimates may be substantial if the pre-event LOFs were over-estimated or under-estimated. Such over- or under-estimates may affect the selection of interventions and their respective assigned priorities. Ideally, the pre-event LOFs of each Societal System should be contained in a current, static database.

Assessment(s) of Current Status

The first task outlined in the DLM and the Relief/Recovery Framework is to define the current status (LOF) of the Societal System(s) and its components being considered. This requires that the community affected be considered in terms of its functional Societal Systems.Reference Pickett ¹⁰ The current LOF must be considered in relation to the essential and critical LOFs of the Societal System (or its components) (Figure VII-5). The priorities assigned to the interventions depend on the urgency of filling the gaps in the LOFs. Levels of function that are below the critical LOF have highest priority, while LOFs that comprise luxury functions are assigned low priorities. The effects (dependencies) of the LOF on other Societal Systems or elements of the Societal System being studied and vice versa constitute a key element in assessment of the current LOFs. Inappropriate assessments of the current status can result in the selection and implementation of unneeded, inappropriate, and even harmful interventions.

Figure VII-5 Levels of Function as a Function of Available Goods, Services, and Other Resources. Below the critical threshold, the crude mortality rate increases. The respective LOFs decline progressively as functional damage increases. As available resources are augmented during recovery, the LOFs return towards their respective pre-event LOFs. Abbreviations: CMR, crude mortality rate; LOF, level of function.

Comparisons in Levels of Functions

Interventions are used to change the current LOF, to mitigate further deterioration in LOFs, and/or fill gaps in essential functions/restore LOF to their respective pre-event state. The changes in LOFs of the Societal System (or its components) are determined from the differences between the current status and the pre-event and pre-intervention LOF baselines. The difference between the current LOF and that which existed prior to the onset of the precipitating event and pre-intervention are used to define the needs (goods, services, and other resources) to return the LOF to pre-event levels (recovery). The current LOF is utilized to determine the current needs—it is this LOF that is used to define what the intended intervention will accomplish. Unless this intervention will be the first, the difference between the current LOF and the LOF determined after the last intervention defines whether the LOF is improving or deteriorating further. Comparisons also must be made between the current LOF and the LOF determined after the completion of the last assessment. It is essential to identify gaps in the LOF of essential services that cannot be met by the local response capacity. Such gaps in LOFs may require outside assistance. By definition, a disaster for the Societal System exists when there are gaps between the essential LOFs and the current LOF that cannot be met by the extraordinary local response capacity of the community affected. Thus, comparing LOFs yields the changes that have occurred in the functions of the Societal System or/and its components. These differences are synthesized into the needs to contain further deterioration in the LOF (relief), fill the existing gaps in the essential functions (relief; Figure VII-6), or to return the LOF to its pre-event status (recovery). Errors in this process can lead to inappropriately defined needs and responses to the errant needs.

Figure VII-6 Production Functions Linking Steps in the Relief/Recovery Framework. Noting that the steps in the Framework require assessment of changes in levels of functions due to functional damage that compromise available resources and that recovery of available goods, services, and other resources must be transformed into functions. Each process is open to evaluations using the Disaster Logic Model.

Identification of Needs

Needs are defined as the differences between the available goods and services and the required goods, services, and other resources.Reference Birnbaum, O’Rourke and Daily ^{11 (p155)} Requirements are those goods and services that must be available to the Societal Systems in order for them to function at least at the level of their respective essential LOFs, ²³ or meet the gaps in LOFs caused by the damage(s) within the Societal System being studied or other Societal Systems upon which it is dependent. Recovery from the disaster has been achieved when the ability to maintain essential functions without outside assistance has been restored, although an emergency still may exist for the Societal System being studied. Needs are comprised of the resources (ie, goods, services, and other resources) required to prevent further deterioration of the functional status of a Societal System or component of the affected System, and of what is required to return its functions or its components to its pre-event level (Figures VII-5 and VII-6). It should be clear that the transformation of LOFs into needs requires expertise and knowledgeable personnel and the infrastructure (physical space; equipment) to perform the transformation, and that the results of processing the available goods and services into functions are critical. The hypothetical diagram in Figures VII-5 illustrates the relationships between available goods and services and those required to mitigate further deterioration in the LOF, fill the gaps in functions, and/or return the LOF to its pre-event state.

Needs do not include those goods and services required to meet the surplus (non-essential functions, luxuries) that may have been present before the onset of the event. If the assessments indicate that the available resources have fallen below the critical threshold (ie, the LOF at which the CMR increases), then a critical need exists.Reference Birnbaum, O’Rourke and Daily ^{11 (p155)} Critical needs require prompt interventions to prevent further increases in the CMR. When an assessment indicates that the levels of available supplies of goods, services, and other resources have fallen below the functional threshold, a functional need for goods and/or services exists.Reference Birnbaum, O’Rourke and Daily ^{11 (pp90-92;155), 23}

The identification of needs is an integrative production process of analysis and synthesis that must combine data and information from several sources: (1) the pre-event LOFs; (2) the current (pre-intervention) LOFs; (3) the LOFs detected by the last assessment; (4) the amount and types of structural damage sustained; (5) the presence of a surge; (6) the demands of the population affected; (7) the culture in which the event occurred; (8) the climate in the affected area; (9) the geography and access to the affected area; and (10) the politics involved. This synthesis requires special expertise to process the data/information on the LOF into the goods and/or services needed. It requires knowledge of the Societal Systems and their dependencies and expertise obtained through education AND experience using the synthesis process during actual disasters and/or during drills and exercises. It is the process that determines the validity of the product that results from the assessment and, as such, the component of the Relief/Recovery Framework that is most vulnerable. Interpretations of the data are only as good as the expertise of those performing the data transformations. This needs identification process is an exercise in the synthesis of many factors, each of which has different weights. Needs identification incorporates numerous assumptions based on the experience and knowledge of the synthesizers. Thus, there is a risk that some of the needs identified may not reflect the actual needs accurately. Needs include the estimation of the costs not only of the process but also include the costs of the intervention compared to the resources required/available.

The outputs of the needs identification process are the needs for which interventions (responses) must be selected or developed. It is not possible to assess needs directly, and it is important to understand the processes involved in the determination of actual needs. It and the above steps constitute part of the Introduction section of a scientific paper. The needs provide the reasons for the intervention.

Strategic Planning

Once the needs have been determined from a synthesis of many assessments, a strategic planning process ensues that is directed at determining ways to meet the defined needs. Strategic planning also consists of a production process in which the identified needs are transformed into a Strategic Plan. A Strategic Plan is formulated that details a method by which a thing is to be done.Reference Birnbaum, O’Rourke and Daily ²⁴ The Strategic Plan, which is the output of the strategic planning process, details the respective overarching goals and some of the objectives that must be accomplished in order to meet the identified needs as well as prioritizing the order of the implementation of interventions selected to meet the needs.

The strategic planning process provides a broad picture of what must be achieved (goals) and in which order, including how to organize a response capable of achieving the overall goal. The Strategic Plan that results from the planning process suggests the types of interventions that must be implemented, the respective goals, and the objectives of the interventions that must be accomplished to meet the identified needs. It also defines the priorities, sequencing, and timeframes over which each of the goals and objectives is to be achieved.

Strategic planning to meet the defined needs is a complex production process that integrates the defined needs with the: (1) existing Disaster Response Plan; (2) interventions that currently are underway or planned; (3) resources available to meet the needs; (4) pre-event functional status compared with the current functional status; and (5) experience and expertise of the planners in managing similar situations. In addition, strategic planning must include consideration of the: (a) site (access; geography, climate); (b) culture of the population affected; (c) safety; (d) views of stakeholders; and (e) existing health care system. The process also must assure that the priority list of the identified needs is reviewed and updated to ensure that the information used for selecting the most appropriate interventions is current. A Strategic Plan always addresses the likely availability of the resources required for its implementation and the consideration of how to access the resources required to develop or to implement the plan. The designated resources may not be available when they are required. Stockpiles of supplies may have been directed to other needs, the pharmaceuticals may have expired, or the warehouses and/or stockpiles may have been damaged by the event. Personnel may have been injured/killed, may not be able to reach the medical facility, or may not report because of other commitments.

Some plans for meeting anticipated needs exist before an event occurs. This is most useful during the early stages of a high-intensity, sudden-onset event when it may not be logistically possible to conduct comprehensive or even rapid assessments in order to identify needs. However, it is not possible to predict exactly which needs will emerge during or following an event. No matter how comprehensive a Disaster Response Plan may be, it must be adjusted to deal with what actually is occurring. The anticipated needs must be reviewed in the light of those that materialize during the actual event.

The resulting Strategic Plan must be discussed and re-evaluated frequently in order to: (1) determine whether it is practical; (2) familiarize responders with the mechanics of the plan; (3) identify the respective roles of possible responders; (4) identify ways in which the planned responses can be improved; (5) identify the supplies, equipment, and/or personnel that must be added to the response capacity; and (6) identify additional arrangements that must be added (ie, contracts/memoranda of understanding (MOUs) with other administrative structures).

The Strategic Plan must be feasible, realistic, and coincide with the goods and/or services and/or other resources that are or can be made available. Plans that seem excellent on paper may never be implemented because they require unavailable resources or are not possible given the environment (ie, culture, weather, or terrain). Thus, each of the factors described above helps to identify whether the plan actually can be implemented and whether its implementation is likely to produce the outcomes for which it has been devised with as few unwanted effects as is possible.

Strategic Plans also must place the defined needs and responses into respective priority lists. Clearly, identified critical needs (ie, those essential to maintain life) must be addressed before functional needs. Following a high-intensity, sudden-onset event, critical needs often can be anticipated based on established science and previous experience. Dependencies between the respective Societal System must be identified and considered. Priorities for recovery activities should begin as early following the onset of the event as is possible. ^{25 , 26}

Similarly, timelines for achieving the goals and the objectives to be attained by interventions must be specified in the Strategic Plan. Specific timelines for each of the benchmarks and milestones of a project are essential elements and must be defined prior to selection of an intervention. Projects cannot be allowed to continue indefinitely. Providers must agree to honor the established timelines. Each of the above factors must be considered in the establishment of timelines.

Costs are an expenditure of resources. ²⁷ The costs associated with the development, testing, and modifications of a Strategic Plan include economic, material, environmental, human, and opportunity costs. Indicators of human and opportunity costs are difficult to quantify, and often the resources consumed by the planning process are not accounted. Substantial amounts of resources are consumed in developing and validating the plan and in the subsequent modifications to the plan.

The strategic planning process is complex and requires expertise for the integration of each of the factors outlined and discussed. The resulting Strategic Plan lays down the overarching goals and major objectives for which an intervention(s) will be selected. The plan must be flexible and can be modified as additional, relevant information becomes available. Strategic planning and the plan that results are fertile areas for study. Strategic Plans may be developed by/for each Societal System. However, the ultimate responsibility for coordinating the Strategic Plan of a Societal System belongs to the Coordination and Control entity. Strategic planning is a function of Coordination and Control.

.

Resources—All interventions provided during the relief and recovery phases of a disaster utilize (consume) the goods and/or services contained within the response capacity, as well as other resources required that must be identified and appropriated to meet the needs. The inventories that are part of the Disaster Response Plan should list all of the available agencies that can provide the required intervention(s), the goods and services they can contribute, who is responsible for them, how they can be mobilized, anticipated mobilization times, and estimated costs.Reference Thompson ²⁸ Often, these agencies have a long history of providing such activities. However, experienced persons may not remain in the organization for extended periods. Ideally, each of these agencies should be accredited (certified) by Coordination and Control as to its ability to provide the interventions it claims it can provide [a registry is needed]. If not part of the government hierarchy, appropriate, completed MOUs or contracts should be included in the Disaster Response Plan.

The resources (costs) required to execute the responses may take the form of personnel and/or equipment and/or supplies and/or infrastructure.Reference Thompson ²⁸ The persons, supplies, and/or equipments that comprise the response capacity may include volunteers to supplement staff and/or the stockpiles of equipment and supplies. Such stockpiles may seem to be an inefficient use of resources until they are needed. In some cases, stockpiled equipment/supplies are rotated to prevent loss due to aging (eg, expiration dates for pharmaceuticals). These materials not only require the expenditure of financial resources but also require transportation and storage facilities. It has been suggested that the “best” supplemental resources should be provided in the form of cash. The cash then can be used to purchase the needed resources locally.Reference Thompson ²⁸

An emergency reserve fund could assure that the bills can be paid and that what is not available immediately can be purchased on the local market or from suppliers from outside of the affected area(s). A good example of the importance of an emergency reserve fund has been demonstrated in South East Asia where, following the 2004 earthquake and tsunami, the Ministries of Health committed their respective countries to contribute to a Regional Emergency Fund that can be used by any Member State of the South East Asia Regional Office of the WHO (WHO-SEARO; New Delhi, India).Reference Cuny ²⁹ This fund played an important role in the management of the floods in Bangladesh and the cyclone that struck Myanmar in 2008. ³⁰ However, some care must be exercised in seeking and acquiring donations in times of crisis. The amount of donations acquired following the South East Asia earthquake and tsunami far exceeded the amount of funds acquired during any other catastrophe. ²⁶ One question is whether such extensive fund-raising for one cause diminishes the amount of donations available for the support of relief and recovery responses for other disasters. As suggested in the standards that resulted from meetings of the stakeholders following the South East Asia earthquake and tsunami, such emergency funds should be included as an endorsed, integral part (line item) of the budget of each level of government, from community to national.Reference Cuny ^{29 , 30}

.

Timelines and Critical Pathways—A responding agency/individual(s) is assigned the responsibility for the intervention by Coordination and Control. Included in the assignment are the objectives of the intervention and a reasonable timeline for completion, including milestones to be reached along the way. This timeline and the milestones to be achieved constitute a critical pathway towards reaching the objectives for which the intervention was selected. ³¹ Critical pathways have been defined in the provision of patient care but also apply here. It is the necessary path or sequence from start to finish, determining the time needed for completion. ³¹ The timelines must be realistic and agreed upon by all of the stakeholders.

Options for interventions not considered previously may become important when the currently available interventions are not likely to accomplish the objectives and contribute to meeting the defined goals. In such instances, and if there is sufficient time, Coordination and Control may issue a Request for Proposal. A Request for Proposal states the overarching goal, as well as the objectives outlined in the Strategic Plan, and requests persons/organizations to submit a proposal on how they will contribute to meeting the identified objectives and the goal.

Agencies and/or individuals that are not included in the Disaster Response Plan, or are not part of an inventory, may volunteer their services and/or goods. Some agencies from the areas outside of the affected area may respond without invitation. Since these intrusions may occur without warning and/or invitations, their activities generally cannot be managed by Coordination and Control. Although the motivation for such responses usually is to benefit the stricken, unfortunately, such responses may confound orderly and coordinated disaster responses and may prove to be more detrimental than beneficial. This circumstance is being studied by the Foreign Medical Team Working Group of the Global Health Cluster and the WHO. ³²

Achieving the strategic goal(s) may require the implementation of several interventions to achieve the stated goal(s), as each selected interventions only may contribute partially to reaching the overarching goal(s). The objectives of several interventions must be combined with the hope that the goal(s) will be achieved. These processes and selection of interventions may be based on assumptions by experts (part of Coordination and Control) who synthesize the many assessments and the information available into needs using standard operating procedures, policies, and their experience, education, and training. Strategic planning may be performed by the Societal System management personnel (or elements of the Societal System) and submitted for approval to the Coordination and Control entity that carries the overall responsibility for the Strategic Plan.

.

Progress Reports—The submission of progress reports by the organizations responsible for providing the intervention is an essential part of the project and must be included in the agreements reached by Coordination and Control with the responsible agency/organization or individual. The structure and forms to be used for the reports should be standardized for all interventions with common, agreed upon indicators and benchmarks. These reports must be public documents and readily accessible. All deviations from the mandated critical pathway should be evaluated by Coordination and Control. Variances from agreed timelines and benchmarks (critical pathway) must be considered in terms of changing conditions and resource availability. Progress reports have substantial value in the evaluation of an intervention/project.

Consideration of the factors discussed will help to assure that the intervention(s) selected most likely will contribute to meeting the goal(s) in a most cost-effective manner, produce the greatest benefit to the affected population, produce the fewest negative effects, and be practical to deliver in the setting.

Operational Planning

Once an intervention has been selected for implementation, the intervention becomes a project. A project is an individual or collaborative enterprise that is planned to achieve a particular aim. ³³ Projects are composed of a task or tasks to be accomplished within a fixed period of time and within defined costs and other limitations. ³⁴ Once a project has been assigned to an agency or individual for implementation, an Operational Plan must be formulated to ensure its smooth implementation, to establish short-, medium-, and long-term targets (milestones) and timetables, to designate appropriate logistical arrangements, and to assign responsibilities and monitoring mechanisms. Operational Plans always address a specific intervention. The Operational Plan takes into consideration the inter-relationships between the Societal Systems of the affected community and designates priorities to maximize each function. As with assessments, operational planning for responses and the implementation are the responsibilities of the organization assigned the intervention and must be approved by the Coordination and Control System. The process should be open for evaluation.

Every intervention must follow an Operational Plan that delineates the actions required to meet an objective that is tasked to contribute to achieving the overarching goal. Operational Plans dictate the critical pathways that will be followed during and following the implementation of the intervention. Operational Plans include: (1) an introduction and brief situation report; (2) an overview of the task(s), objectives, and the overarching goal to be addressed; (3) methods that will be employed; (4) planning considerations; (5) the resources that will be required, including personnel, equipment, infrastructure, and supplies; (6) timelines, indicators, benchmarks, and milestones including startup time, end-point, and estimates of the time required for completion; (7) administrative structure to be employed; (8) operating budget; (9) strategy for acquiring the funding required; (10) roles, responsibilities, and required competencies of the personnel, including position descriptions (Terms of Reference); (Reference Birnbaum, O’Rourke and Daily11) mechanisms for monitoring the progress, including the indicators that will be used; (12) safety issues, including health of the personnel, required immunizations, and accommodations; (13) methods to be used for self-sufficiency; and (14) requirements for reporting, evaluating, and accounting. Without a detailed Operational Plan, interventions not only are likely to fail, but may impact negatively upon other projects underway in the area.

The Operational Plan may be generated at the time a project is approved or may be part of the existing Disaster Response Plan. The Operational Plan used for previous projects that have been successful should be retained and activated, as appropriate. There are many generic items in an Operational Plan that can be used in successive plans. Existing Operational Plans may not match the new situation. These plans must be tempered with the new situation, including modifications of the needs, the location, geography, climate, culture, language, and political circumstances (external factors). Ideally, the plans should be constructed with maximum flexibility for easy adaptation when circumstances change. Existing Operational Plans also should be updated and modified in accordance with the evolving science and the definitions of best practices. The use of operational-planning templates can facilitate the development of Operational Plans. The military has a large repository of such templates and planning guides.

Each Operational Plan must include the steps necessary to implement the selected intervention(s). In the previous example, if a helicopter was to be used to supply water, the source of the water had to be identified, arrangements had to be made for its acquisition and for its transportation to the helicopter base, for the loading of the helicopters, for transportation using the helicopters, for identification of landing sites, and for the establishment of centers for the distribution of the supplemental water, including security for the personnel staffing the distribution centers.

Operational Plans outline the specific processes (critical pathway) used in the implementation of any project and should include methods of evaluation of the project to determine which components of the process went well, which could have been done better, which parts did not work, and which parts were not beneficial. Critical points of failure in the processes used in planning should be identified and the plan adjusted so that the points of failure are corrected, and thus, enhance the utility of the plan the next time it is considered for implementation.

Description of the Strategic Plan, the intervention selected, and the Operational Plan are included in the Methods Section of a scientific paper.

Implementation

The next step in the Relief/Recovery Framework is the implementation of the intervention selected. This is the principal production process of the project. The output from the implementation process is provision of the intervention. Implementation is the carrying out, execution, or practice of a plan, a method, or any design for doing something. As such, implementation is the action that must follow any preliminary thinking in order for something to actually happen. ³⁵ Implementation consists of the process preceding initiation of the intervention through its completion. The specific mechanisms involved in implementing the selected intervention will differ by area and culture. Entrusting a provider with the project for the implementation of the intervention carries with it a mandate and the authority, as well as the responsibility for its implementation, completion, and modifications, as necessary.

The implementation of a project is a production (transformation) process in which an organization(s) or individual carries out numerous activities to transform the Operational Plan into the intervention. The production process used for implementing an intervention is complex and, for analysis purposes, has been deconstructed into its components. The tasks required are extensive and include (not listed in rank order of importance or by priority, and are not limited to): (1) reassess current status and verify needs; (2) activate the Operational Plan; (3) set-up and operate the administrative structure defined in the Operational Plan; (4) identify, acquire, and organize resources (including human resources); (5) assign roles and responsibilities; (6) educate and train personnel; (7) brief the staff; (8) prepare/ready resources for transport to the area-of-need or into storage; (9) assure that the project is self-sufficient; (10) arrange for personal necessities; (11) ensure the safety of personnel and the security of equipment and supplies; (12) insure personnel; (13) coordinate with other projects/actors; (14) coordinate with other Societal Systems; (15) communicate with community leaders; (16) provide standardized progress reports; (17) deploy personnel, equipment, and supplies; (18) initiate the intervention; (19) report the start of the intervention; (20) monitor and report the progress of the project; (21) complete the project; and (22) complete and submit a formal report. The above list may serve as a checklist for use in evaluation and quality control.

The Operational Plan for the project is activated by the Project Director and the respective administration. Each of the elements in the Operational Plan must be followed. Deviations/variants from the Operational Plan must be negotiated with Coordination and Control. Some components of a project are sequential and some are concurrent. If there is no endorsed Operational Plan, one must be developed and approved by the tasking body (Coordination and Control).

Changes that may have occurred in the status of the affected Societal System of the community or the environment since the assessments that were used to identify the need(s) may have profound effects upon the implementation of the project. Given that the implementation of interventions always carries a time delay, and that disasters are dynamic, the current status should be re-assessed before implementation of any intervention. Changes in needs may dictate changes in the project. When such changes are detected, Coordination and Control must be informed as the goal and objectives of the project may require modification(s) in accordance with the changes. Perhaps the goal may have been accomplished by another project, the ability of the project to solve the problem may be questioned, security in the area may have changed, the weather may dictate a change in action, or secondary events have occurred or may occur. A project may be recalled or modified at any time by Coordination and Control.

Personnel who are employed permanently or associated with the provider agency, or who already have been recruited, must be assigned to a specific, appropriate job. Position descriptions and qualifications (job/position descriptions; terms of reference) should be included in the Operational Plan. The respective role and responsibilities of each person involved in the project must be defined and be consistent with those outlined in the Operational Plan. The assignments must include the lines of authority and the allocation of available resources.

Once individual roles and responsibilities have been assigned, the selected staff may require additional education and training to augment their respective competencies for their assigned role and responsibilities in accordance with those mandated in the Operational Plan (especially the knowledge and skills that are mission-specific). For accountability purposes, participation in such education and training programs and attainment of the essential competencies must be documented, and this information should be added to their respective personnel files. Proof of competency for their respective roles should be required for all personnel. Professionalization of disaster workers should yield a more competent workforce.

All personnel involved must be brought up-to-date about the situation and be aware of the: (a) mission; (b) organization of the project; (c) methods (projects, instruments, and collaborating partners) that will be employed; (d) culture, religions, customs, and language of the affected population; (e) geography of the affected area into which they will be deployed; (f) public health specifics; (g) timelines; (h) monitoring systems; (i) other projects underway in the area; and (j) required reporting mechanisms.

The administrative structure to be used is defined in the Operational Plan and includes the people (defined by competencies) and the functions that are part of the required structure to make the project happen. In order to implement the intervention, the administrative staff must interact not only with persons within their own organization, but also with a myriad of outside administrative structures. The administration is responsible for contracts and MOUs, payroll, purchase and/or rental of equipment, acquisition of supplies, and acquisition of the funds required. Leadership and management of personnel not only include recruitment, but also arrangements for staff rotation as well as replacements for anticipated and unanticipated staff turnover. In addition, the administrative arm of the organization responsible for the project must continuously monitor the progress, effects, effectiveness, and cost encumbered by the project.

Financial resources are essential to mount and sustain any intervention. The resources available for the project must be inventoried relative to their amount and appropriateness for the tasks at hand. Resources not available to accomplish the mission must be acquired.

Using the previous example of supplying needed water, the location and ownership of the supplemental water supplies must be identified, the supplements purchased and accessed, transported, loaded, and flown into the area in which the water is needed. The landing sites must be secured, and the distribution centers for the supplemental water and guidelines/protocols for the distribution must be established, staffed, and secured as well as resources (costs) encumbered by the project.

The end of a project is determined by: (1) the achievement of the objective(s) for which it was selected; (2) the exhaustion of available resources required for its continuation; (3) the achievement of the overarching goal(s); (4) exceeding the timeline (variances from the critical pathway); and/or (5) a termination order from Coordination and Control (eg, when an intervention has been recognized as ineffective in either reaching the goal or when other negative effects of the intervention outweigh the benefits to the affected community). The disaster ends for any given Societal System when its essential functions can be maintained and sustained without outside assistance.

In summary, for an Operational Plan to be implemented, a long list of detailed activities and issues must be addressed. Many agencies that are likely to be tasked with a project already have covered these elements in their Standard Operating Procedures and only need to activate them.

Monitoring and Assessments of Project

No intervention is complete without identifying all of the effects resulting from implementation of the intervention. Such effects include the expected impacts to the community associated with the change in its status. Assessments of the outputs use the same selected indicators of function to define the effects and outcome, impacts, and costs of the transformation process (intervention). The impacts and costs not only are documented by use of these indicators of function, but also by separate indicators (of impact and costs). The current status is assessed and compared to the previous assessment(s). The effects produced by the intervention are compared with the objectives of the intervention to determine the outcome(s) (effects related to objectives). Effects not related to the objectives also are accounted; some outputs from the transformation process (intervention) may not have been expected and may require modification of the selected indicators. All the effects of the intervention are related to the overarching goal (short-, mid-, and long-terms impacts) for which the intervention was selected. What was the impact(s) of the intervention on the stakeholders? The implications (discussion) of the effects are not included—only the assessed results and costs (resources consumed) of the intervention are included. There are no evaluations (definition of worth) of the interventions—only the findings.

Thus, the assessments provide only the data/information required to gain a clear picture of what happened—not why it happened. What was(were) the impact(s) that resulted from the intervention?

Assessments of New Level of Function

Every intervention is implemented to change or preserve the current LOF of the Societal System and/or its components from the LOF that existed prior to the intervention. Did it achieve its objective? Assessments of the new LOF are conducted using the same indicators of function that were used to define the pre-intervention LOF and the pre-event status. The effectiveness of the intervention is whether the effect(s) of the intervention (project) met its targeted objectives.Reference Thompson ⁵ What changes in the LOF were achieved by the intervention? The new LOF then becomes the current LOF.

Assessments of Process(es)

The process of changing the current functional state to a new functional state, maintaining the current LOF (preventing further deterioration in LOF), or filling the gaps in essential functions can be considered a single production process. The results of the process are judged by the effects and outcome of the intervention.

As noted, the whole process from the assessment of the pre-intervention LOF to the new LOF consists of numerous sub-processes that have as their respective objective, successfully attaining the next step in the Relief/Recovery Framework (Figure VII-2). Thus, each of the sub-processes can be analyzed as a production function. The current functional status is the step in the Relief/Recovery Framework preceding the transformation process, while the output (effects) is the next step in the Relief/Recovery Framework (Figure VII-3).

It is important to identify if this outcome was achieved, as well as the other effects produced by each of the sub-processes, in order to determine the success or failure of the sub-process. In addition, the barriers to each of the sub-processes can be identified. Analyses of these processes are essential in determining how they might be modified for the next time. A simple check list may help to inventory these processes. Figure VII-2 may serve as the basis for such a process check list. Indicators for the evaluation of these processes must be developed.

Reporting the results of the assessments of an intervention is akin to describing the Results Section of a scientific paper.

Synthesis

Synthesis is the combining of separate elements [or substances] to form a coherent whole.Reference Pickett ⁴³ Data collected by assessments have little meaning by themselves—the findings must be synthesized into useful information. This element of the Relief/Recovery Framework provides the interpretation of the results of the intervention. Synthesis is based on comparing the findings with other evidence available for the same area. The outputs from this synthesis should contribute to establishing the value (benefit(s))Reference Pickett ⁴⁵ of a specific response (intervention) to the affected community and to integrate the findings with the existing evidence.

The synthesis process converts impacts into benefits to the affected community. A benefit is something that promotes or enhances well-being; an advantage. ⁴⁴ Satisfying the need is the goal of all interventions and achieving the goal, at least theoretically, should be associated with a benefit to the community. Thus, the value of any output from the production process not only must be judged by whether it met all or part of the objectives for which it was selected, but whether it produced a benefit (positive impact) to the affected community. Even though an intervention met its objective(s), it may not have produced a benefit to the community. Benefits often are difficult to quantify; indicators of benefit generally have been qualitative. Nonetheless, indicators must be selected with care and validated. The value (impact) of the intervention on the population of a specific effect could be either positive (benefit) or negative for the community. Some interventions may not change the current LOF, but may hold function constant and keep it from deteriorating further (relief). Such actions also are benefits.

The impacts of the effects of an intervention must be compared with the strategic goal(s) for which the intervention was selected; achieving the goal should be associated with a benefit to the community. The impacts of the intervention may be short-, medium-, and/or long-term. As mentioned, even though an intervention has met its objective(s), it may not have produced a benefit to the community for which it was provided. The “value” of any effects of an intervention (project) must be judged not only by whether it met all or part of its objectives, but whether it produced a benefit (positive impact) to the affected community.Reference Zint ⁴⁰ If the effect(s) did not result in a benefit to the community, it is important to determine where in the processes used was the critical point(s) of failure of the intervention, and how the process can be modified the next time such a goal or objective is established. What was the reason(s) that the intervention did or did not contribute to achieving the goal? Was the goal unrealistic? Were/Was the data/information acquired appropriate for the analysis? Did the indicators used reflect what was needed for the analysis? Was the selection of recipients flawed? What do the findings about this intervention contribute to the science of Disaster Health and management? How can the process be improved, or should the intervention be abandoned, and why? What elements of the project fostered its success, and which of the processes could be improved? What resources will be required to sustain the new level of resilience? Using the Relief/Recovery Framework should assist in answering these questions.

Additionally, some of the data relative to the effects, outcome, and impacts can be synthesized into derived variables such as effectiveness in contributing to achieving the stated objective and goal, the efficacy of the intervention for meeting the defined needs or similar needs, or efficiencies (the time and type and amount of resources consumed); cost:effectiveness, cost:benefit, and needs:effectivenessReference Thompson ⁵ can be computed. The costs of the benefits accrued are essential components of the evaluation of any project (intervention) and not only should be estimated for the resources consumed in achieving the objective(s), but also should be estimated for the intervention’s other effects. Much work remains to be done in identifying the costs and benefits of actions/interventions in Disaster Health.

The success and usefulness of the intervention can be identified and the project defined as a “success,” “partial success,” or “failure” in the setting in which it was implemented. Depending on this judgment, it may be recommended that the intervention be duplicated in the responses to the next event or to an event in another setting.

As noted, each step in the Relief/Recovery Framework is achieved by a process that moves it from the preceding step. Therefore, each of these sub-processes can be evaluated using the DLM. If the project was a success, it is helpful to understand which elements in the process used to transform the pre-intervention status into effects, outcome(s), and impact(s) were critical in attaining the desired goal outlined in the Strategic Plan. If the project was not a success, was the failure related to the intervention, or was it related to one or more sub-processes involved in the intervention? Were the objectives of a selected intervention appropriately aligned with the needs? Was a component of the transformation process inefficient and inadequately performed, thereby resulting in wasted resources? Process evaluations are performance evaluations.

Evidence obtained from the evaluation of one intervention is strengthened by similar findings from other evaluations. The synthesis process converts the effects and outcomes and the short-, medium-, and long-term impacts, and the processes used for the intervention into some level of evidence that can be incorporated into best practices and hence, competencies. The results of structured quality studies produce evidence which then can be synthesized into standards of practice. These standards become the basis for defining best practices that are likely to produce a positive change in the functions of a Societal System in a given setting.

The findings from individual evaluations of relief or recovery interventions do not have high internal validity (prove cause and effect). However, each study contributes to the body of evidence; the strength of the evidence comes from repeated documentation of similar findings in the same or similar settings. Thus, the findings from any evaluation of a disaster intervention must be compared with the findings from other similar studies in similar settings. This comparison is an essential part of the Discussion section of the evaluation of any intervention. This strengthens the evidence and facilitates the development of best practices. Lastly, the synthesis of the findings, as well as the recommendations for future studies, form the basis of the Discussion section of a scientific paper.

Evaluations are a form of quality improvement and provide the evidence required for the development/implementation of interventions to enhance resilience. Evaluations should not be used as a basis for withdrawal of support for current and future projects. Donors should be aware that evaluations are for their benefit as well as for the providers and the recipients of the interventions. Evaluations produce good returns for all involved. Thus, evaluations of disaster or crisis interventions should be studied in much greater detail than generally is done today.

The ultimate goals of conducting disaster research are to determine how to decrease the morbidity and mortality associated with disasters and to enhance the recovery of the affected community. The systematic study of interventions provided during a disaster are necessary to build the body of evidence needed to inform practices and achieve the stated goals. Information on the number of procedures performed or the number of patients seen does not provide the necessary components for synthesis and evaluation of the intervention. This requires the use of a framework that provides common terminology and processes that allow comparisons to be made. Research of the effects of disaster interventions (projects), including the processes utilized, based on the structure in the Relief/Recovery Framework, is essential to the building of the science of Disaster Health. Conducting research prospectively during a time of disaster is fraught with numerous problems. However, the retrospective and methodical study and evaluation of all disaster-related interventions is required to build the evidence and the science.

Synthesis of Evidence into Standards and Best Practices

Standards are not based on findings from a single study. A Systematic Review brings the same level of rigor to reviewing research evidence as should be used in producing that research.Reference Hemingway and Brereton ⁴⁵ A Systematic Review is a summary of available, carefully designed studies that provides a high level of evidence on the effectiveness of interventions. ⁴⁶ Systematic Reviews are based on a peer-reviewed protocol and pre-determined questions. They seek to: (1) identify all relevant published and unpublished evidence; (2) select studies/reports for inclusion; (3) assess the quality of each study or report; (4) synthesize the findings without bias; and (5) interpret the findings and present a balanced and impartial summary.Reference Hemingway and Brereton ⁴⁵ Systematic Reviews of Disaster Health interventional studies are necessary in order to transform evidence from specific studies into standards and best practices to be applied in disaster risk-reduction. Priorities for such reviews must be established and their use must be coordinated. This process will be facilitated by the using the structure provided by the Frameworks and by a universal terminology to be used in all reports on the health aspects of disasters. Such structure allows the development and implementation of repositories of information from studies of the epidemiology of disasters and for health-related interventions provided before, during, and following disasters.