Introduction
In October 2010, nine months after a catastrophic 7.0 magnitude earthquake centered outside Port au Prince, Haiti, the Haitian Ministry of Public Health and Population (MSPP; Port au Prince, Haiti) reported a cholera epidemic caused by Vibrio cholerae O1, serotype Ogawa, biotype E1 Tor. Piarroux et al concluded that the outbreak resulted from contamination of the Artibonite River by a United National Stabilization Mission in Haiti (MINUSTAH) camp with inadequate sanitation.Reference Piarroux, Barrais and Faucher 1 Prior to the outbreak, cholera never had been documented in Haiti, thus meeting the World Health Organization (WHO; Geneva, Switzerland) definition of an epidemic. 2 As of May 2016, 780,140 cases and 9,317 deaths (1.2% case fatality rate) have been reported by MSPP. 3 Factors such as limited access to potable water and health care services, inadequate facilities for excreta disposal, potential increased susceptibility due to the effects of the earthquake, and poor hygiene practices had created an environment that facilitated the rapid transmission and the high fatality rate.Reference Shuller and Levey 4
Following the outbreak, the local and international community launched an extensive response that followed the WHO’s Cholera Outbreak Protocol that consisted of surveillance, case detection, treatment with rehydration, establishment of cholera treatment centers, community health education, and water and sanitation efforts. 5 Actions by the MSPP, the National Directorate for Water Supply and Sanitation (DINEPA; Port au Prince, Haiti), and the international community included: (1) a large-scale public information and awareness campaign; (2) strengthened epidemiological surveillance; (3) establishment of cholera treatment centers; (4) development of clinical management protocols; (5) repair of water systems within selected health centers; (6) chlorination of water systems and water delivery to camps for internally displaced persons (IDPs); (7) distribution of household water treatment products; (8) instillation of potable water stations and water-quality laboratories; and (9) provision of health infrastructure to selected schools and health centers, among other activities. 6 As a result of the combined efforts of the national and international community, the number of suspected cases has drastically reduced from >350,000 reported in 2011 to <30,000 in 2014. 7 Approximately 15,800 new cases were reported between January and May 21, 2016, similar to the reported incidence during the same period in 2015. 8
Challenges associated with responses to the cholera epidemic are well documented, including: limited baseline sanitation coverage, particularly in rural areas; logistical challenges of reaching multiple IDP camps; unclear division of services among more than 100 nongovernmental organizations (NGOs); and emergency interventions not aligned with long-term DINEPA plans, among others.Reference Gelting, Bliss and Patrick 9 Additionally, the need for interventional research that addresses the knowledge gaps and informs quality improvement efforts has been well established.Reference Lurie, Manlio and Patterson 10 Numerous best-practice guidelines, statistics on caseloads, investigations of the source and serotype, and reports of efforts by local and international partners have been published. However, despite the wealth of literature pertaining to the 2010 Haiti earthquake and subsequent cholera outbreak and responses, specific data on implementation, costs, and results generally are unavailable. Given the extent of the resources already spent and to be committed to combating cholera in Haiti, it is logical to examine previous interventions, including comparing approaches and assessing effects, outcomes, impacts, cost, and efficiency, in order to best inform future efforts. The purpose of this study was to examine publicly available documents relating to the interventions provided to cope with the 2010 cholera epidemic to answer the following questions: (1) What information is publicly available on interventional studies conducted during the epidemic, and what was/were the impact(s)? and (2) Can the interventions be compared, and what lessons can be learned from their comparison?
Methods
A PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA) literary review was conducted using the search parameters “Haiti” and “cholera.” Studies were categorized as “interventional research,” “epidemiological research,” and “other reports.” Only research papers published in English and whose primary focus was the Haitian cholera epidemic were included for analysis. Articles published between January 2010 and December 2014 were examined. The PubMed database was selected for its broad accessibility. Following the PRIMSA model for systematic reviews, articles were identified, screened for inclusion, and full text assessed for eligibility (Figure 1).Reference Alessandro, Altman and Tetzlaff 11 Snowballing technique was initially used within the PubMed database, without identification of additional studies meeting inclusion criteria.
Figure 1 PRIMSA Flow Diagram.
For the purpose of this study, the following definitions were used:
Epidemiological Study—a detailed investigation and analysis of factors determining and influencing the frequency and distribution of disease, injury, and other health-related events and their causes in a defined human population for the purpose of establishing programs to prevent and control their development and spread;Reference Birnbaum, Daily and O’Rourke 12
Interventions—action(s) by humans to prevent, attenuate, create, or enhance change;Reference Birnbaum, Daily and O’Rourke 13
Interventional Study—a detailed investigation and analysis of the effects, outcomes, costs, impacts, and process of interventions;Reference Birnbaum, Daily and O’Rourke 12 , Reference Birnbaum, Daily and O’Rourke 13 and
Narrative Report— a descriptive report that does not include a detailed investigation of the intervention (ie, does not include a Methods section, information on beneficiary selection, technical approach, guidelines followed, outcome, and impact of the intervention).
The following definitions, adapted from the Glossary of the Organization for Economic Co-operation and Development (OECD; Paris, France), were retroactively applied to the results in an attempt to identify which studies included data on output, outcome, or impact measures:
Output—the products, capital goods, and services which result from an intervention;
Outcome—the likely or achieved short-term and medium-term effects of an intervention’s outputs; and
Impact—the positive and negative, primary and secondary, long-term effects produced by an intervention, directly or indirectly, intended or unintended. 14
Articles that consisted of narrative reports of interventions were not examined in sufficient detail for further classification using the categorization scheme devised by Birnbaum et al.Reference Birnbaum, Dudek, Adibhatla and Ramsel-Miller 15
Finally, recognizing that most organizations responding to or reporting on disasters do not publish in peer-reviewed journals, an expanded search of publicly available documents also was conducted, including: the United States Agency for International Development (USAID; Washington, DC USA) Development Experience Clearinghouse (DEC); the WHO; and the US National Library of Medicine (NLM; Bethesda, Maryland USA) Disaster Informational Management Research Center.
Results
The PubMed search yielded 192 papers, of which 21 were excluded because they did not meet inclusion criteria. Of the 171 papers examined, 12 (7.0%) met the criteria to be included as an “interventional study” (Figure 2). An additional 60 (35.1%) were “epidemiological studies,” and the remaining 99 articles (57.9%) were characterized as “other.” Many epidemiological studies were conducted to identify the origin and strain of Vibrio cholerae responsible for the outbreak, and to characterize pre- and post-earthquake conditions that may have contributed to the proliferation of cholera. Those characterized as “other” did not meet the definition of an interventional study. Articles in this category did not provide a Methods section and/or detailed description of the intervention or its effects. They consisted of narrative descriptions outlining interventional efforts by various agencies, news reports, comments on other articles, and several cholera modeling studies. Of these, 26 (15.2% of total papers; 26.3% of “other”) were unstructured, narrative reports of interventions. The expanded search of the DEC, WHO, and NLM databases yielded many additional narrative reports on the Haiti cholera response; however, no additional interventional research studies were identified.
Of the 12 interventional studies identified, six (50.0%) assessed oral cholera vaccination, four (8.0%) reported on one oral cholera vaccination pilot project and feasibility study conducted by Partners in Health (PIH; Boston, Massachusetts USA) and Le Groupe Haitienn d’Etude du Sarcome de Kaposi et des Infections Opportunistes (GHESKIO; Port au Prince, Haiti), and seven studies (58.3%) evaluated various other cholera interventions, including use of household disinfection kits, implementation of an alert and response system, treatment outcomes during pregnancy, two cholera training programs, and a cholera preparedness simulation (Table 1).Reference Teng, Thomson and Lascher 16 - Reference Tauxe, Lynch and Lambert 27 Information was not presented using a consistent structure/format and provided varying levels of detail, even among the four studies examining a single oral cholera vaccination pilot project. Thus, it was not possible to make direct comparisons between interventions, approaches used, cost, outcome, or impact of interventions.
Table 1 PubMed Interventional Studies using the Organization for Economic Co-operation and Development (OECD) Definitions
Reports of the impact(s) of the intervention were particularly inconsistent between the interventional studies. Using the OEDC definitions, a distinction between “output,” “outcome,” and “impact” was not possible. While each study included output and outcome measures (generally number of people who received the intervention; achievement indices), only two of the studies included “impact” data (Table 1). In some cases, “outcome” information was clearly presented (eg, increases in knowledge following training programs as demonstrated from post-course tests).Reference Rajasingham, Bowen and O’Reilly 24 However, some outcomes were implied (eg, the successful demonstration of Sanchol oral cholera vaccine immunogenicity leading to the implied outcome that such a vaccine could be used in future cholera vaccination campaigns, which may have the potential to decrease the incidence of cholera).Reference Charles, Hilaire and Mayo-Smith 25 Additionally, impact could have been inferred, although no interventional study commented on the effect(s) of the intervention on long-term health or morbidity or mortality related to cholera. One study directly stated that it was not possible to draw any conclusions about the impact of the intervention.Reference Aibana, Franke and Teng 26
One epidemiological study noted that due to the nearly US$100,000 provided for water, sanitation, and hygiene (WASH) services to IDPs living in temporary settlements around Port-au-Prince, “residents of IDP camps have been largely spared from the [cholera] outbreak because of safer water supplies and improved sanitation in the camps.”Reference Tappero and Tauxe 28 Although vague, this statement goes further in documenting the impact of an intervention than any of the 12 interventional studies. In contrast, a characteristic finding of the interventional studies was that of Ivers et al, which stated the aim of the study was to “demonstrate the acceptability and feasibility of a reactive oral cholera vaccination campaign in the context of the ongoing Haitian epidemic…. We deemed the project successful by virtue of the uptake and coverage rates described.”Reference Ivers, Teng and Lascher 29 Only one study reported a decrease in case fatality rate following a cholera management training program, although it also noted that this decrease could not be attributed solely to the training.Reference Tauxe, Lynch and Lambert 27 Therefore, it was not possible to ascertain the impact of individual interventions nor to make comparisons between them.
Discussion
Studies of interventions provided before, during, or after a disaster-causing event are essential for the determination and validation of the effectiveness, efficiency, and benefits of interventions/responses provided. They supply the evidence required for the development and validation of standards and best practices based on these standards. Thus, interventional studies must contain sufficient detail on methodology, outcomes, and impact such that they are comparable, can be used to determine best approach, and replicable. The current study raises a number of concerns, including: (1) limited interventional studies in the publicly available peer-reviewed literature; (2) the paucity of impact data provided; and (3) the inconsistent structure and insufficient detail limiting replicability and comparisons of the studies.
Paucity of Documentation
The current study adds credence to prior documentation of the paucity of available interventional studies in the peer-reviewed literature required to inform standards and best practices. 14 , 30 , Reference Adibhatia, Dudek, Miller and Birnbaum 31 Only seven percent of the articles examined met the criteria for an interventional study. The proportions of the respective categories found in the current study are in close agreement with those found in other reported studies.Reference Ivers, Teng and Lascher 29 , 30 Furthermore, none of the articles examined in the extensive supplemental searches of the Center for Research in the Epidemiology of Disasters (CRED; Brussels, Belgium), USAID-DEC databases, the NLM Disaster Informational Management Research Center, and WHO publications could be classified as an interventional study. This paucity of published interventional studies could be related to the possibility that: (1) interventional studies are not being conducted; (2) there is a lack of information sharing; (3) the findings are not being reported in the peer-reviewed literature; and/or (4) the distinction between interventional studies and narrative reports used in the current study is too restrictive.
Interventions Are Not Being Studied—The austere conditions, continually evolving needs, numerous actors, priorities, and capacities, along with the lack of standardization of reports across donors and responding agencies makes interventions in the field of Disaster Medicine difficult to study. It is known that interventions are being provided, but there is little incentive for in-depth studies of the interventions provided. After-action and situation reports seemingly fill the needs of the donors, particularly during disaster response, and the donors have not insisted on the conduct of detailed evaluations and research of the interventions they have funded.
Information Sharing—Although many of the papers examined recognized the importance of and called for increased coordination of interventions, no single agreed-upon mechanism has been developed to collect and disseminate the information needed to do so. Mechanisms such as the Cluster System exist to facilitate information sharing, yet this information often is siloed according to sector (Societal System) with poor cross-cutting integration. 32 Several central information-sharing systems have been developed, including ReliefWeb (United Nations Office for the Coordination of Humanitarian Affairs; Geneva, Switzerland), the NLM’s Disaster Information Management Resource Center, country-specific sites such as the Haiti NGO Aid Map, and disaster-specific pages housed within larger organizations such as the UN-Inter Agency Standing Committee’s (Geneva, Switzerland) Clusters and the WHO. Each platform has been designed to allow agencies to house and share surveillance data, situation reports, assessments, and maps. However, the large number of sites, inconsistent search mechanisms and data organization, as well as variable use based on awareness of the site or participation in the assessment has severely hampered the development and consistent use of a disaster “evidence base” for the health aspects of disasters. Instead, efforts frequently are duplicated and intervening bodies continue to rely on the opinions of subject experts and general guidelines. For the most part, information has been viewed as parochial to the organization that provided the intervention.
Reports Not in the Peer-Reviewed Literature—Reports are being formulated, but the findings are not being reported in the peer-reviewed literature because the responding organization believes that: (a) it does not possess the capability or resources (capacity) to conduct studies or the special skills required to prepare them for publication in a peer-reviewed journal; and/or (b) it is concerned that any negative outcomes or rejection of a submission by a peer-reviewed journal may impair its ability to obtain future funding. In addition, it seems that there is little association between providers and the academic community; and/or that the reports are published in the gray rather than peer-reviewed literature; and/or the reports provided by the providers to their donors are not publicly available.
Differentiation between “Studies” and “Narrative Reports”—Although seemingly a minor distinction, the current study differentiated between a “study” and a “narrative report.” Papers characterized as narrative reports may include data on the number of people reached, value of donated material(s), or anecdotes from beneficiaries. However, they generally did not include a Methods section or provide information on beneficiary selection, technical approach, guidelines followed, or the outcome or impact of the intervention. Thus, narrative reports are neither replicable nor can they be compared to other interventions. This distinction may have substantial utility for future research into the health aspects of disasters. However, it is also likely to have contributed to the paucity of identified interventional studies.
Paucity of Impact Data
Another important finding of the current study relates to the paucity of identification of the actual impacts of the interventions identified on the affected population or the community-at-risk. Only two of the 12 interventional studies (16.7%) described an actual impact of the intervention on the population affected. Some of the studies inferred the possible impacts but did not actually assess the impact(s). This may be related to the rather loose definitions of “outputs,” “outcomes,” and “impacts” provided in the OECD Glossary.Reference Birnbaum, Daily and O’Rourke 13 For example, the definitions of an outcome and an impact in the OEDC Glossary related to the time (short-, medium-, long-term) at which the respective effects were assessed. These definitions do not relate the effects of the intervention to the objectives and goals of the intervention. Further, the use of achievement indices does not include the actual effects of the intervention. The application of the definitions of interventions using the Disaster Logic ModelReference Birnbaum, Daily, Kushner and O’Rourke 33 may clarify the description of the effects of interventions in the future. These definitions include:
Effect (output) — the result or consequence of an action [intervention]; products of the intervention provided; the outputs of a transformation process (production function) constitute the effects of any intervention;
Output — the effects of any intervention; the product of a [transformation] process, the direct product of an activity [intervention] including types, levels, and targets of services delivered;
Outcome — the changes or results that a specific intervention aims to achieve; the results of an intervention relative to the objectives; achievement of the level of function stated in the objective(s) of the intervention; and
Impact — a measure of the tangible and intangible effects (consequences) of one thing’s or entity’s action or influence upon another; changes that occur within a community, organization, society, or environment as a result of the outcomes [of an intervention].Reference Hemingway and Brereton 34
The use of these definitions should help to clarify description of the effects of an intervention. It is suggested that all reports of interventions should follow the structure provided by the Disaster Logic Model. 14
Inconsistent Structure and Insufficient Detail
Another finding in the reports and studies cited in the current study is the lack of a uniform structure. Without a standard format for epidemiological or interventional studies, it was difficult to make meaningful comparisons or draw conclusions. Since no randomized, controlled studies or even non-randomized, controlled (quasi-experimental) studiesReference Overtveit 35 were identified in the current study or in those previously referenced,Reference Ivers, Teng and Lascher 29 , 30 evidence to determine standards and best practices requiring the use of comparisons and syntheses such as are used in systematic reviews is not available.Reference Hemingway and Brereton 34 , 36 The lack of structure impairs the ability for comparisons and even complicates the implementation of systematic reviews, and thus, the material must be structured before comparisons can be made.
Finally, despite growing recognition of these challenges, global consensus has not been reached on how to move forward. While there is widespread appreciation for the needs for accountability and evidence-based interventions, there is no consensus on who has the expertise and authority to establish standards or where to house collected information. As a result, each organizational body continues to develop its own assessments, best-practice guidelines, and reports. Until such consensus can be reached, efforts at developing, conducting, and using evidence-based research will continue to be fragmented. Therefore, any attempt(s) at strengthening the evidence base for disaster interventions must begin with consensus building around a central mechanism to house and disseminate research, as well as developing standards for data collection. Once consensus is reached, donor agencies are uniquely suited to push for standardization across agencies, as well as to promote information sharing.
Limitations
Although no grey literature was included in the primary review, several sources outside of those included in the leading repositories of the peer-reviewed literature were examined. These reports did not add any additional interventional studies to those included in the current review. Some additional studies potentially may exist in non-indexed reports. The NLM currently is attempting to index some of the grey literature dealing with the health aspects of disasters. 37 In addition, during the examination of each of the articles identified, the scientific content was not examined or reviewed. This potentially could have skewed the results. In addition, the narrative reports were not accounted or categorized and may have rendered the number of studies less than anticipated. However, seeing as the narrative reports and the studies identified did not have any common structure, such an analysis would have been very difficult and most probably would not have contributed to the conclusions reached from the current study.
Although several of the studies related to the use of an immunization process, no attempts were made to identify whether the use of the immunizations contributed to increasing the absorbing capacity of the population exposed.Reference Birnbaum, O’Rourke and Daily 38 No systemic reviews or comparisons were identified. This limits the value of the studies examined. Therefore, the studies will have little, if any, impact on the development of standards and best practices for future risk-reduction efforts.
Conclusions
The current study found there to be few interventional studies with limited data on impact of the intervention. To this end, it is unclear what interventions were provided, their effects in terms of outcome and impacts, and/or the costs associated with the interventions. It is not known how much of the resources provided actually were used. The few studies identified had no common structure, and therefore, it was not possible to compare their effectiveness, efficacy, or efficiency thereby limiting macro-level conclusions. Thus, they have little value for informing standards and best practices. The cholera epidemic presented an opportunity for learning about the control of infectious diseases so that they do not result in a disaster. Many strategies for better studies of interventions in disasters should be implemented to ensure that future opportunities to learn from these interventions are not lost.
Figure 2 Distribution of Articles by Category.
