Improving public health emergency preparedness (PHEP) is at the top of the national agenda; however, the ability to conduct research aimed at improving PHEP is limited. Large-scale public health emergencies are relatively rare, but this good fortune has hindered the development of a PHEP “evidence base” because there are few opportunities to observe, measure, and study the myriad elements involved in PHEP outcomes. This, in turn, has slowed progress in identifying evidence-based practices, developing performance measures and standards, and otherwise improving the nation’s preparedness for large-scale incidents with health consequences (see Abramson et alReference Abramson, Morse and Garrett1 for a review of the existing literature on PHEP).

To address the gap in evidence, the recently enacted Pandemic and All-Hazards Preparedness Act (Public Law 109-417, 2006, § 101 et seq) directed university-based (and federally funded) Preparedness and Emergency Response Research Centers to begin conducting public health systems research related to PHEP. A recent report from the Institute of Medicine, commissioned by the Centers for Disease Control and Prevention, recommended a set of near-term research priorities for the Preparedness and Emergency Response Research Centers, including training, communication, preparedness and response, and metrics.Reference Altevogt, Pope and Hill2

Although identifying topics for PHEP systems research is important, it is also necessary to step back and develop a working consensus on what kinds of knowledge should be generated through the research and what kinds of approaches are most likely to generate it. Given the urgency of improving PHEP systems and the limited resources available for research, we argue that initial funding priorities should focus disproportionately on research that is practice oriented and aimed at producing actionable knowledge that can rapidly be put into practice. Based on our own experience in developing tools to support PHEP policy and practice, we believe that most or all PHEP systems research studies—regardless of their substantive focus—would do well to address the following 3 issues: identifying core PHEP processes for whatever specific capabilities are being studied, developing workable and transferable strategies for measuring those core processes, and developing performance improvement approaches that can support practical application of findings.

IDENTIFYING CORE PHEP PROCESSES WILL HELP FOCUS THE RESEARCH AGENDA

Developing a strategic focus for PHEP systems research will be critical given the need to generate actionable knowledge on a short timeline and with limited resources. The tremendous variation in public health threat profiles, response infrastructures, and community characteristics, combined with a limited experience base, has made it difficult to identify high-priority functions and processes that can form the basis of a clear and focused research agenda. This absence of clear priorities has been evident in federal PHEP program measures and guidance, which have shifted considerably over the years and have varied from agency to agency.

The Institute of Medicine’s adoption of a consensus panel’s PHEP definitionReference Nelson, Lurie and Wasserman3 in its report on research priorities for the Preparedness and Emergency Response Research Centers is a step toward developing consensus on a broad set of key PHEP elements. That definition, however, is pitched at a general level, and progress in defining specific research questions will require a more granular articulation of high-priority subcapabilities and processes. Similarly, there have been efforts to define critical pathways and measures of effectiveness for humanitarian disaster relief, but these efforts need to be adapted to cover the full range of large-scale public health emergencies.Reference Burkle4, Reference Spiegel, Burkle and Dey5 Finally, the Department of Homeland Security’s Target Capabilities List provides a starting point but does not achieve the specificity required, nor is it focused closely enough on public health practices.

One approach to identifying core PHEP processes is to adopt methodologies used in engineering, as the quality movement has done in health care.6 Process mapping has long been used to identify critical components of manufacturing processesReference McClain, Thomas and Mazzola7 and, increasingly, to identify key drivers of patient safety in clinical settings.Reference Dean, Hutchinson and Escoto8 By creating a detailed picture (or map) of the steps required to accomplish key PHEP functions, process mapping can be used to identify both high leverage and failure-prone components of PHEP. These, in turn, can help identify logical priorities for measurement, data collection, analysis, and improvement, enabling a resource-efficient approach to research on complex systems. Given the paucity of systematic process knowledge in PHEP, we have found it useful to construct process maps in consultation with expert PHEP practitioners.

For example, process mapping a complex activity such as mass countermeasure distribution/dispensing—which involves dispensing medication, warehouse and supply chain operations, security, traffic management, public communication, and other functions—would deconstruct it into smaller-grained activities and identify cross-cutting “building blocks” required for multiple functions. Such building blocks may include staff call-down and site activation, which are required for security, dispensing, warehousing, distribution, and operating an emergency operations center. Each building block, in turn, can be deconstructed further, providing specific and high-value focal points for research.Reference Nelson, Chan and Sloss9, Reference Lotstein, Seid, Ricci, Leuschner, Margollis and Lurie10 Observing exercises, reviewing after-action reports, and consulting with expert practitioners can help determine which of these building blocks are most failure prone and which are most worthy of additional study. For instance, recent work by RAND has identified pick-list generation, a subprocess of inventory management, as a critical and failure-prone step worthy of focused study and measurement for the countermeasure distribution/dispensing capability.Reference Nelson, Chan and Sloss9

Mapping out the process can be the first step in developing models for analyzing the dynamics of complex systems. Such models may be mathematical, in which the system is represented by a series of equations, or may involve simulations, in which a computer plays out a sequence of events. Models can explore how PHEP systems perform under different circumstances to help identify which system components have the most influence on desired outcomes,Reference Moore, Chan and Lurie11 to analyze costs and benefits of alternative strategies,Reference Fowler, Sanders and Bravata12 and to identify conditions under which certain responses may be favored over others.Reference Kaplan, Craft and Wein13, Reference Bozzette, Boer and Bhatnagar14

PHEP RESEARCH REQUIRES NEW DATA COLLECTION AND MEASUREMENT STRATEGIES

Measurement is also critical to public health practitioners, providing a foundation for data collection, analysis, hypothesis testing, and process improvement.Reference Altevogt, Pope and Hill2 The rare-event nature of PHEP requires creativity in recognizing and exploiting new data sources and making better use of existing sources.

Exercises for PHEP, which have grown from just a handful in 2002 to the thousands in recent years,15 provide an obvious data source of preparedness processes and capabilities. In addition, researchers can and should seek to learn more from the abundance of “free lessons”Reference Weick and Sutcliffe16 provided by more frequent, even routine, events and operations, such as the annual influenza season and its related activities,Reference Aledort, Lurie and Ricci17 around-the-clock case reporting,Reference Dausey, Buehler and Lurie18 outbreaks of foodborne illnesses, and other small-scale incidents.Reference Rendin, Welch and Kaplowitz19 In many instances, these events provide opportunities to observe the operation of building-block capabilities such as staff mobilization, public communication, and decision-making under uncertainty—all of which are required in large-scale responses.Reference Nelson, Lurie and Wasserman20 For instance, the decision to close a beach due to the presence of bacterial contaminants often involves calling in staff, preparing and disseminating messages to the public, and grappling with conflicting and uncertain data.

Additional investment is required, however, to take full advantage of these data collection opportunities. First, knowledge of core PHEP processes (described above) is needed to recognize which real-world events test specific cross-cutting capabilities. Second, additional investment is needed to develop standardized data elements that can support comparisons across settings and over time. Most of the data produced by exercises and incidents is contained in after-action reports, the structures of which are, despite efforts at standardizing formats,21 almost as varied as the individuals who produce them. In addition, there is no central repository for collecting, circulating, and comparing after-action reports.

PHEP researchers could look to accident reports of the National Transportation Safety Board,22 near-miss reports from the Aviation Safety Reporting System,23 reports on terrorism incidents from the RAND-Memorial Institute for the Prevention of Terrorism database,24 and elsewhere for ideas about how to extract comparable data elements from singular events. This standardization, in turn, will provide a foundation for research that seeks to identify the drivers behind exemplary practices, upstream predictors of performance, and standards that describe adequate levels of performance during emergencies.

Of course, an important limitation to collecting data from exercises and smaller-scale events is that it is not always clear how well these findings apply to large-scale events. Thus, when possible, some effort should also be devoted to exploring how well data from these sources reflect performance in real-world larger scale events.

ATTENTION TO PERFORMANCE IMPROVEMENT STRATEGIES WILL HELP ENSURE PRACTICAL RELEVANCE

A hallmark of practice-oriented research is a clear focus on how findings can be used in real-world contexts. The need to focus on practical, concrete application of research findings is particularly salient given that successful execution of capabilities involved in PHEP involve skills such as communication, coordination, and problem-solving—activities and skills that are difficult to codify and standardize.Reference Seely-Brown and Duguid25 Furthermore, variations in structure and functioning of state and local public health systems may imply that what works in one context will not work well in others.Reference Turnock and Atchison26 This has been a consistent finding in implementation research from other human services fields such as education,Reference Berman and McLaughlin27 substance abuse prevention,28 and personal health care.Reference Greenhalgh, Robert and MacFarlane29

Accordingly, PHEP research should make the state- and local-level process of customizing and adapting knowledge and practices an explicit focus of study. One promising avenue for such research is continued exploration of the applicability of quality improvement techniques (eg, process mapping, plan-do-study-act cycles) that can help PHEP-related organizations generate new, locally useful knowledgeReference Batalden and Davidoff30 (see Seid et alReference Seid, Lotstein and Williams31 for a review of quality improvement practices relevant to PHEP).

Most approaches to quality improvement involve analysis of trend data over time, which is difficult given the paucity of large-scale incidents. Although initial efforts have begun to link quality improvement methods to “free lessons” provided by more routine proxy events,Reference Lotstein, Seid, Ricci, Leuschner, Margollis and Lurie10 more research is needed to increase health departments’ ability to draw PHEP-related lessons from these less severe incidents. Another challenge lies in developing strategies that can be used to develop system-level improvements in a setting in which multiple levels of government (federal, state, and local) and multiple disciplines (public health, hospitals, law enforcement, emergency management, private businesses, and citizens) collectively produce preparedness and response.Reference Lurie, Wasserman and Nelson32 This implies that the PHEP research portfolio should include a balance of nationally, state, and locally focused studies to ensure adequate attention to implementation issues.

Acknowledgments

This work was funded by the Department of Health and Human Services/Office of the Assistant Secretary for Preparedness and Response.

Authors' Disclosures The authors report no conflicts of interest.