Exercise Purpose and Design

The primary objective of the exercise itself was to provide an opportunity for the state and local authorities to learn about and assess the presence of effective legal authorities needed to implement social distancing measures in the event of a pandemic, and to establish the level of competence and willingness of public health professionals to apply those laws. Using program guidance provided by the Association of State and Territorial Health Officials, the tabletop exercise was designed and led by faculty and staff of the HSPH-CPHP19 and planned in conjunction with the Commonwealth of Massachusetts Department of Public Health and the Executive Office of Public Safety.

The exercise was preceded by expert presentations on pandemic influenza and the public health response and a review of relevant Massachusetts laws. Such presentations incorporated an overview of legal authorities in 2 types of declared public health emergencies (public health emergency and state of emergency) and legal authorities in the absence of declared emergency. Federal and state powers were examined as was the authority of local agencies and law enforcement. Examples referring to previous public health legal challenges such as the severe acute respiratory syndrome epidemic in 2003 were highlighted. The didactic material used during these presentation and the exercise scenario can be found on the HSPH-CPHP Web site.19

The exercise scenario was designed to provide opportunities to address issues such as respect for gatherings of religious obligations; the role of university, local, and state police officials in enforcing social distancing measures; distribution of food and medications; workplace safety; discrimination against residents of a specific ancestry; and assistance to special needs populations. The scenario also provided opportunities to explore conflicts that would test authorities at the state and local levels and in the private sector. In particular, universities and food manufacturing plants played a role in the scenario and the consequences of decisions made by business owners were discussed during the exercise.

To meet these objectives, the scenario opened with the announcement of the first suspected human case of avian influenza type A (H5N1) in New York, diagnosed in a businessman returning from a trip to eastern Europe, is where in our scenario more than 1000 cases had been reported in the previous months. Within 2 days, the first suspected cases were reported in Massachusetts: 2 students who lived on a university campus returning from a trip to New York. Consequently, the health department of the town where the university is located ordered the suspension of classes and the return of students to their family homes. The scenario spanned 15 days, and was designed to test the impact of social distancing measures on the economy of the communities affected by the epidemic in terms of roles and reactions of private and public entities to the outbreak. Within this time frame, the exercise was divided into 3 separate modules focusing respectively on emergency declarations and authorities; isolation, quarantine, and mass prophylaxis; and restrictions of public gatherings and assembly. Although this specific scenario was not pilot tested before it was used, the exercise’s design was based on more than 2 dozen other exercises that have been designed and implemented in Massachusetts and Maine in recent years.Reference Biddinger, Cadigan and Auerbach20

Participants included local-, regional-, and state-level professionals from a variety of disciplines such as public health, law, health care, public safety, and emergency management. In advance of the exercise, these participants were divided into groups of 8 to 10 individuals who convened around a table. The participants in each roundtable were chosen within regions and communities such that members of the same or neighboring communities were seated together. An expert who was knowledgeable about the public health infrastructure of the geographical area being tested facilitated the discussion within each group at each table.

Evaluation and Statistical Methods

To measure the impact of participating in an exercise on the participants’ knowledge of and level of confidence in current legal authorities for infectious disease emergencies, participants were asked to complete a questionnaire that covered 6 issue areas: declaration of emergencies, isolation and quarantine, restrictions on the movement of people, curfew, closure of public places, and mass prophylaxis. For each area, participants were asked to rate the availability and sufficiency of both legal authorities and policies and procedures using a Likert scale ranging from 1 (not currently available) to 5 (available and sufficient). In addition to this generic evaluation, when appropriate, participants were asked to determine whether authorities needed to be strengthened at the local or state level and/or in the private sector. A “do not know” response option was available for all of the items. The initial questionnaire was collected before the exercise began and after the didactic sessions were completed. At the conclusion of the exercise, participants were asked to complete a second questionnaire with questions identical to the first. The questionnaires were administered in this way to focus the evaluation on the impact of participating in the exercise per se, but the authors recognize that this may have been influenced by the didactic presentations that preceded the exercise.

The questionnaire was anonymous, and pre- and postexercise forms were paired by means of a unique identifier. Job affiliation (local, state, and federal government; hospital and health care providers; nonprofit organization; and others) and profession (legal, public health, public safety and emergency management, health care providers, health care administrators, and others) were the only respondent characteristics identified. Survey implementation was approved by the Humans Subjects’ Committee of the HSPH.

Data were analyzed to assess the impact of the event in terms of knowledge gained in the 5 issue areas and changes in the participants’ level of confidence in the legal preparedness of the public health system. In addition, data were analyzed to identify areas most in need of further improvement. Subsequently, subjects were grouped by professional role to identify the differences in outcomes and perceptions between legal and nonlegal professionals.

To measure knowledge gained, we used the proportion of respondents reporting a substantive answer—that is, something other than “do not know”—before and after the exercise. To assess the level of confidence in the legal preparedness of the public health system, data from the Likert scale were analyzed in 2 ways: as a continuous variable and in terms of the proportion of respondents reporting that legal authorities, policies, and procedures are available and sufficient (Likert scale = 5). We regard changes in the participants’ assessment of their confidence in legal authorities, whether positive or negative, as indirect evidence of knowledge gained through participating in the exercise.

Fisher exact test for count data and paired t test for continuous variables were performed to compare pre- and postexercise results. The level of significance was set at α = .05. All statistical analyses were performed using STATA statistical software, version 9.0 (StataCorp, College Station, TX).

Respondents’ Characteristics

Fifty-six subjects, equivalent to approximately 63% of the registered exercise participants, completed both the pre- and postexercise questionnaires. An additional 22 subjects completed the preexercise form only, and 2 completed the postexercise form only; these subjects were not included in the analysis. The 56 respondents with completed pre- and postexercise questionnaires form the basis for the results presented in this study. The nonresponse rate was 29%. Because the questionnaire was anonymous, we could not determine the characteristics of nonrespondents.

All of the respondents reported their profession: 43% were public health officials; 34% were legal professionals; and 23% included public safety, emergency management personnel, health care providers, and health care administrators. The rest reported a profession different from the above-mentioned categories. All of the subjects reported their job affiliation: 25% local government, 50% state government, 9% federal government, 5% hospital health care providers, 5% nonprofit organizations, 2% business and unions, and 4% other type of affiliation.

Knowledge Gained

After the exercise, participants were more likely to report a substantive answer (different from “do not know”) for all of the items. Across all of the categories, the average increase in the proportion of questions answered was 25%. With regard to knowledge of policies and procedures, the improvement was statistically significant in all of the topic areas (P < .05), with the largest gains in curfew (36%) and restrictions on the movement of people (31%). Knowledge about the availability and sufficiency of legal authorities also increased by 30% on average, although the increase was statistically significant only for mass prophylaxis (P = .002). Participants also were more likely to be able to judge the need to improve authorities, policies, and procedures at the state and local level and in the private sector for all tested issues; however, statistical testing could not be performed due to the large number of zero cells. Table 1 provides a detailed description of these results.

TABLE 1 Proportion of Participants Reporting a Substantive Answer Before and After the Exercise

Level of Confidence in Legal Authorities

As illustrated in Table 2, participants reported a higher level of confidence in the availability and sufficiency of the legal authorities in all issue areas after the event (P < .05). On average, there was an increase of 12% in the proportion of participants reporting that legal authorities are available and sufficient (5 on the Likert scale), with increases ranging from 10% for isolation and quarantine to 15% for mass prophylaxis. Analyzing the Likert scale data as a continuous variable, the mean value per topic area increased on average by 1.2 points, ranging from no increase in isolation and quarantine to 2.2 points in curfew, closure of public places, and mass prophylaxis. The improvement in the level of confidence was statistically significant in 4 of 6 areas: curfew (P < .001), closure of public places (P < .001), restrictions in the movement of people (P = .02), and mass prophylaxis (P = .003).

TABLE 2 Change in the Participants’ Level of Confidence in the Availability and Sufficiency of Legal Authorities, Policies, and Procedures After Participation in the Tabletop Exercise

Level of Confidence in Policies and Procedures

The results in Table 2 show that the proportion of respondents reporting that policies and procedures were available and sufficient (5 on the Likert scale) did not significantly change for any of the topic areas. In analyzing the responses as a continuous variable, however, we were able to detect a significant improvement after the exercise in the average level of confidence regarding the availability and sufficiency of policies and procedures. Only for the topic area of isolation and quarantine was the change in the negative direction, that is, a decrease in the level of confidence (P = .002), after the exercise. A similar pattern was found for the topic area of declaration of emergencies, but in this case the decrease was not significant.

State and Local Levels and Private Sector

Overall, participants felt more comfortable after the exercise about the availability and sufficiency of legal authorities, policies, and procedures at the local and state levels and in the private sector. In particular, a significant increase in confidence was reported for the issue area of curfew, where confidence increased for both local (P = .008) and state (P = .003) authorities. Also, after the exercise, participants were more confident about authorities at the local level for enacting isolation and quarantine (P = .006) and at the state level for mandating restrictions in the movement of people (P < .001).

Identification of Legal Issue Areas in Need of Further Improvement

Based on the postexercise results, 33% (on average) of respondents felt confident about the availability and sufficiency of legal authorities. Only 14% (average of proportions), however, were confident about the policies and procedures to use those authorities. For the issue area of the restrictions on the movement of people, participants’ opinions on the sufficiency of legal authorities were high, with 35% of the sample reporting that they are sufficient; at the same time, this issue area was judged the worst in terms of availability of policies and procedures, with only 5% of respondents reporting that they are sufficient. On the contrary, participants seemed to be more satisfied with the availability of policies and procedures in the areas of closure of public places, curfew, and mass prophylaxis with 20%, 19%, and 17% of participants, respectively, judging them as sufficient.

In addition, participants were asked to assess the necessity of strengthening legal authorities, policies, and procedures at the state and/or local levels. After the exercise, 79% of participants reported that authorities, policies, and procedures should be strengthened at the local level, and 71% of participants reported that they need to be strengthened at the state level. In addition, 90% of participants agreed that there is a need to strengthen authorities, policies, and procedures in the private sector.

Comparison of Results by Professional Role

The level of knowledge of legal and nonlegal professionals was compared based on the postexercise results. Only in the topic area of mass prophylaxis was self-reported knowledge for legal professionals higher than for nonlegal professionals (91% of responses different from “do not know” compared with 61%, P = .04).

Comparing postexercise results by professional category, nonlegal professionals were less confident than legal professionals on the availability of legal authorities, particularly in 2 areas: curfew (19% vs 13%, P = .013) and closure of public places (19% vs 17%, P = .04). For all other topics, responses were similar, and as a consequence the level of confidence reported by each of the 2 professional groups was identical.