Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-02-11T12:08:56.552Z Has data issue: false hasContentIssue false

Exploring International Views on Key Concepts for Mass-gathering Health through a Delphi Process

Published online by Cambridge University Press:  23 May 2016

Malinda Steenkamp*
Affiliation:
Flinders University, World Health Organization Collaborating Centre for Mass Gatherings and High Visibility/High Consequence Events, Adelaide, Australia Torrens Resilience Institute, Flinders University, Adelaide, Australia
Alison E. Hutton
Affiliation:
Flinders University, World Health Organization Collaborating Centre for Mass Gatherings and High Visibility/High Consequence Events, Adelaide, Australia School of Nursing and Midwifery, Flinders University, Adelaide, Australia
Jamie C. Ranse
Affiliation:
University of Canberra, Australian Capital Territory, Australia
Adam Lund
Affiliation:
Department of Emergency Medicine, University of British Columbia, Vancouver, British Columbia, Canada School of Nursing, University of British Columbia, Vancouver, British Columbia, Canada Justice Institute of British Columbia, British Columbia, Canada
Sheila A. Turris
Affiliation:
Department of Emergency Medicine, University of British Columbia, Vancouver, British Columbia, Canada School of Nursing, University of British Columbia, Vancouver, British Columbia, Canada Justice Institute of British Columbia, British Columbia, Canada
Ron Bowles
Affiliation:
Justice Institute of British Columbia, British Columbia, Canada
Katherine Arbuthnott
Affiliation:
London School of Hygiene and Tropical Medicine, London, United Kingdom
Paul A. Arbon
Affiliation:
Flinders University, World Health Organization Collaborating Centre for Mass Gatherings and High Visibility/High Consequence Events, Adelaide, Australia Torrens Resilience Institute, Flinders University, Adelaide, Australia School of Nursing and Midwifery, Flinders University, Adelaide, Australia
*
Correspondence: Malinda Steenkamp, PhD Torrens Resilience Institute, Flinders University Box 15, Mark Oliphant Building Laffer Drive, Bedford Park South Australia 5042 E-mail: malinda.steenkamp@flinders.edu.au
Rights & Permissions [Opens in a new window]

Abstract

Introduction

The science underpinning mass-gathering health (MGH) is developing rapidly. However, MGH terminology and concepts are not yet well defined or used consistently. These variations can complicate comparisons across settings. There is, therefore, a need to develop consensus and standardize concepts and data points to support the development of a robust MGH evidence-base for governments, event planners, responders, and researchers. This project explored the views and sought consensus of international MGH experts on previously published concepts around MGH to inform the development of a transnational minimum data set (MDS) with an accompanying data dictionary (DD).

Report

A two-round Delphi process was undertaken involving volunteers from the World Health Organization (WHO) Virtual Interdisciplinary Advisory Group (VIAG) on Mass Gatherings (MGs) and the MG section of the World Association for Disaster and Emergency Medicine (WADEM). The first online survey tested agreement on six key concepts: (1) using the term “MG HEALTH;” (2) purposes of the proposed MDS and DD; (3) event phases; (4) two MG population models; (5) a MGH conceptual diagram; and (6) a data matrix for organizing MGH data elements. Consensus was defined as ≥80% agreement. Round 2 presented five refined MGH principles based on Round 1 input that was analyzed using descriptive statistics and content analysis. Thirty-eight participants started Round 1 with 36 completing the survey and 24 (65% of 36) completing Round 2. Agreement was reached on: the term “MGH” (n=35/38; 92%); the stated purposes for the MDS (n=38/38; 100%); the two MG population models (n=31/36; 86% and n=30/36; 83%, respectively); and the event phases (n=34/36; 94%). Consensus was not achieved on the overall conceptual MGH diagram (n=25/37; 67%) and the proposed matrix to organize data elements (n=28/37; 77%). In Round 2, agreement was reached on all the proposed principles and revisions, except on the MGH diagram (n=18/24; 75%).

Discussion/Conclusions

Event health stakeholders require sound data upon which to build a robust MGH evidence-base. The move towards standardization of data points and/or reporting items of interest will strengthen the development of such an evidence-base from which governments, researchers, clinicians, and event planners could benefit. There is substantial agreement on some broad concepts underlying MGH amongst an international group of MG experts. Refinement is needed regarding an overall conceptual diagram and proposed matrix for organizing data elements.

SteenkampM , HuttonAE , RanseJC , LundA , TurrisSA , BowlesR , ArbuthnottK , ArbonPA . Exploring International Views on Key Concepts for Mass-gathering Health through a Delphi Process. Prehosp Disaster Med. 2016;31(4):443–453.

Type
Special Report
Copyright
© World Association for Disaster and Emergency Medicine 2016 

Introduction

A mass gathering (MG) is an event where there is a concentration of people at a specific location for a specific purpose over a defined period of time in sufficient numbers to potentially strain the planning and response resources of a host community, state, and/or nation.Reference Steenkamp, Hutton, Ranse, Lund, Turris, Bowles, Arbuthnott and Arbon 1 Key public health (PH) risks include the potential for delayed emergency response and wide-scale health effects because of access issues or environmental features. 2 The science underpinning mass-gathering health (MGH) is developing rapidly.Reference Arbon 3 , Reference Lund, Turris and Bowles 4 However, current evidence still fails to adequately inform the understanding of MGs and related health aspects, largely because of the lack of theory development, as well as non-standardized terminology and concepts.Reference Ranse, Hutton and Turris 5 There is, therefore, a need to develop consensus and standardize concepts and data points of interest to aid in creating a robust MGH evidence-base that will permit meaningful comparisons and meta-analysis between events to inform different levels of governments, event planners, responders, clinicians, and researchers.

A recent paper, from which this work derives,Reference Arbon 3 proposed some foundational MGH theory. The paper described various nested populations to consider during a MG event (host community; audience/attendees; people delivering services at the event; and patients presenting for health care). It also presented a diagrammatic depiction of two population models (a basic and an expanded model), together with a more complex figure, to represent the health care system that applies during a MG. Moreover, it identified three phases in an event (ie, before, during, and after).

The previously published paper evolved from ongoing work between the authors (as described in detail elsewhere)Reference Arbon, Cusack and Verdonk 6 with the ultimate goal to initiate international agreement on key MGH concepts, data definitions, and data elements. This work will form the basis of a MGH minimum data set (MDS) to serve small, medium, and large-sized events taking place in low, medium, and high-resource settings. The proposed MDS and its accompanying data dictionary (DD) will provide a common set of concepts and terminology for data users. The MDS will allow for the collection of consistent PH and acute care data that will inform MG operations and planning, advance MGH theory and practice, and support research.

After the publication of the population model paper mentioned above, the authors were keen to gain insights into how a wide range of MGH experts viewed the concepts put forward. More importantly, they wanted to engage the broader MGH audience in collaborations to move towards the establishment of an internationally agreed upon MGH-MDS and DD. They employed a two-round Delphi process involving volunteers from two international MG groups to test agreement on the published concepts.

Report

Process

A Delphi process is an established and practical approach that allows a group of individuals, who are not in one location, to facilitate decision making and agreement/consensus with regard to complex concepts.Reference Turris, Steenkamp and Lund 7 The first online Delphi round for this project was conducted from August through September 2014 with its questions largely based on the abovementioned paper.Reference Arbon 3 The content of the survey was drawn together by the first author and transferred into an online format (Qualtrics 2015, LLC; Provo, Utah USA) by the second. The other authors commented on the content and four tested the survey, together with five colleagues from Flinders University (Adelaide, Australia).

Members of the World Health Organization (WHO; Geneva, Switzerland) Virtual Interdisciplinary Advisory Group (VIAG) on Mass Gatherings and the MG Section within the World Association for Disaster and Emergency Medicine (WADEM; Madison, Wisconsin USA) were invited to participate via the relevant groups’ email distribution lists. The World Association for Disaster and Emergency Medicine is an international member-based professional association that aims to improve prehospital and emergency health care, PH, MGs, and disaster health and preparedness. The WHO VIAG is an international expert network of around 140 experienced MG specialists and senior practitioners. Membership is by invitation. The MG Section of WADEM is a special interest sub-group of approximately 60 members. There was little overlap between the two groups at the time of the surveys (ie, less than five VIAG members were WADEM MG Section members). Members from both groups who responded to the email invitation were sent a link to the online survey. Participation was voluntary and participants were asked to give their consent for taking part and as to whether they wanted to be acknowledged in any publications that resulted.

The survey provided background information about the project, then presented questions related to demographic information (age, type of organization, and country of residence). The core of the survey concerned specific concepts (Table 1): (1) the use of the term “MG HEALTH” vs “MG MEDICINE;” (2) the purposes of the MGH-MDS and DD; (3) the conceptual diagram of MGH; (4) the basic and expanded population models; (5) the phases of an event; and lastly (6) a data matrix for organizing MGH data elements. The questions and responses were designed to determine whether the participant agreed (“Yes”) or not (“No”) with the concept. This approach forced a finite decision to facilitate the Delphi consensus process, and Likert-type scales were therefore not provided. Participants had the opportunity to provide commentaries to support their decisions through open-ended questions. The survey could take between 10-40 minutes, depending on the level of input provided. Participants could complete the survey in one sitting or in stages with input being saved automatically. Data analysis was undertaken using descriptive statistics and content analysis. Concepts were considered to have achieved consensus if 80% or more of the participants indicated “Yes” in their answer.

Table 1 Overview of Survey Content, Percent Agreement Reached during Round 1, and Subsequent Revisions, as well as Agreement Reached during Round 2

Abbreviations: DD, data dictionary; MDS, minimum data set; MG, mass gathering; MGH, mass-gathering health.

a Not all questions were answered by all participants and the number of responses and the denominators for each question therefore varied.

Three months after the completion of Round 1, participants were provided with a summary of the findings along with an invitation to participate in Round 2. The second round was conducted from January through February 2015. The survey presented refinements to, and revisions of, the concepts based on the participants’ input from the first round (Table 1). The aim of the second round was to interrogate consensus around key concepts where there was less than 80% agreement. The input from Round 1 was used to formulate five MGH principles. Where ≥80% agreement was reached on a concept, an underlying principle was formulated and participants were asked for further comments only. In cases where agreement was <80%, proposed principles were put forward. Participants were then asked whether they agreed (“Yes”) or not (“No”) with the proposed principle. For one newly formulated principle, a Likert scale was included as a broad range of views were anticipated on the details proposed. The data matrix from the first round also was revised. The second survey was designed to take approximately the same amount of time to complete as the first. It asked for updates to the demographic information and then required feedback on the underlying and proposed principles, as well as the revised data matrix. The content of the survey was compiled and put online by the same two authors using the same software and approach (eg, automatically saving of entries). The other authors commented on the content and three of them tested the online survey. Only participants from Round 1 were invited to participate in Round 2, thus ensuring the “Delphi” process that draws a group of experts from broad agreement to greater consensus was maintained. Consent for participation and acknowledgment in publications were again obtained for Round 2. Feedback on the findings from the second survey was presented in a WHO Collaborating Centre (CC) workshop and two oral presentations at the 19th World Congress for Disasters and Emergency Medicine (WCDEM) in Cape Town, South Africa in April 2015.

Findings and Interpretation

Participants

Fifty-one volunteers (26% of 195) expressed an interest in participating in the first survey. Of these, 44 accessed the survey with 38 starting it and 36 (71% of 51; 19% of 195) answering all questions. Twenty-four participants (67% of 36; 47% of 51; 12% of 195) completed Round 2. Table 2 provides more details about the participants in the two rounds who indicated that it:

“Has been quite useful to collaborate with the Delphi because it raises many important questions that one has to deal with while organizing an event.”

Table 2 Characteristics of Delphi Participants

Abbreviation: WHO, World Health Organization.

a Mean (Standard deviation); Range.

b Other countries were: United Kingdom, Switzerland, Sweden, Portugal, New Zealand, Italy, Israel, Ireland, Germany, Greece, France, Czech Republic, China, Canada, Belgium, and Austria.

c Counts will add up to more than 36 as participants could tick more than one option.

d Other types of organizations included: international airport, private consultancy, non-governmental organizations, and professional representative body.

e Other roles included: field epidemiologist, consultant for disaster management, manager of emergency response coordination, medical advisor, municipal emergency management director, and emergency care provider in hospital.

f Other languages spoken were: Czech, German, Greek, Portuguese, Dutch, Italian, Dari, Pushto, Chinese, Hebrew, and Italian.

Agreement on Concepts

Table 1 provides an overview of the survey content for the online surveys and indicates the percent agreement obtained for each of the key concepts presented in both rounds. Agreement was reached on: using the term “MGH” (n=35/38; 92%); the stated purposes for the MDS and DD (n=38/38; 100%); the basic and expanded MG population models (n=31/36; 86% and n=30/36; 83%, respectively); and the event phases (n=34/36; 94%). Consensus was not achieved on the overall conceptual MGH diagram (n=25/37; 67%) and the proposed matrix to organize MGH data elements (n=28/37; 77%). In Round 2, agreement was reached on all the proposed principles and revisions, except for the conceptual diagram where agreement was 75% (n=18/24). More detailed participant feedback about the concepts is presented below.

Use of the Term MG HEALTH Rather than MG MEDICINE

In general, respondents were of the opinion that “health” was a more comprehensive and “appropriately broad” term that included important aspects such as “public health;” “a multidisciplinary approach;” “environmental and social impact of mass gatherings;” “public safety;” “harm reduction, injury prevention, and health promotion;” “first aid, mental health;” “mental resilience;” “health legacy;” and “nursing.” It was also stated that:

“Most who attend a mass gathering are well and should not need medicine.”

Respondents indicated that:

“Medicine … refers specifically to more clinical and medical aspects,” and that it is a “narrow” concept.

There were, however, some reservations about the proposal, and there was the opinion that there are advantages and disadvantages with both terms. One participant pointed out that these pros and cons were found:

“[W]ith travel health (as proposed by WHO) and travel medicine (proposed by the founders of the Society).”

It was suggested that term mass-gathering medicine (MGM) should be reserved for a:

“Specific subset that represents just physicians - in other words, a subset of the broader topic of MGH.”

Those who did not agree with the proposal indicated that the term “health” was “fashionable” and “politically correct” and that health:

“By definition, is a state of being; [while] Medicine is actions taken to achieve health.”

They also indicated that:

“Mass-gathering medicine [MGM] … is linked … with emergency medicine and with disaster medicine” and that “mass gathering ‘health’ could be too general.”

The authors conceptualize MGM to be an important, founding sub-set of the broader umbrella of MGH.

Purposes for the MDS and DD

All participants agreed with the stated purposes for the MDS and DD (as outlined above in the Introduction, and as previously published),Reference Arbon 3 and indicated the proposed purposes were sound, useful, “clear, simple, tangible, and achievable,” and that:

“Standards/guidelines for practice would be useful for management of mass gatherings,” “fill an important gap,” and that there is a “need to be able to compare data,” even though it is difficult “to have organizers share data.”

This aspect was not explored in Round 2.

Event Phases (Before, During, After)

Ninety-four percent of the participants (n=34/36) agreed with the concept in Round 1. They indicated that the “rationale is sound” and that the distinction between the three phases is of the:

“Utmost importance for calibrating the response of the host community” and that the “different phases of the event should be clearly noted so that changes in the time and changes between events can be monitored.”

This is because there are:

“Different operational requirements during [the] different phases of the event” and “populations are concerned about different things at various times during the event.”

One participant stated that the:

“Before and during [phases] are much more important than post the event” and another that “defining the pre- and post-event time period may be difficult.” Also, “both the ‘pre-event’ and ‘post-event’ information may be extremely hard to capture.”

It was recognized that MGs and the populations involved in these events change and evolve over time. One participant pointed out that the:

“Population[s] should be modelled for each separate phase of the event” and another that attention should be paid to the “different population characteristics, such as age (young vs elderly).”

Operationally, there are relevant times/phases when event populations are predictably in flux, notably: before (eg, arriving in the host community or leaving to avoid the event); during (eg, attending the event itself and moving between events/locations); and after (eg, leaving for home or returning to the host community). Times and populations for each separate phase of event should be considered longitudinally to capture the fluctuations of the model as an event unfolds. This aspect was not explored further in Round 2, but was restated as an underlying principle (ie, that MGs are temporary, time-limited events).

Population Models

Respondents indicated that “in general,” the basic population model (Figure 1a) was “a good starting point,” “clear and easy to understand,” that the “rationale is logical,” and that it reflects “real life;” but, also that it was perhaps “oversimplified.” It was also stated that the:

“Model clearly illustrates the different the populations affected by the event” and that there is agreement with the “penumbra concept,” (ie, people on the fringe of the event).

Figure 1 Original and Revised Population Models with Percentage Agreement Achieved for each Model.

Among those who agreed with the proposed model, there were reservations, suggestions, and clarifications offered. One participant indicated that the:

“Term ‘patient’ population is misleading” as it takes “a medical approach” and the “focus needs to be on the health of everyone in every circle, including the people living in the host community.”

Another participant stated that the:

“Model does provide a single view of event populations” and does not “include tourist and international visitors.”

For some participants, the idea of the population on the fringe was new and important because:

“Social media is becoming a part of crowd making source and [it] seems … [that] potential participants are motivated by this communication tool [to join the event in some way].”

It was recognized that this population is, however, difficult to identify, define, and count. There were a number of comments about where “event workers fit into the event population” as they are an important “sub-set.” Feedback also indicated that the expanded population model (Figure 1b) is not necessary and that one model could capture the national and international aspects. It was pointed out that “flow between populations” was possible and that this needs to be captured in the diagram. These comments formed the basis for revising the population model, as in Figure 1c, and for formulating the second underlying MGH principle (ie, that MG populations are fluid and encompass different groups that can be organized as a system comprised of “nested,” sometimes overlapping populations). This aspect was not explored further in the second round.

Conceptual Diagram Representing MGH System

Overall, the diagram (Figure 2a) was seen as an “adequate,” yet “simple presentation” of a single venue, planned event that is “generally applicable” and did “capture some of the aspects” of MGH. However, agreement was not reached for this concept. It was suggested that the diagram provides a clear pathway to further medical aid and presents an accurate picture of the “medical model of care” for an event with a single client group. One participant stated that the diagram:

“Reflects that when event-specific medical operations are created, they not only serve that event, but must have a relationship with existing health resources in the community.”

Figure 2 Representation of Interaction between the Health Care System that Applies during a Mass Gathering Event. Abbreviation: MG, mass gathering.

There were concerns that the model was:

“Too simplistic,” “far too medical” in that it refers “to individuals who are suffering from an adverse health event to be treated,” and it ignores “the preventive part.”

Specific gaps were identified and changes suggested. These could be organized into different themes around the perspective and scope of the diagram, event populations and provision of health, and the interaction with the host community’s health system.

In terms of perspective and scope, the participants indicated that the global focus is missing, that the focus on medical operations “is a bit misleading and missing some aspects” of MGH, such as “surveillance/monitoring that is needed from a public health perspective.” Also, there:

“Are many other considerations such as public safety (eg, overcrowding and stampedes), security (eg, terrorism attacks), services provision, and environmental hazards” that should be considered and the diagram “does not reflect these concerns.”

Another point raised was that the model is not applicable for mass casualties or disaster situations. The diagram also did not represent events that:

“Span multiple locations or may be integrated into the host community in a way that appears to represent a ‘surge’ in the community population through the influx of visitors extended over a multi-day or multi-week period (ie, Olympics or World Cup).”

In terms of the event populations, attendees:

“May not come from the community (ie, for some gatherings the largest number of participants comes from outside the host community and this has a significant impact on the community itself) and may not have associated medical operations self-contained at a specific site.”

The diagram also does not convey whether:

“The ‘event’ or ‘event population’ also includes the support staff/employees who are servicing the event.”

This will, of course, include the “medical operations.”

Comments related to health care aspects highlighted that not all patients seen at an event are transported to the host community’s health service and that this might:

“Bolster the idea that value of an event medical service is not just to triage all patients [sent] to the local hospital.”

It was also pointed out that the diagram does not capture the health impact of the event on the host community, for example:

“The event population attending community health services outside of ‘the event’ can displace the community’s access to those health services” (ie, have an impact on waiting times). Also, “the event population mixing with the community can transmit infectious diseases not previously present in that community.”

An important aspect raised was that health services may lie outside of the event but that:

“Planning for the event must include health services, just as medical operations is (sic) included.”

Another was that the diagram implies that only a small percentage of the event population will require care from the local health providers and lead to the misconception that:

“All events would have [a] minimal effect on the health services.”

Lastly, the diagram does not represent patients who bypass on-site health operations and attend health services in the host community directly, or events with a moving footprint (ie, long distance cycling event) that involves more than one health team.

These comments formed the basis for revising the diagram, as in Figure 2b. The original conceptual diagram was rephrased as a conceptual diagram of MGM, which is part of MGH. The diagram also attempted to detail the relationships between the fluid and overlapping event populations, health incidents, and the health care system. The diagram was intended to represent general concepts, and as such, did not depict multiple health care sites at an event, and/or multiple venues in a host community. The comments also lead to the formulation of the proposed principle: MG events interact with and affect health systems ranging from a local to (sometimes) global scale.

Consensus on the revised diagram was not reached. Feedback was that it was:

“Better than the original,” “put the mass gathering in a regional/international context,” “included more definitions,” was “more specific,” and reflected “the actual realistic situation” and “the broader impact that MGH events have.”

One participant did not agree with labelling the diagram “mass-gathering medicine.” The opinion was that:

“It should be called the health care aspect of mass-gathering health.”

Specific comments were that the word “community” was left off of the innermost square; that the lines of the various boxes should be dashed lines “to indicate the porous nature of the movement of people;” that the “existing health resources should also overlap the regional/national and international areas;” that the term “referred to” should be used rather than “transport;” and that the diagram should represent the “portion of event attendees who present directly to local hospitals.” Some participants did indicate that “it’s a bit more complicated diagram” and that “less text” would be helpful. It was also pointed out that regional/national and international elements appear static and that the diagram does not “represent the reality [well].” Perhaps taking a systems approach would be more helpful here to unpack the complex issue.Reference Keeney, Hasson and McKenna 8

Conceptual Approaches to MGH

Based on the comments on the various concepts, but especially on feedback about the term MGH and the conceptual diagram, the following principle was proposed: MGH embeds multiple conceptual approaches to health and safety. Within MGH, health is defined as per the WHO definition as a state of complete physical, mental, and social well-being, and not merely the absence of disease or infirmity. Accordingly, MGH is meant not only to protect individuals’ health, but also, where appropriate, to be used as an opportunity to promote health and have a positive influence (and legacy) upon health systems and the wider determinants of health. Mass-gathering health is not only about protecting individuals but also about the health of populations. Two broad and 12 more specific conceptual approaches were put forward for consideration in Round 2 and are shown in Table 3. This table also shows the percent of participants who strongly agreed or agreed with the proposed approaches according to a five-point Likert scale.

Table 3 Consensus on Broad and Specific Conceptual Approaches to MGH

Abbreviations: MG, mass gathering; MGH, mass-gathering health.

a Not all questions were answered by all participants, and the number of responses and the denominators for each question therefore varied.

Feedback on the principle was that it:

“Highlights the important factor that MGH involves many conceptual approaches.”

It was stated that adding the terms “health and safety” “has improved the concept and the application,” but the term could be misinterpreted to mean occupational health and safety as the two terms are often used interchangeably. It was also stated that the principle should clearly articulate “injury:”

“As this is clearly demonstrated in the literature and one of the major impacts on an MG.”

Another participant stated that an:

“Explanation of what is included in “health” (eg, public health, promoting health, etc) is important.”

The statement “embeds multiple conceptual approaches” in the principle was unclear and providing examples (eg, health promotion, disease prevention, mitigation of risk, and management of illness) “might make it clearer what is meant.” One participant stated that:

“In Round 1, I wrote that I would prefer MGM instead of MGH; however, in the light of this new and interesting concept of health promotion and its positive effect on participants, I agree with MGH principle.”

Participants agreed with the idea that MGH was about more than health protection and that aspects such as health promotion, legacy, and psychosocial should be included. It was pointed out that it might be ambitious to “cover all of the areas.” Some participants indicated that MG medicine should be combined with MG PH as:

“All the activities under MG medicine are covered under the PH’s items listed.”

Many of the participants’ comments related to specific MG aspects rather than to conceptual approaches: early warning and response; epidemic intelligence; weather protection; safe drinking water; environmental health; food safety; fire prevention; “intoxication prevention;” hazardous material facilities; volunteer services; Incident Command System; mental health services; health communication (before, during, and after); legacy; chemical, biological, radiological, and nuclear emergency preparation; security; MGM triage; as well as public safety and policing. Some participants indicated that public safety and policing should not appear under PH. Another participant stated that on-site health should be subdivided into smaller categories: diagnostics; reporting; records; training; responsibilities; duties; and insurance.

Data Matrix

Feedback on the original matrix (Table 4) was that it “is clear” and “looks easy to use.” Participants commented that it is a “good concept” or “start,” and that:

“This would be useful to structure the identification of the data elements.”

Table 4 Original MGH Data Matrix Presented in Round 1 (77% agreement reached)

Note: Only two cells are filled in here for illustrative purposes.

Abbreviation: MGH, mass-gathering health.

However, it was stated that:

“Additional work could be done on client groups;” “the structure of the matrix may need some further refinement;” “more attention should be given to the support personnel in general;” and additional components need to be added.

Specific suggestions for the matrix content were to include psychosocial aspects and likely threats. Those who did not agree with the model indicated that the:

“Event population and patient populations need to be better characterized in terms of chronic conditions and preventive measures;” that the focus was “on patients only, with little attention to the whole population participating to the event;” and that “the matrix should have different ‘rows’ instead of the different populations.”

Consensus was reached on the revised data matrix (Table 5) and feedback was that it highlights important components; organizes them clearly; and “captures in a single table the most important concepts.” The matrix was seen as a “good framework;” a “useful way of organizing information;” and that “having consistent criteria for data collection will help with standardizing research.” The matrix was seen as having the potential to be “further expanded in detail as needed.” Participants liked that the definitions were tightened and saw the matrix as:

“A starting point that will allow the collation of some data points – [and] if these broad headings are unable to accommodate the data points suggested, then the model can be reviewed/revised.”

Table 5 Revised Data Matrix Presented in Round 2 (83% agreement reached)

Abbreviations: MG, mass gathering; PH, public health.

It was recommended that security issues be a separate heading and that MGM be incorporated into MGH. Other aspects suggested for inclusion were legacy, communications, and planning. It was pointed out that the matrix refers to “domains,” but elsewhere in the surveys, the term “conceptual approaches” was used. It was recommended that consistent terminology be used and the terms “aspects” and “components” were alternatives. It was also recommended that the term “data” should be unpacked. One participant summarized the issues very well:

“The matrix is OK at a very high level, but definitions are required, including an explanation of what a data point is. Is the data point an event or a benchmark? The high-level principles are good. The model and the matrix still require more work to make them useful and relevant. There is no mention of timeliness or accuracy of data. Only if all of the above are met will such a tool be useful for organizers and government.”

Discussion

The Delphi process has shown that there is support for the idea of standardizing concepts and data elements amongst members from two representative groups of international MG experts. In April 2015, a half-day workshop was conducted with members of the WHO CCs on MGs, the WHO VIAG on MGs, and the WADEM MG Section, as well as other delegates ahead of the 19th WCDEM. During this meeting, a consensus statement was agreed upon indicating the support of the workshop participants for a MDS. There is, therefore, international support for moving forward on standardizing MGH concepts and data elements. Furthermore, the participants of the Delphi process outlined in this paper reached a high level of agreement on some broad concepts and principles underlying MGH: (1) MGs are temporary, time-limited events; (2) MG populations are fluid and encompass different groups that can be organized as a system comprised of “nested,” sometimes overlapping populations; (3) MGH embeds multiple conceptual approaches to health and safety; and (4) MG events interact with, and have an impact on, health systems ranging from a local to (sometimes) global scale. The one area where consensus was not reached was how to depict MGH graphically. The Delphi rounds and the WCDEM workshop have moved the MGH agenda forward, but the key question is “where to go from here?” Agreement was achieved on the revised data matrix presented in Table 5. This paper suggests that this matrix be the starting point for further international discussions for identifying and organizing more detailed MGH domains with relevant data elements. The WCDEM workshop also served as an opportunity to inform others of the work thus far and to gain input on how to best to proceed with this initiative. It is foreseen that a working group taking this forward will involve some of the authors as well as the heads of the WHO CCs on MGs. The work will involve international consultation, most likely through further Delphi rounds, with at least one face-to-face meeting if funding is obtained.

Limitations

Some limitations of the Delphi process were that the participants were members of the WHO VIAG and/or WADEM MG Section. As such, they may not be representative of all parties with expertise in MGH. In addition, although the Delphi approach is an established way to elicit agreement amongst experts, it is a challenge to fully convey the depth of abstract concepts in writing for an international audience where cultures and language differ. There are also no low resource countries represented in this project, which is an important consideration as the MDS is intended for use across all resource settings.

Conclusion

Governments, event planners, responders, clinicians, and researchers need a robust MGH evidence-base to inform the planning and hosting of events so that there are few adverse health effects and the positive health outcomes and event legacy outweigh these. The move towards standardization of data points and/or reporting items of interest will strengthen the development of a solid MGH evidence-base.

Acknowledgements

Dr. Mark Salter, Public Health England (London, England), contributed to the formulation of the key concepts. The following people participated in the online surveys and gave their permission to be acknowledged: Dr. Jana Prattingerova, Prof. Mo Salman, Daniel Smiley, Walter Gaber, Ms. Sophie Tyner, Dr. Jan Fizzell, Dr. Rowena Christiansen, Dr. Demetrios Pyrros, A/Prof. Joan Valas, Prof. Virginia Murray, Prof. Robert Steffen, Prof. Jeffrey Duchin, Dr. Luiz Henrique Hargreaves, Mr. Peter Kara, Dr. Kurt Anseeuw, Ms. Patsy Trethowan, Dr. Philippe Gautret, Dr. Alessandra Revello, Dr. Ann Knebel, Dr. Stephanie Davis, Mr. Kostas Kononovas, Mr. Alan Eade, Prof. Assaddullah Reha, Dr. Anders Johansson, Dr. Efraim Laor, Dr. Graham Dodd, A/Prof. Sarah Thackway, Dr. Judith Fisher, Dr. Francesco Grossi, Ms. Lara Payne Hallström, Dr. Ricardo Mexia, Sherie Sampson, and Dr. Mick Molloy.

References

1. World Health Organization. Public Health for Mass Gatherings: Key Considerations. Geneva, Switzerland: World Health Organization; 2015: 1-177.Google Scholar
2. Arbon, P. Mass-gathering medicine: a review of the evidence and future directions for research. Prehosp Disaster Med. 2007;22(2):131-135.CrossRefGoogle ScholarPubMed
3. Lund, A, Turris, SA, Bowles, R, et al. Mass-gathering health research foundational theory: part 1 - population models for mass gatherings. Prehosp Disaster Med. 2014;29(6):648-654.Google Scholar
4. Ranse, J, Hutton, A, Turris, S, et al. Enhancing the minimum data set for mass-gathering research and evaluation: an integrative literature review. Prehosp Disaster Med. 2014;29(3):280-289.CrossRefGoogle ScholarPubMed
5. Arbon, P, Cusack, L, Verdonk, N. Mass gathering public health and emergency medicine literature review: levels of evidence. Australasian Journal of Paramedicine. 2013;10(1):1-5.Google Scholar
6. Turris, SA, Steenkamp, M, Lund, A, et al. International consensus on key concepts and data definitions for mass gathering health: process and progress. Prehosp Disaster Med. In press.Google Scholar
7. Keeney, S, Hasson, F, McKenna, H. “Analyzing data from a Delphi and reporting results.” In: Keeney S, McKenna H, (eds). The Delphi Technique in Nursing and Health Research. Malaysia: Wiley-Blackwell; 2011: 84-95.Google Scholar
8. Homer, J, Hirsch, G. System dynamics modeling for public health: background and opportunities. Am J Public Health. 2006;96(3):452-458.Google Scholar
Figure 0

Table 1 Overview of Survey Content, Percent Agreement Reached during Round 1, and Subsequent Revisions, as well as Agreement Reached during Round 2

Figure 1

Table 2 Characteristics of Delphi Participants

Figure 2

Figure 1 Original and Revised Population Models with Percentage Agreement Achieved for each Model.

Figure 3

Figure 2 Representation of Interaction between the Health Care System that Applies during a Mass Gathering Event. Abbreviation: MG, mass gathering.

Figure 4

Table 3 Consensus on Broad and Specific Conceptual Approaches to MGH

Figure 5

Table 4 Original MGH Data Matrix Presented in Round 1 (77% agreement reached)

Figure 6

Table 5 Revised Data Matrix Presented in Round 2 (83% agreement reached)