Introduction/Background
Case reports are commonly used to report the health outcomes of mass gatherings (MGs), and many published reports of MGs demonstrate substantial heterogeneity of included descriptors. As such, it is challenging to perform rigorous comparisons of health services and outcomes between similar and dissimilar events. The degree of variation in published reports has not yet been investigated.
Case reports, case series, and field reports all have a recognized role to play in advancing knowledge about MGs, and many health care specialties have recommended specific standards for reporting as an important strategy for improving data collection.Reference Rison, Kidd and Koch1-Reference Kulling, Birnbaum, Murray and Rockenschaub7 In the context of MGs, standardized reporting has been discussed for years, with several authors having argued that this strategy will improve the quality of case reports.Reference Fattah, Rehn and Wisborg8-Reference Schwellnus, Kipps and Roberts11 These researchers have proposed diverse approaches to the shared goal of standardizing data collection for the health outcomes of MGs.Reference Fattah, Rehn and Wisborg8-Reference Schwellnus, Kipps and Roberts11 To date, no such standardization has been achieved.
The purpose of this study was to explore patterns of post-event medical reporting in the existing literature and to identify inconsistencies in reporting. This paper is part of a larger body of work that aims to detail the evidence supporting the development and widespread use of a specific, standardized reporting template that can evolve over time. At the time of publication, there is no available reporting template to assist with rigorous evaluation of MG characteristics as well as their effects on health outcomes through comparison of the outcomes for similar and dissimilar events.
Methods
The present study is a systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology.Reference Moher, Liberati, Tetzlaff and Altman12 The research questions were formulated using a Population, Intervention, Comparability, and Outcomes (PICO) method (Table 1).
Table 1. PICO
Sampling Strategy
The original search strategy was discussed with a research librarian (PB) who searched multiple databases (Table 2). This attempt to locate case reports provided scant results, therefore a decision was made to hand-search the Tables of Contents of two journals known to regularly publish case reports and/or case series. Prehospital and Disaster Medicine (PDM) and Current Sports Medicine Reports (CSMR) were chosen for this review for two reasons. Both PDM and CSMR are widely recognized as journals of MG research and evaluation, and both journals regularly publish post-event medical reports. The inclusion time period was from January 2009 through December 2018, with the literature search beginning in January 2019 and concluding in February 2019. Inclusion and exclusion criteria are described (Table 3). As this study may support the development of a post-event medical reporting template that proposes essential data points, both full manuscripts and conference proceedings were included.
Table 2. Summary of All Search Terms Used
Table 3. Search Strategy Inclusion and Exclusion Criteria
Data Extraction
All papers that met the inclusion criteria were subjected to a review protocol and independently reviewed by two separate authors (CC, EC, HR, and ST) for the presence or absence of specific reported contents. The research team employed an integrated knowledge translation approach, and as such, the methodology evolved as the study progressed.13
As the goal of the project was to examine and document the current state of post-event health outcomes reporting, the research team generated a short list (a priori) of descriptive data categories related to event descriptors and health outcomes. This list of variables evolved as the analysis progressed, eventually producing 27 categories of data for analysis (Table 4). Each category of data was sought in every publication through a line-by-line review and then recorded as either present or absent using Smartsheet (Smartsheet Inc.; Bellevue, Washington USA; 2018). Data points were grouped in the following categories: use of language/terminology; reporting on event context; reporting on hazard and risk; and reporting on clinical outcomes. Of note, the list of 27 data categories was subsequently used to inform the development of domains of data for post-event reporting, described in a subsequent paper in this series.
Table 4. Mass-Gathering Reporting Variables Marked as Either Present or Absent in Each Study
Data Analysis
Descriptive statistics were generated for each individual report as well as across the entire sample. Bias was mitigated by having multiple authors review the same study to ensure quality reviews. Any non-alignments in categorizing data were resolved through internal team discussion. Gaps in reporting were identified and documented.
Results
Reviewed Publications
Seventy-five abstracts were identified by applying the search terms (Table 2). These abstracts were reviewed using inclusion and exclusion criteria (Table 3). Forty-eight abstracts were accepted for further analysis after eliminating duplicates and papers that did not meet the inclusion criteria (Figure 1).Reference Liberati, Altman and Tetzlaff14 An additional six publications from PDM were added to the study after a search of references amongst included papers. A final total of 54 international publications were reviewed.Reference AlAssaf15-Reference Webster and Jones68 Forty-two (78%) were full case reports or case series and 12 (22%) were conference proceedings published from the MG medicine (MGM) track at the World Association for Disaster and Emergency Medicine’s (WADEM; Madison, Wisconsin USA) Congress on Disaster and Emergency Medicine. Table 5 describes the types of events reported. The “Other” category denotes unspecified sampling of multiple MG types (n = 7). Figure 2 illustrates the frequency with which a given data point was reported.
Figure 1. PRISMA Chart for Literature Retrieval.
Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Table 5. Summary of Event Types in Case Reports Reviewed
Figure 2. Frequency Rates for Reporting on Specific Topics.
Use of Language and Terminology
The studies frequently failed to self-identify as a case report (or equivalent) with no declaration made in 72% (n = 39) of studies. In the case reports reviewed, there were no consistent labels used to name and categorize events. In addition, use of terminology in post-event reporting was inconsistent. For example, the terms “transport” (ie, moving a patient on-site) and “transfer” (ie, moving a patient to a health care service located off-site) were used interchangeably.Reference Turris, Callaghan, Rabb, Lund and Munn58 In the United Kingdom literature, the term “casualties” was used, whereas in North America, the term was “patients.”Reference Burton, Corry, Lewis and Priestman19
The terminology used for post-event reports included: “field report” (2%; n = 1); “case report” (9%; n = 5); “after-action report” (4%; n = 2); and “case series” (13%; n = 7). Similarly, authors used various terms to describe forms related to patient documentation, including: “patient encounter form” (n = 10); “minor treatment log” (n = 4); “patient record” (n = 3); and “patient report forms” (n = 3).
Reporting on Context
Keywords (n = 215) were extracted from 39 of the papers; four papers and eleven conference proceedings did not report keywords. Of the 215 keywords employed, only six keywords were cited five or more times, including: “Mass Gathering” (n = 17); “Emergency Medical Services” (n = 11); “Mass-Gathering Medicine” (n = 9); “Event Medicine” (n = 6); “First Aid” (n = 5); and “Mass Gatherings” (n = 5). Of note, there were a total of seven different ways of labelling MGs, leading to inconsistent use of punctuation and divergent uses of the same term (ie, “Mass Gathering” [n = 17]; “Mass-Gathering Medicine” [n = 9]; “Mass Gatherings” [n = 5]; “Mass Gathering Medicine” [n = 3]; “Mass-Gathering Events” [n = 3]; “Mass Gathering Health” [n = 1]; and “Mass- Gathering Medicine” [n = 1]). The more recent term “crowded spaces” (drawn from the safety and security literature) as employed to describe MGs was not used in the papers reviewed.Reference Williams69-71
Researchers provided data about event demographics (ie, event type [100%; n = 54]); attendance (estimated [89%; n = 48]); setting (indoor/outdoor [63%; n = 34], bounded/unbounded [33%; n = 18], or remote/rural/urban geography [43%; n = 23]); event culture (ie, history of the event [if applicable], target audience, and genre); climate and weather conditions (for outdoor events; eg, season [24%; n = 13], temperature [43%; n = 23], humidity, and wind chill); and temporality (eg, day/night, duration [70%; n = 38], and operating hours of the event).
The five most commonly discussed variables were: scope of event (100%; n = 54); type of event (100%; n = 54); activity level (98%; n = 53); country (93%; n = 50); attendance (89%; n = 48); and number of patients seen (89%; n = 48). The five least frequently addressed variables were: percentage of patients seen and transported (17%; n = 9); medical direction (20%; n = 11); ambulance transfer rate (20%; n = 11); whether an event was ticketed (22%; n = 12); and the season that the event occurred in (24%; n = 13). Of the 27 categories of data analyzed, 13 variables were found to be in use in over 50% of studies.
Reporting on Hazards and Risks and Risk Mitigation
Hazards and risk factors were identified/discussed in 38/54 studies, which produced 142 hazards and risks (Table 6). The most common hazards identified were crowd factors (20%; n = 28) and environmental factors (19%; n = 27), comprising nearly 40% of all reported hazards. Crowd dynamics as reported related largely to the mood of the crowd (eg, calm, agitated, or aggressive); crowd intention (eg, observation versus participation); activity level; and crowd size. Discussion about environmental hazards focused primarily on temperature, inclement weather, dust, and humidity. Although alcohol was written about slightly more often (n = 14) than substance use (n = 10), the two were frequently cited in conjunction as event-related risks.
Table 6. Identified Hazards and Risks
Discussions about event-related hazards and risks were focused on bounded versus unbounded events, the purpose of event, accessibility of the site, event duration, and ambient noise level. Geography pertained mainly to the altitude, natural features/hazards of the event (eg, rivers), and terrain. Activity level was mainly focused on whether the individual was a participant (actively engaging with the event) or an attendee (passively spectating the event), with those participating being at a higher risk for event-related injuries. Sanitation was also identified as a hazard, with communicable diseases and sewage management issues being common themes.
On-site health services was an area of focus infrequently reported (20%; n = 11). Information on medical direction, as seen in Table 4, were generally not reported. Additionally, gaps were consistently identified in the description of the composition (eg, scope of practice of team members, number of medical personnel) of on-site medical teams and on-site equipment and supplies. Whether medical teams were private (ie, not drawing on public infrastructure) or public (ie, provided by provincial or state authorities) was generally not discussed.
Reporting on Clinical Outcomes
Forms of reporting varied with regard to the type of patient presentations and the statistical methods employed. The majority of papers (89%; n = 44) reported the total number of patients seen, sometimes including non-medical encounters such as requests for sunscreen, condoms, or other health promotion products. Forty-nine studies included information about the illnesses and injuries that occurred during a specific MG (91%; n = 49). Nearly one-half of the 49 studies (45%; n = 22) categorized illness/injuries using a mix of categories to describe the patient population (ie, a mix of a body systems and specific diagnoses). The specific types of patient presentations were inconsistently categorized according to body system affected (10%; n = 5), presenting complaint (20%; n = 10), acuity designation (12%; n = 6), diagnosis at discharge (10%; n = 5), and injured versus not injured (2%; n = 1). Descriptive statistics also varied.
Discussion of Current State
To the authors’ knowledge, no researchers have undertaken a critical analysis of the current state of post-event medical case reporting for MGs. Case reports are essential tools for researchers and event operations team members such as medical directors and event producers. Accurate, comprehensive reports have the potential to grow the science of MGM. In general, the authors found that reporting was descriptive rather than explanatory; that is, focused on describing the outcomes as opposed to exploring possible connections between the data domains of event, the crowd, the hazards and risks, and health outcomes.
Previous efforts towards standardization have encompassed theoretical, operational, and clinical approaches. In 2004, Arbon proposed a conceptual model based on biomedical, environmental, and psychosocial models for MG health outcomes.Reference Arbon72 Over the last decade, Ranse and Hutton (along with collaborators) have used existing literature to further develop Arbon’s model, generating more granular categories of variables.Reference Ranse, Hutton, Turris and Lund73,Reference Hutton, Ranse, Gray, Turris, Lund and Munn74 Using an operational lens, Lund, et al sought to formalize and standardize data collection and analysis by developing an online Event and Patient Registry to capture and house data on the outcomes of MGs.Reference Lund, Turris, Amiri, Lewis and Carson75 More recently, Schwellnus, et al have proposed a clinical approach to data collection proposing a series of data points specifically for reporting the health outcomes of endurance sport events.Reference Schwellnus, Kipps and Roberts11 Mass-gathering medicine more broadly would similarly benefit from standardization of key terminology and classifications.
However, each group of researchers has employed a different lens in determining the relative importance of different variables (eg, a theoretical lens, an operational lens, or a clinical lens). There are at least three challenges embedded in understanding and adopting a unified approach to reporting on health outcomes that is both concise yet comprehensive. First, the silos of knowledge that are currently in place may encourage researchers, clinicians, and policy makers to remain in their individual silos with many missed opportunities for collaboration and the advancement of the science that underpins MG health. Second, ultimately the goal of MG research is not simply to describe outcomes, but also to explain outcomes, and ultimately to predict outcomes. This in term may ultimately influence planning and event design to create safer, healthier events. To accomplish the latter, health outcomes ought to be captured and understood within the larger context of the event and through multiple lenses. The third challenge acknowledges that events occur mostly outside of public and academic institutions with limited budgets. Researchers are not necessarily embedded in the planning or operational teams, which may limit access and reach into the event communities.
The work described above is taking place in the context of a larger movement to create a minimum dataset for MGs. A series of papers published in 2014-2016 described potential conceptual models for understanding and examining MGs.Reference Steenkamp, Hutton and Ranse76-Reference Turris, Steenkamp and Lund79 This work was meant to underpin the development, at some future point, of a universal data set. The present project, focused on harmonizing and standardizing published reports, is related to this earlier work. The papers cited above underline the necessity of creating a templated approach to reporting on event-related health outcomes. For example, the infrequent (and lack of) reporting on variables such as temperature or ambulance transfer rate impacted the universality of the data and lessen comparability across studies. Use of a standardized reporting template would permit the rigorous evaluation of MG event characteristics as well as their effects on health outcomes through comparison of the outcomes for similar and dissimilar events.
In current health care literature, there is a focus on understanding and measuring the impact of on-site medical teams that include not only first aid attendants, but also nurses, paramedics, physicians, and other team members relevant to health services and outcomes. Per the present analysis, case reporting consistently falls short of providing researchers with the information required to make meaningful comparisons between health outcomes at events supported by first aid teams and events with integrated higher level of care (HLC) teams. Higher level of care teams are those that integrate first responders, nurses, physicians, and other health care providers to safely monitor and provide care with the goal of release, avoiding a hospital transfer.Reference Turris, Callaghan, Rabb, Lund and Munn58 Without information about the influence of HLC teams on patient transfer rates, there is little robust evidence upon which to base the development of team size and composition guidelines for MGM.
As discussed by Lund and Turris in their review of the approaches used by researchers to categorize patient encounters, several methods are used including: categorization according to body system most affected (eg, neuro, cardiac); chief complaint; diagnosis; acuity; or some blend of the above.Reference Lund, Turris, Amiri, Lewis and Carson75 This makes comparisons between event outcomes difficult. A recurring theme in the present analysis was the extent to which variability in reporting impacted the overall quality of the data pool available to researchers and clinicians. Information provided was incomplete and/or approaches to data analysis were neither systematic nor consistently applied.
Finally, a challenge underlined in this review was that the statistical metrics used to report patient encounters were inconsistent. Two common examples included the use of mathematical formulas and the need for a denominator. Metrics such as the Patient Presentation Rate (PPR), Patients Per Ten Thousand (PPTT), and Medical Utility Rate (MUR) were each reporting on a similar clinical outcome using a different mathematical formula. Such a lack of standardization created challenges to accurately cross-reference the findings.Reference Ranse and Hutton80 And the rates that describe the number of patients seen require the total attendance at the event. This denominator is the basis for the calculation of the PPR, PPTT, and MUR. However, the denominator may not be entirely accurate or consistently calculated (ie, with or without workforce included) and an incorrect attendance report can skew the reporting metrics.
Future Research
A case study reporting template could provide notable improvements to the way that case reports are written, disseminated, and analyzed. Ideally, the development of a template for event reporting, supported by an online Registry, would proceed simultaneously; however, coordinating multiple simultaneous projects can delay forward momentum. The authors acknowledge that the present review and the soon to be published post-event medical reporting template are in the first iteration and will evolve rapidly with the support and input of the international community.
Limitations
Variables for the “current state” of post-event reporting were mined from two key journals. It is possible that case reports published in other journals differ from those reported in PDM and CSMR. It is possible that the inclusion of conference proceedings created a methodological challenge. For example, Wendell, et al published a conference proceeding in PDM, which was subsequently expanded and published as another abstract in the Annals of Emergency Medicine.Reference Wendell, Bitner, Ossmann and Greenwald64,Reference Wendell, Bitner, Ossmann and Greenwald81 Undoubtedly, the conference proceedings provided an abbreviated version of individual studies. There cannot be an expectation that conference proceedings will have “full data reporting” compared to full reports. Publication bias may be present as events without significant or notable outcomes (eg, small or medium-sized events without any major injuries) may not be published. The research team included conference proceedings to assist in mitigating the potential for publication bias.
An unexpected limitation arose halfway through the literature search when the PDM database would not allow for certain conference abstract retrievals. This led to the authors using other databases to retrieve the pre-identified abstracts. However, it is unknown how many other conference abstracts are inaccessible due to this search challenge.
Conclusion
This study describes the current state of post-event reporting on health outcomes. The authors found that based on a review of ten years of case reports, there is substantial variability in reporting. Systematizing reporting would allow comparisons and improve the ability to identify effective strategies for making events safer, and such a template might serve as a de facto planning tool. This paper proposes a single, overarching recommendation: the creation and utilization of a reporting and publication template to support researchers and clinicians. Such a template would permit comparisons between events and event outcomes, allow for meta-analysis of similar events, improve the comprehensiveness and accuracy of data being reported, and strengthen the science underpinning the field of MGM.
Acknowledgment
Dr. Paul Arbon has been, and continues to be, a leader in the field of MG health. He has worked tirelessly, at both individual and systems levels, to build the science that underpins MG health. The Australian/Canadian collaboration in this field began more than a decade ago, under Paul’s leadership. And so, the authors dedicate this series of papers, the culmination of more than a decade of thinking, exploration, and research, to Dr. Arbon. They thank him for his mentorship and for sharing his expertise and commitment to the health and welfare of the millions of people who participate in MGs every year. The authors also wish to thank Dr. Alison Hutton at the University of Newcastle for reviewing a mature version of this manuscript.
Conflicts of interest/funding
Turris is a shareholder with a medical services company that provides health care services for mass gatherings. Rabb, Chasmar, Ranse, and Callaghan have no Conflicts of Interest to declare. Munn is the medical director for an annual major music festival and provides volunteer and paid services working as a director and clinician at other events. Lund is the medical director and a shareholder of a medical services company that provides health care services for mass gatherings. All the authors take on both volunteer and paid roles providing medical services at mass gatherings. None of the authors received income for this project, which is unfunded.
Supplementary Materials
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X21000066
