Introduction
Mass-Gatherings Health (MGH) is an evolving niche of prehospital care rooted in emergency medicine, emergency management, public health, and disaster medicine. Mass gatherings attract large crowds and can strain the planning and health resources of the community, city, or nation hosting the event. Some mass gatherings have international attendance and may present risks to international health and security.Reference Guy, Prager, Turris and Lund 1 , Reference Al-Tawfiq and Memish 2 Patterns of injury and illness at mass gatherings often differ from those in the host community, based on risks intrinsic to the events.Reference Memish, Stephens, Steffen and Ahmed 3 Mass-Gatherings Health researchers aim to improve the evidence base, to optimize emergency planning, to improve health care for specific event populations, as well as to understand and mitigate the impact that mass gatherings may have on host community health resources.Reference Milsten, Maguire, Bissell and Seaman 4 , Reference Lund, Turris and Bowles 5
Currently, the MGH literature is filled with individual case studies and descriptive case series chronicling injuries and illnesses at specific mass gatherings.Reference Memish, Stephens, Steffen and Ahmed 3 , Reference Lund, Turris and Bowles 6 , Reference Arbon 7 A number of variables limit the generalizability of these reports, including differences in the size of events, geography, types of events, and a lack of standardized reporting. An inability to generalize the findings from case-based research has prompted calls in the literature for more theory-driven, quantifiable MGH research.Reference Lund, Turris and Bowles 6
Specifically, there have been concerted international efforts to develop a minimum data set for MGH research to address heterogeneity of data between nations, regions, and events.Reference Michael and Barbera 8 - Reference Steenkamp, Ranse and Hutton 12 A minimum data set addresses which data should be routinely collected at mass gatherings and allows for standardized reporting. A minimum data set is evolving.Reference Ranse, Hutton, Turris and Lund 9 , Reference Ranse and Hutton 10 As well, individual research groups are developing conceptual models to enhance understanding of the health effects of mass gatherings.Reference Turris, Steenkamp and Lund 13 , Reference Arbon 14
Importantly, in debates surrounding the quality of MGH research, there is currently little discussion on the core issue of poor data quality. Because scientifically rigorous research cannot be carried out in the absence of high- (or adequate-) quality data, in the current report, the authors explore fundamental front-line issues related to primary data quality in the context of mass gatherings.
The need to improve data quality was highlighted to the authors after an unsuccessful attempt to characterize patient presentation rates (PPRs) and model resource utilization at endurance events. When data were evaluated post-event, it was discovered that the data were incomplete due to missing fields, illegible entries, incomplete charts, and other factors that severely limited the intended analysis. After discussing data quality within the team and with external collaborators, the authors found that this was not an isolated incident. A review of the literature echoed similar issues within many MGH publications, but no authors specifically characterized contributing or mitigating factors.
The current lack of consistent, high-quality data may contribute to a poor research evidence base. As such, there is little concrete evidence to support individuals or organizations in proposing national or international standards for MGH research or clinical practice.
Report
Methods
This special report draws on several decades of experience providing care at mass gatherings and the creation and maintenance of a five-year, web-based, event and patient registry containing more than 20,000 patient encounters at 90 events spanning 239 event days.Reference Lund, Turris and Bowles 15 Data in the registry were collected and maintained with approval from the University of British Columbia Clinical Research Ethics Board (Vancouver, British Columbia, Canada) in accordance with the Tri-Council policy statements for ethical conduct for research involving humans.
The data quality limitations discussed here were generated through a stepwise approach combining both a review of the literature and anecdotal experiences from the authors and other collaborators providing care and performing research at mass gatherings. The following report provides citations where applicable; otherwise, observations are based on the authors’ clinical and research experience in this area.
Data Collection at Mass Gatherings
In the field of MGH research, patient-related data are typically extracted from non-standardized Patient Encounter Forms (PEFs). In the event context, the primary purpose of the PEF is to record the clinical details of a patient visit, not to collect data for researchers. The PEF is a health record and a legal document primarily intended to record appropriate clinical information, and the related care, treatment, and disposition provided by the care providers attending to the patient.
After patient disposition has been determined and the PEF is completed, research assistants may extract data into a spreadsheet, registry, or other clinical database. At some events, this can be done in real time on-event. At events with a predictable patient surge over a short time, or when care is provided over a large geographic footprint (eg, marathon or triathlon with geographically distributed aid stations), data transcription may have to be done post-event. Anecdotally, in a substantial number of cases, research data is incomplete due to missing fields, incomplete notes, or illegible documentation. An example of the “usual” flow of patient record keeping during events is presented in Figure 1.
Figure 1 The Context of Clinical Documentation in the Field.
Results
Complete documentation during a special event presents many unique challenges, detailed below. Based on observations, the authors suggest that human, environmental, and logistical factors contribute to poor data quality in the setting of mass gatherings (Table 1).
Table 1 Human, Environmental, and Logistical Factors Affecting Data Quality in Mass-Gatherings Health ResearchReference Guy, Prager, Turris and Lund 1 - Reference Memish, Stephens, Steffen and Ahmed 3
Human Factors
Human factors contributing to poor data quality at mass gatherings include interdisciplinary variability in provider documentation styles and a less structured, more “relaxed” clinical atmosphere where documentation may not be emphasized. Documentation style, and specifically comprehensiveness and legibility, varies greatly among health care providers (HCPs). Anecdotally, legibility is a common reason for unusable data; however, it has received relatively little attention in the literature. Illegible data are unusable data. As well, hospital-based HCPs may be accustomed to a narrative style documentation while some prehospital providers utilize charts that incorporate predominantly pre-determined data fields. Point form or checkbox documentation may be more uniform but restricts the recording of subtle, though important, clinical findings. Mass gathering medical teams are often multi-disciplinary; no “perfect” PEF exists to meet the needs and preferences of every group of HCPs.
In addition, although the fun, relaxed atmosphere of mass gatherings is attractive for many providers, it may decrease emphasis on documentation. For example, providers may be drawn to cheering on athletes at the finish line or listening to their favorite musician rather than putting their usual polish on charting. Some providers may better appreciate the importance of documentation in the context of their non-event clinical environment (ie, their “day jobs”) than in the casual setting of a community mass gathering.
Environmental Factors
Team workload creates environmental factors at mass gatherings that influence data quality. The clinical load affects documentation, and the quality of documentation inevitably influences data quality.
Changes in PPRs, the acuity of patient presentations, and the case mix all determine the level of intervention required and influence workload for the entire team. Additional factors that may influence the PPR and workload on event include: changing or extreme weather, surges in patient volume, event type, event location (eg, remote, rural, or urban), event activities that increase risk for illness and/or injury, and crowd demographics and behaviors.Reference Memish, Stephens, Steffen and Ahmed 3 , Reference Lund, Turris and Bowles 6 , Reference Arbon 7
Of note, the type of mass gathering may also influence data quality (eg, marathon vs music festival). These two event types have very different patient profiles, and the type of patients can influence the load on HCPs independent of the PPR (eg, surge of arrivals at peak finishing times, multiple reassessments, duration of stay, and complexity of care).
In a bustling medical tent, professionals may produce more incomplete forms as practitioners are focused on providing care to their patients and often must intake new patients prior to completing their existing documentation. Additionally, many medical tents are organized with different areas that are designated for different levels of acuity (eg, first aid, supine care areas, or resuscitation). As patients “step down” from higher acuity to lower acuity areas, care providers may not transition with them. This leaves the person who completes charting as someone only involved with the final phase of care, which ultimately can affect data completeness and quality.
Logistical Factors
Logistical factors also can contribute to poor data quality, primarily in relation to PEF design and use. The design of the form should ideally be appropriate to address both clinical and research needs. Staff must be familiar with forms and standards of documentation in order to reduce data variability and optimize completeness. Health care teams are often comprised of providers with vastly different skill levels, experience, and ability to provide clinical interpretation (eg, discharge diagnosis and diagnostic coding).
As well, as described in Figure 1, multiple forms for a single patient may be generated. For example, a patient retrieved in the field may have a first aid form, on-site ambulance service form, a main medical tent form, and a transfer ambulance form, which in subsequent data entry and analysis may unintentionally be interpreted as separate encounters. This circumstance is particularly common when medical teams are comprised of professionals from several agencies, each of which may have their own documentation requirements (eg, First Responders employed by a sport stadium working with a higher level of care team that includes nurses and physicians).
Discussion
The issue of poor data quality during mass gatherings is rooted in the complex interplay of human, environmental, and logistical factors. As such, there is unlikely to be a single solution for solving the problems of poor data quality. The authors suggest that a combination of robust pre-event planning, on-site actions and support, and post-event follow-up will improve data quality (Table 2).
Table 2 Actions to Improve Data Quality in Mass Gatherings Health Research
Abbreviation: MGH, Mass-Gatherings Health.
Pre-Event Planning
Creating a culture of excellence surrounding documentation is central to improving MGH data quality and begins during pre-event planning. Specifically, PEF design, staffing needs, and the potential integration of electronic medical records (EMRs) should be considered. Whether a PEF is used to collect clinical and/or research data, it should be concise, intuitive, and capable of meeting both clinical and research needs. Separate forms for research and clinical purposes lead to additional documentation workload, and compliance with data capture may be reduced. Separate forms, however, might be necessary when investigating a specific variable not usually recorded on standard PEFs.
Inadequate staffing can also contribute to poor data quality; therefore, staffing needs should be assessed based on the type and size of event. This can help prevent staff from becoming overwhelmed by patient care duties – a contributor to incomplete documentation.
Finally, although an expensive investment for many medical service providers, EMRs have been shown to improve documentation in dynamic clinical environments.Reference Lund, Turris, Amiri, Carson and Lewis 16 In addition, EMRs allow for more rapid analysis of data, as additional on-event or post-event manual entry of data into a centralized database can be bypassed. Electronic medical records also may provide injury and illness surveillance on-site, allowing for real-time hazard reduction or public health alerts.Reference Eden 17
On-Site Actions
Although some environmental and logistical influencers of data quality can be mitigated through pre-event planning, human factors affecting data quality are best addressed through on-site actions. In a prehospital ambulance-based setting, involving staff in ongoing research and the related review of charting led to improved documentation.Reference Zeitz, Zeitz, Arbon, Cheney, Johnston and Hennekam 18 On-site briefing and documentation training is especially important in a mass-gathering setting given the diverse range of HCP experience and because many staff are unaware of the expectations surrounding documentation.Reference Zeitz, Zeitz, Arbon, Cheney, Johnston and Hennekam 18 A “walk-through” orientation to the PEF may be useful, as staff may not be familiar with paperwork during mass gatherings.
Anecdotally, presenting fictionalized samples of atrocious charting for humorous effect has been used to educate staff regarding the importance of legible, appropriate documentation. As well, assigning a team leader in each zone or care space ensures accountability for the documentation of each clinical encounter. The individual in charge can take responsibility for data quality and documentation through real-time audit and feedback. Review of forms for completeness improves data capture, and if necessary, follow up with HCPs can be done while everyone is still on-site. As well, the team leader can help answer any questions and give friendly reminders about clinical care and the quality of documentation.
Post-Event Follow-up
Finally, all mass gatherings should have post-event follow-up, in order to debrief staff and other event stakeholders on clinical care as well as documentation quality. This provides a forum for staff to voice concerns and suggest improvements. Anecdotally, teams have responded well to incentivizing excellent charting using small rewards. Timing of debriefing is critical, and can occur post-shift, post-event, or in the weeks following; however, it should ideally be performed while the event is still fresh in the minds of team members. Similarly, if changes are not proposed and recorded in the early post-event period, they are often forgotten, especially for events that only occur once per year.
Conclusion
Although data quality at mass gatherings is complex and multi-factorial, MGH organizers should consider implementing a combination of the above approaches when planning for medical care at mass gatherings. While certainly not an exhaustive analysis of the issue, the authors hope to spark further discussion and action to improve primary data quality at mass gatherings.
To advance MGH research, researchers must continue to identify factors adversely affecting data quality and develop effective strategies to mitigate them. This is an exciting time for MGH research as higher order questions are beginning to be addressed; however, quality research must start from the ground up to ensure optimal primary data capture and quality.
