Recommendations and decisions to conduct unplanned school closures (USCs) are made by school and public health officials based on available information regarding the threats to community and student safety. 1 Reasons for such closures include severe weather, natural disasters, and facility problems. Reference Wong, Shi and Gao2 For example, preemptive school closures and dismissals may be recommended as community-level, nonpharmaceutical interventions to slow influenza pandemic transmission in the community. Reference Qualls, Levitt and Kanade3 The rationale for such preemptive closing is that schools serve as amplification points for influenza transmission, as children exhibit higher transmission rates than adults. Reference Fowlkes, Dasgupta and Chao4,Reference Nesti and Goldbaum5 Therefore, USC monitoring helps track outbreak response and control, and inform future pandemic preparedness efforts. Reference Wong, Shi and Gao2,Reference Zheteyeva, Gao, Shi and Uzicanin6
Representativeness, accuracy, cost-effectiveness, and timeliness are important attributes to consider in developing and evaluating public health surveillance systems. Reference German, Lee, Horan, Milstein, Pertowski and Waller7,Reference Groseclose and Buckeridge8 With the increasing popularity of social media, Reference Ewaldsson9 community-based surveillance incorporating social media monitoring may represent access to affordable, relevant real-time data. Reference Salathe, Freifeld, Mekaru, Tomasulo and Brownstein10 Researchers at the Centers for Disease Control and Prevention (CDC) have demonstrated the utility of “online systematic searches” (OSS) for USC monitoring. This strategy consists of systematic scans and searches of online media and data, using Google Alert, Google News, and LexisNexis Reference Wong, Shi and Gao2 (Supplemental Figure S1). In addition, a study on USCs in Michigan, demonstrated how monitoring posts on Twitter can enhance USC surveillance Reference Jackson, Mullican and Tse11 (Supplemental Figure S2). We expand the literature on the potential use of Twitter to monitor USCs by reporting on a comparison of the coverage of USCs in Georgia public schools. For this purpose, we compared tweets and OSS-based reports of USCs for 2015-16 and 2016-17 school years.
METHODS
The list of 2299 Georgia public schools and 232 school districts were downloaded from the National Center for Education Statistics (NCES) website 12 in August 2017. Based on the NCES data and using the Google and Twitter search functions with additional checking of school websites, schools with a Twitter account were manually identified in August 2017. The same procedure was applied to the list of school districts to identify district-owned Twitter accounts. Only official school and school district Twitter accounts were included. All publicly available tweets posted by these accounts were downloaded using the Twitter Search Application Programming Interface (API) on October 9, 2017. Schools that had ever posted a tweet on their page or were under school districts that had ever posted a tweet on their page, were coded as having active Twitter coverage (Supplemental Figure S3).
Using a list of keywords (provided by CDC researchers as in their OSS method) to identify tweets likely to include information on USC, all likely tweets on school closures were filtered from the tweets downloaded, and then stratified by school year. Relevant tweets were then manually coded to identify USCs and type of closure (school or school district closure). All duplicates were identified and eliminated manually. Twitter USC announcements were then matched to all the schools in the NCES data.
CDC researchers provided OSS-acquired USC data. (See Supplemental Materials for details.) We filtered the data to retrieve USC events announced by Georgia public schools and school districts, from August 2015 to June 2017. USC data obtained from OSS and Twitter were compared with individual school as a unit of analysis. We compared the frequency of schools with 1 or more USC announcements in a year to schools without USC announcements.
Statistical Analysis
Statistical analyses were performed in R version 3.4.3, by means of RStudio version 1.1.383. Statistical significance was determined a priori at α = 0.05. To compare USCs determined by Twitter with that by OSS, some characteristics (such as school locale, total student population, student-teacher ratio, reduced price/free lunch proportion) of schools with at least 1 USC determined by Twitter and OSS, respectively, were summarized by frequencies if the character variable is categorical (such as school locale) or by sample means if the character variable is continuous (such as student population). These descriptive statistics were then compared using Chi-square test for frequencies and Welch 2-sample t-test for sample means. Univariate and multivariable logistic regression were performed to determine whether certain school characteristics/factors were associated with USCs announced by Twitter and by OSS, respectively. We used city as the reference category for the school locale variable, facilitating comparison with our results from Michigan. Reference Jackson, Mullican and Tse11 Using the R package “sjstat”, the crude and adjusted odds ratios were converted to crude and adjusted relative risks (aRR).
This study was approved by the Institutional Review Board of Georgia Southern University (H15083) under the B2 exempt category, as the social media data analyzed are considered publicly observable behavior.
RESULTS
Of the 2299 Georgia public schools, 1864 (81%) schools had active Twitter coverage (627 schools had an active Twitter account, and 1237 schools were under districts with an active Twitter account). The proportion of schools with active Twitter coverage varied by geographic location: 89% (372/417) for city, 97% (830/857) for suburban, 59% (171/288) for town, and 66% (467/708) for rural, and 83% (24/29) for schools missing locale data (Supplemental Figure S4; Supplemental Table S9). Schools with active Twitter coverage had a significantly higher mean student population (806 students vs 595 students; P < 0.0001) and lower mean proportion of students receiving free/reduced-price lunch (66% vs 73%; P < 0.0001) than those without (Supplemental Table S9).
Among the 1864 Georgia public schools with active Twitter coverage, 638 (34%) and 981 (53%) with 1 or more USCs were identified in 2015-16, and 2016-17, respectively. In 2015-16, among the 638 schools with USCs, 222 were identified by Twitter alone, 215 by OSS alone, and 201 by both. In 2016-17, among the 981 schools with USCs, 178 were identified by Twitter alone, 107 by OSS alone, and 696 by both. Given that OSS is the current method adopted by CDC researchers, the marginal benefit of adding Twitter as a data source was an increase in the number of schools with USCs identified by 53% (222/416) in 2015-16 and 22% (178/803) in 2016-17 (Table 1).
TABLE 1 Number (%) of Public Schools With and Without USC Announcements Identified in 2015-17, Among 1864 Public Schools in Georgia With Active Twitter Coverage
Abbreviations: OSS, online systematic searches; USC, unplanned school closure.
Stratified by locality, in 2015-16, in cities or suburban areas, 182 schools with 1 or more USCs were identified by Twitter alone, 115 by OSS alone, and 75 by both; in town or rural areas, 40 were identified by Twitter alone, 97 by OSS alone, and 125 by both. In 2016-17, in cities or suburban areas, 111 schools with 1 or more USCs were identified by Twitter alone, 26 by OSS alone, and 458 by both; in town and rural areas, 64 were identified by Twitter alone, 81 by OSS alone, and 231 by both (Supplemental Table S19).
In 2015-16 school year, compared with city schools, suburban schools were the least likely to have an USC identified by OSS, while in 2016-17 school year, city schools were the least likely to have an USC identified by OSS (Table 2[i]). In 2015-16 school year, schools in all other locales, when compared with city schools, were more likely to have at least 1 USC announced on Twitter in 2015-16, but no differences were observed in 2016-17 (Table 2[ii]). However, additional analysis excluding tweets on USCs due to Tropical Storm Hermine (August 31 to September 2, 2016) and Hurricane Matthew (October 4 to 10, 2016), found that suburban schools were the most likely to have an USC identified by Twitter in 2016-17. (See Supplemental Tables S20 and S21.)
TABLE 2 aRR of (i) Schools Having a USC Identified via OSS and (ii) Schools Having a USC Identified via Twitter Search, Among 1864 Georgia Public Schools With Active Twitter Coverage
Abbreviations: aRR, adjusted relative risk; OSS, online systematic searches; USC, unplanned school closure.
DISCUSSION
Among public schools in Georgia with active Twitter coverage for the 2015-16 and 2016-17 school years, using both Twitter and OSS resulted in higher identification of schools with USCs compared with using either method alone. Furthermore, we found that school locale was an important predictor of identifying USCs by means of Twitter and OSS.
Twitter has a high penetration among Georgia public schools. A total of 81% of Georgia public schools had active Twitter coverage compared with the national average Twitter penetration of 24% among US adults. 13 Among schools with active Twitter coverage, slightly more schools with at least 1 USC announcement were identified by means of Twitter than OSS (7 more in 2015-16; 73 more in 2016-17). In 2015-16, suburban schools were the least likely to be identified by OSS in reference to city schools, but this was not the case for Twitter. This is noteworthy because most of the schools in Georgia are in the suburban locale. While this does not translate to Twitter being a better alternative, it provided additional USC data that the OSS did not pick up and viceversa.
The 2016-17 school year saw an increase in the amount of schools with at least 1 USC. When compared with the previous school year, there was a change in the relationship between the predictor variables and the ability to detect schools with at least 1 USC. There was an increase in the probability of schools with at least 1 USC in the suburban locale to be identified by means of OSS, while schools were equally as likely to be detected by Twitter regardless of locale. Based on our additional analytical findings (Supplemental Table S20 and S21), we infer that these changes were chiefly due to Tropical Storm Hermine and Hurricane Matthew that struck the southeastern United States on September 2 and October 7, 2016. Reference Stewart14,15 Due to the severe weather, several schools closed in preparation for the storm and to deal with the aftermath.
Spatial generalizability was assessed by comparing our results with those reported from Michigan. Reference Jackson, Mullican and Tse11 We found that, in both Michigan and Georgia, geographical locations of schools are predictors of whether schools with USCs are identified by either method. In Michigan, city schools had the highest risk of being identified by Twitter, but they had the lowest risk of being identified in Georgia. We hypothesize that this difference could be attributed to geographic differences in the actual use of Twitter to announce USCs. Because the direction of association between rurality and schools with USCs being identified by means of OSS is the same in both states, we believe this difference may be related to Twitter behavior of school officials in both states. There are 2 levels to this behavior: The adoption of Twitter by schools and school districts and their use of Twitter to announce USCs. Although Georgia schools (81%) have higher Twitter coverage than schools in Michigan (58%), city and suburban schools have higher Twitter coverage than rural and town schools in both states. This rules out the adoption of Twitter as a potential cause, leaving us with the use of Twitter to announce USCs.
There are several limitations associated with our study. First, it is a cross-sectional study. Retrospective Twitter data, up to 3,200 tweets per Twitter account as per the limit of Twitter API, was retrieved in 1 day. Twitter coverage was assumed constant over the 2 school years. We did not investigate the specific date on which a school or school district joined Twitter. If some schools or school districts joined only in 2016-17, it would not have been possible for them to make Twitter USC announcements in 2015-16. Second, human errors were possible even with 2 coders reading the tweets. Finally, Twitter use is neither uniform across time nor geography. When and where tweets are less frequent, the performance of the Twitter method may suffer. Future research can extend the analysis across more years or other states to establish a comprehensive picture on how the use of Twitter to announce USCs may vary. Future studies can also collect information on how long schools stay closed during an USC event.
In conclusion, 81% of Georgia public schools have active Twitter coverage. Through Twitter, 222 and 178 additional Georgia public schools with at least 1 USC were identified in 2015-16 and 2016-17 school years. Twitter was able to fill in a gap in OSS coverage in the suburban locale. Altogether, our study demonstrates that Twitter remains a complementary USC data source across 2 years in Georgia. If future results are consistent with our findings, Twitter has the potential to become an important supplement to OSS, creating a more robust USC database. (See discussion in Supplemental Text S3.) This would improve the ability for public health professionals to make better informed decisions. It would also help identify how well public health measures are followed during a public health emergency.
Acknowledgments
The authors thank Dr. Amra Uzicanin, Dr. Hongjiang Gao, Dr. Yenlik Zheteyeva, and Ashley Jackson, MPH, of the Community Interventions for Infection Control Unit, Division of Global Migration and Quarantine, National Center for Emerging Infectious Diseases, Centers for Disease Control and Prevention, for sharing their unplanned school closure online systematic search data. The authors are grateful to the late Dr. Toby Merlin of the Division of Preparedness and Emerging Infections for his leadership and institutional support. The authors thank Mr. Dharamendra Kumar of the University of Georgia for Twitter data retrieval.
Funding
Dr. Isaac Chun-Hai Fung acknowledges salary support from the National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention (18IPA1808820; 19IPA1908208).
Disclaimer
The findings and conclusions of this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention or the United States Government.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2020.65
Author Contributions
Authors Ahweyevu and Chukwudebe contributed equally to this work.
