METHODS

Analytical Approach

To identify health events associated with the CO poisoning outbreak, we ran detection algorithms on selected time series derived from all 3 data sources (ED, enhanced EMS, and SFD data) for intervals including the week following the event. We used the Holt-Winters forecasting algorithm recently adapted for syndromic surveillance,Reference Elbert and Burkom8 which is a generalization of simple exponential smoothing. This approach treats the time series of interest as a level term modified by linear trend and cyclic seasonality factors. The generalized smoothing enables continuous updating of all 3 components to accommodate the short-term effects often seen in syndromic series. The level m_t, trend b_t, and seasonality c_t are updated by using the following equations:

where α, β, and γ denote constant smoothing coefficients. A seasonality length s of 7 was chosen to represent weekly patterns common in biosurveillance data.

The forecast function for k-step ahead forecast is as follows:

The detection statistic a_t is derived from this forecast by scaling the residual with an estimate of baseline variance σ:

Residuals can be assumed to be standard Gaussian. We used next-day forecasts and α = .25, β = 0.01, γ = 0.1 for the algorithm realization.

Spatial and Spatiotemporal Cluster Detection

We analyzed the patient location field data in the EMS and ED data sets for their usefulness in cluster detection. The SFD data record fields did not include any geographic information, so spatial analysis was not possible. In the enhanced EMS data set, zip codes were missing for more than 70% of cases. Geographically coded address information was available for all but 2.1% of patients. However, maps of the geographically coded squares suggested substantial coverage gaps, especially in the eastern part of the county. In the ED data, less than 1% of cases were missing a zip code with good county coverage. Therefore, we attempted cluster detection in the ED data at the zip code level.

We were interested in identifying the geographic areas that were most heavily affected by the CO poisoning event. Scan statistics, popularized in health surveillance by the free software SaTScanReference Kulldorff9 (Boston, MA), indicate the location and extent of a potential outbreak while controlling for expected clustering patterns. In considering a large search space of possible cluster locations and sizes, this approach also attempts to control the number of cluster alarms resulting from multiple testing.

In many scan statistics applications, the test statistic is a function of observed and expected values inside and outside each candidate cluster. Thus, an accurate estimate of the expected spatial distribution is necessary for determination of relevant clusters at reasonable false alarm rates.Reference Xing, Burkom, Moniz, Edgerton, Leuze and Tokars10 For the implementation applied in this study, we tried 3 methods for estimating the expected values underlying the scan statistics: (1) 2004 census populations of the subregions; (2) mean baseline data distributions for each subregion; and (3) mean baseline data distributions for each subregion, stratified by weekday or weekend/holiday.

Neither the census population distribution nor flat baseline averages gave satisfactory results in test runs. When we used the 2004 zip code census populations, the scan statistics produced many statistically significant clusters with only 1 zip code before and after the windstorm. Use of the mean baseline averages gave excessive clustering on weekend days, indicating changed data distributions likely resulting from regionally varying hospital schedules and use patterns. The weekend/weekday-stratified baseline averages had yielded reasonable background clustering rates in outpatient clinic data sets in which weekly patterns were more obvious,Reference Xing, Burkom, Moniz, Edgerton, Leuze and Tokars10 and they also gave stable results for these ED visit counts. Therefore, we based the cluster detection runs on this stratified estimation method.

In applying scan statistics to the zip code–based ED data, we considered purely spatial and space-time scan statistics for the detection of localized clusters. For each day or group of most recent days, we found the candidate clusters with highest test statistic values over all cylinders centered at King County zip code centroids. To avoid including more than half the covered population in any single cluster, we chose to limit their spatial size to a 10-km distance from the central cluster point. As described by Xing et al,Reference Xing, Burkom, Moniz, Edgerton, Leuze and Tokars10 we used Monte Carlo simulation runs to decide the significance of the candidate clusters with highest statistics. To limit the number of required simulation runs and to derive significance values on a continuous scale, we derived P values for each candidate cluster based on the Gumbel, or extreme value, distribution.Reference Abrams, Kulldorff and Kleinman11 We sought purely spatial and spatiotemporal clusters by using both single-day cylinders, limiting the data test interval to 1 day and cylinders considering test intervals of up to 7 days back. Several P value thresholds were tested for the CO poisoning category, and after analyzing the runs for all days of calendar 2006, we considered clusters with Gumbel P values less than 0.01 to be statistically significant.

RESULTS

During the period December 15-24, 2006, our syndromic surveillance system captured 16 277 ED visits. Of the patients who sought emergency care, 164 were classified as having a CO-consistent syndrome. The volume of CO-consistent syndrome visits was highest on the first day after the storm (39.0%) and decreased thereafter (Figure 1). The majority (57.4%) of the 164 patients with a CO-consistent syndrome provided a zip code that mapped to South, Southeast, or Southwest King County, and 45.1% went to a single hospital. Whereas the overall proportion of ED visits among males and females was equal during this time frame, 67.1% of patients with a CO-consistent syndrome were female. The age of ED patients with a CO-consistent syndrome ranged from younger than 1 year to older than 90 years; 34.1% were aged 18 to 44 years (Table). Of all ED patients, 19% were younger than 18 years, but this percentage was much higher, at 43.9%, for the patients with a CO-consistent syndrome visit.

FIGURE 1 Frequency of patients presenting to the emergency departments with a carbon monoxide–consistent illness as identified by syndromic surveillance, December 15-24, 2006

TABLE 1 Demographic Characteristics of 164 Patients Who Went to Emergency Departments With a Carbon Monoxide–Related Illness as Identified by Syndromic Surveillance, December 15-24, 2006.

Of the 241 CO cases confirmed by chart review, 190 (78.8%) were identified in the syndromic surveillance data as exact matches to CO-consistent syndrome visit cases; 18 (7.5%) were likely matches; 14 (5.8%) were possible matches; and 19 (7.9%) could not be matched on the basis of the available fields. The 241 confirmed cases went to 14 of 19 EDs that provided syndromic surveillance data during the period of analysis; the unmatched records came from 5 of the 14 EDs.

We classified chief complaints and diagnoses collected by the syndromic surveillance system to identify the most common reasons patients sought care. Excluding the cases identified by chart review only that could not be matched to the syndromic records, 62.6% of the patients (139/222) had a chief complaint or diagnosis containing one or more of our CO-consistent syndrome search terms. Among other chief complaint classifications we examined, headache (17/222 [7.7%]) was the most commonly coded one, followed by nausea (15/222 [6.8%]), dizziness (12/222 [5.4%]), vomiting (10/222 [4.5%]), loss of consciousness (6/222 [2.7%]), and shortness of breath (6/222 [2.7%]). Of records with an exact match, 62.6% went to the ED with a CO-consistent syndrome. Among the 164 patients with a CO-consistent syndrome, 119 were an exact match to a confirmed case (72.6%), 11 were a likely match (6.7%) using the criteria described in the “Methods” section, and 9 were a possible match (5.5%).

Figure 2 represents daily visits for the chosen CO-consistent categories from each data set. The results of running the Holt-Winters algorithms supported our variable selection and showed significant values of test statistics for “Inhalation Gas Smoke” (enhanced EMS data) and “Breathing Problem (Over 12 Years)” (SFD data) on the day of the event (December 15, 2006). The ED chief complaints related to CO poisoning were strongly correlated (ρ = 0.7) with “Inhalation Gas Smoke” cases from the enhanced EMS data. Both series showed a dramatic increase on the day after the windstorm and no seasonal patterns.

FIGURE 2 Daily time series of carbon monoxide (CO)-consistent patient records from emergency department (ED) visits and from routinely collected and enhanced emergency medical services (EMS) calls for 3 months, ending the week after the windstorm. SFD indicates Seattle Fire Department.

By using ED data, we identified significant clusters using the CO-consistent syndrome grouping. Figure 3 shows that the first significant cluster occurred on December 15, 2006, on the day of the windstorm. The single blue-shaded zip code shows the location of this cluster, which was composed of only 2 cases (compared with 0.03 expected cases). The scan statistics flagged the CO-consistent syndrome event a day earlier than the time series detection algorithm based on county-wide CO-related counts. Figure 4 shows the progression of the outbreak over space and time; the top image displays results of the single-day cluster searches for CO-consistent syndrome by day for the period December 15 to 19, and the bottom image shows the progression based on ED visit counts looking back up to 7 days.

FIGURE 3 Zip code map of Western King County (Washington) showing an initial significant cluster (blue) of emergency department (ED) visits found in carbon monoxide–consistent syndrome data on December 14, 2006, the night of the windstorm.

FIGURE 4 Day-by-day progression of carbon monoxide–consistent clusters in Western King County on days 2-6 of the December 2006 windstorm, based on single-day counts of emergency department (ED) visits (top) and based on counts of ED visits looking back up to 7 days (bottom). Each color (red, dark blue, light blue, green) represents a different cluster.

We were interested in evaluating whether loss of power led to an increase in patients going to EDs with GI illness. Most of the confirmed CO poisoning cases appeared in zip codes where a larger number of homes were without power in the week after the storm. For seeking clusters of food poisoning, none of the ED chief complaints contained strings suggestive of food poisoning or even the word “food,” so we grouped the data by the following: (1) GI syndrome; (2) subsyndrome for diarrhea; and (3) diarrhea subsyndrome for patients aged 18 years or older, because inclusion of children's records made the syndromic time series much “noisier.” In the 2 weeks following the windstorm, none of the GI-based groupings in the ED data yielded any significant localization. This finding suggests that if GI-related illness resulting from food poisoning occurred, the cases did not cluster in space.

COMMENT

After one of the most severe windstorms to ever hit the Pacific Northwest, unintentional CO poisoning was made reportable to Public Health in King County, and ED medical charts were reviewed to describe the full extent of the outbreak.Reference Gulati, Kwan-Gett and Hampson2 By virtue of having access to ED data in near real time as part of our ongoing syndromic surveillance system, we also monitored for CO-consistent syndromic ED visits and attempted to identify whether there was an increase in GI-related illnesses following the extensive power outages in our region. However, at the time of the outbreak, we did not have a validated definition of CO poisoning, and, therefore, we were uncertain how well our system was capturing true cases. The availability of ED-based syndromic data in combination with “gold standard” chart reviews provided us with a unique opportunity to retrospectively describe the performance of syndromic data sources for near real time “situational awareness.” Specifically, our goals were to estimate the percentage of patients whose illnesses were correctly classified as true CO cases and describe morbidity associated with the outbreak.

Our retrospective analysis revealed that patients with a CO-consistent syndrome identified by the syndromic surveillance system were demographically similar to cases of CO poisoning identified by chart review with regard to sex and age group. The majority of patients identified by the syndromic surveillance system lived in South King County, which has a high concentration of immigrant families. Although information about race and ethnicity is not collected in the PHSKC syndromic surveillance system, we know from the chart reviews that immigrant households were disproportionately affected by CO poisoning following the outbreak and were more likely than nonimmigrant households to use charcoal devices as alternative sources of heat.Reference Gulati, Kwan-Gett and Hampson2

By using our syndromic surveillance ED data, we were able to definitively identify 78.8% (190/241) of the confirmed cases, of which 62.6% had a CO-consistent syndrome in our syndromic data set classification. The majority (72.6%) of patients with a CO-consistent syndrome in our syndromic surveillance system were an exact match for a confirmed case, but 15.2% of ED patients (n = 25) with a CO-consistent syndrome could not be matched to a confirmed case. Based on the available data, we were unable to determine whether these 25 unmatched records were true cases of CO poisoning that were not diagnosed or reported by hospitals and, therefore, not captured by the chart review process; whether the patients came into the ED with a CO-consistent syndrome but were deemed not to have CO poisoning; or whether there were data quality issues that prevented matching of records. Regardless, knowing the range within which the system correctly identified true cases will be valuable for describing future outbreaks of CO poisoning using syndromic data.

The relative distribution of patients with a CO-consistent syndrome over time was similar between syndromic surveillance and chart review—65% of cases identified by chart review vs 66% of patients with CO-consistent syndrome identified by syndromic surveillance sought care within the first 2 days of the storm.

Our experience highlights some of the challenges of classifying chief complaints and diagnoses into syndromic categories. Symptoms of CO poisoning are nonspecific and include common symptoms such as headache, nausea, dizziness, and fatigue.12Reference Varon, Marik, Fromm and Gueler13 Although most patients went to EDs with a recorded chief complaint that specifically mentioned CO poisoning, 37.4% had nonspecific complaints. (Based on the available data, we were unable to identify which patients were transported by EMS personnel, who would have been likely to identify exposure to CO before arrival at the ED.) A syndromic classification that included CO poisoning as well as common sources of CO such as propane, charcoal, and kerosene identified the majority of patients. Including terms such as headache, nausea, and dizziness could have increased the sensitivity of the classification but would have resulted in an unacceptably large loss in specificity. Searching the chief complaint and diagnoses fields together proved to be the optimal method for identifying true cases, rather than relying exclusively on chief complaints.

We were interested in examining whether data sources other than EDs could be beneficial for future surveillance for CO poisoning. The ED chief complaints related to CO poisoning were strongly correlated with “Inhalation Gas Smoke” cases from the enhanced EMS data. We also found that most confirmed cases of CO poisoning appeared in zip codes where many homes were affected by loss of power. Weak temporal GI signals were detected in the ED data but not in the EMS data, but these signals might be explained by seasonality increases not completely filtered out by the detection algorithm during the weeks of the storm. These findings suggest that enhanced EMS data could be useful for monitoring a CO poisoning event if they were available to the health department promptly in near real time.

A general remark about the usefulness of the EMS data are that the clinical fields contain detail for respiratory or neurological complaints, but little or no detail for GI problems. Thus, the usefulness of these data for outbreak detection and tracking may vary greatly depending on the nature of patient symptoms relative to on-site EMS clinical description. The usefulness of the data might also differ in environments where other syndrome selection criteria are used. Of note, in this analysis “Breathing Problem (Over 12 Years)” was selected as most representative of the CO poisoning event in the SFD data set. While the selection of this patient type code was based on correlation analyses, it inherently limited the ability to detect CO-related illness in younger children.

Another question that we tried to answer in the days following the storm was whether the extended loss of power was accompanied by an increase in GI illness complaints, potentially as a result of consumption of spoiled foods, as has previously been described.Reference Marx, Rodriguez and Greenko14 Related algorithmic analysis was not conclusive. Purely temporal methods alerted on some of the days during the week after the outbreak for groupings related to GI complaints. These alerts may have been indicative of a seasonal increase not completely filtered out by the adaptive Holt-Winters algorithm. As noted in the “Results” section, no significant spatial or spatiotemporal clustering was found for any of the syndrome or subsyndrome groupings tested. One limitation of this ecological data analysis is that we were unable to determine whether patients who sought care for GI complaints had loss of power or had consumed spoiled food products. It is important to note that the unsafe heating practices causing CO poisoning could have occurred immediately after the storm caused power outages, while food poisoning would have required a day or more without refrigeration.

Our results suggest that the scan statistic method can provide greater timeliness and localization compared with time series methods if the increase in the total case burden is gradual. We found a small CO-consistent syndrome cluster composed of 2 cases the day of the windstorm. Such a small cluster might typically be ignored by a health monitor reviewing the algorithm output. (There were 7 other such small, significant CO-consistent syndrome clusters observed in the first 11 months of daily runs for 2006.) However, because of the alert status following the windstorm, the scan statistic cluster could have triggered additional public health measures (eg, public education campaigns targeting the geographic areas at highest risk), had we been using cluster detection methods prospectively.

We applied spatial (single-day clusters) and spatiotemporal (up to 7-day clusters) analyses for cluster detection. The relative usefulness of the top and bottom images in Figure 4 depends on epidemiology and data quality, and the health monitor must keep in mind that the effect of a very significant cluster will be informative or cause bias, depending on the monitoring objectives and the linkage of cases over succeeding days. If the health monitor is interested in examining the geographic distribution of newly reported cases only, then the view based on single-day cylinders may be preferred. However, if cases on succeeding days are likely to be linked or if there are delays in capturing the data, the health monitor may prefer the variable cylinders. An experienced monitor may want both views.

To our knowledge, this is the first published study to systematically compare the performance of syndromic surveillance data sources with a gold standard for describing a CO poisoning outbreak. Our experience suggests that in an emergency, traditional reporting of cases to health departments by phone or fax may be delayed when health care institutions are responding to a large influx of patients. Often, to supplement baseline reporting, health departments must quickly establish new enhanced data collection systems to support surveillance during an emergency, which can be labor-intensive and time-consuming. The ability of a health department to access and analyze automated data to support its surveillance efforts can facilitate more rapid analysis of data, situational awareness, and public health response measures. Data from this and other studies1516Reference Jhung, Shehab and Rohr-Allegrini17 lend support to the validity of using automated systems to augment traditional surveillance for CO poisoning.