Bloodstream infections are detected regularly in hospitalized children and result in substantial morbidity and mortality.Reference Raymond and Aujard1–Reference Spaulding, Watson and Dreyfus4 These infections significantly prolong hospitalization,Reference Goudie, Dynan, Brady and Rettiganti5 and mortality rates can be as high as 10%–20%.Reference Burke, Halpern, Baron and Gutierrez6-Reference Gray, Gossain and Morris8 Early effective antibiotic therapy is critical for reducing mortality.Reference Nguyen, Rivers and Abrahamian9 Subsequently, targeting antibiotics to the causative organism reduces broad-spectrum antibiotic use, limiting adverse effects, cost, and antibiotic resistance.
Rapid diagnostic platforms for positive blood cultures have enabled significant reductions in time to identification of organisms and resistance genes.Reference Avdic and Carroll10 Implementation of these platforms has reduced time to effective therapy, length of stay, and mortality in adult patients, especially when coupled with antimicrobial stewardship program (ASP) interventions.Reference Timbrook, Morton, McConeghy, Caffrey and Mylonakis11 However, relatively few pediatric studies have been conducted, and they show mixed impacts on length of stay and no mortality benefit.Reference Felsenstein, Bender, Sposto, Gentry, Takemoto and Bard12–Reference Veesenmeyer, Olson and Hersh17 Prior studies are also limited by study design; many compare only means, medians, or proportions before and after an intervention. This approach does not account for temporal changes in outcomes due to other factors,Reference Harris, Bradham, Baumgarten, Zuckerman, Fink and Perencevich18,Reference Ramsay, Brown, Hartman and Davey19 which can lead to over- or underestimation of an intervention’s effect. Furthermore, many studies have not controlled for patient characteristics, which may be important confounders, and controls or nonequivalent dependent variables have rarely been utilized. Thus, we sought to evaluate the impact of a rapid diagnostic platform in a large pediatric population, incorporating a nonequivalent dependent variable and using statistical methods that adjust for patient characteristics and temporal trends.
Methods
Study design and population
We used a quasi-experimental design to evaluate the impact of implementing the VERIGENE gram-positive blood culture test (BC-GP; Luminex, Austin, TX) with ASP support. The BC-GP is a multiplex nucleic acid test that detects 9 species and 3 genera of gram-positive bacteria, plus 3 genetic resistance determinants: mecA (Staphylococcus aureus or S. epidermidis methicillin resistance) and vanA or vanB (Enterococcus faecalis or E. faecium vancomycin resistance).
We retrospectively enrolled patients aged 0–21 years admitted to Children’s Hospital of Philadelphia (CHOP) with positive blood culture events ~1 year before BC-GP implementation (November 11, 2012, through October 31, 2013) and 1 year after BC-GP implementation (November 1, 2013, through October 31, 2014). Events were defined by a blood culture turning positive during admission, with no positive blood cultures in the prior 30 days. We excluded fungal or mycobacterial isolates, polymicrobial bacteremia (≥2 different bacterial species isolated within 48 hours of initial positive blood culture), and Gram stain results (positive vs negative) not matching the cultured organism. Patients with gram-negative bacteremia were included as a nonequivalent dependent variable,Reference Schweizer, Braun and Milstone20 which is a group similar to the intervention group (gram-positive events), expected to be affected by all factors that could impact the intervention group except the intervention. The study was approved by the CHOP Institutional Review Board.
Study procedures
Before the intervention, blood culture isolates were evaluated by conventional identification and susceptibility testing: preliminary biochemical identification ~24 hours after culture positivity (eg, catalase test for S. aureus) followed by identification and susceptibility results ~24 hours later using the VITEK 2, plus ETEST as needed (bioMérieux, Durham, NC). The S. aureus isolates were tested for methicillin resistance using a penicillin-binding protein latex agglutination assay following preliminary identification. Gram stain results were reported to the patient’s nurse or frontline clinician; other results were made available immediately in the electronic medical record (EMR). The CHOP ASP did not receive culture result notification, but the ASP pharmacist reviewed positive cultures daily (Monday–Friday). Infectious diseases consultation was available on request.
After the intervention, in addition to conventional testing, BC-GP testing was performed around-the-clock for all blood cultures with gram-positive isolates, immediately following Gram staining. Gram-stain reporting to clinicians continued, and BC-GP results were also reported to clinicians. BC-GP results were also communicated to the ASP via text message from 7:00 am through 11:00 pm daily (Monday–Sunday). BC-GP results that became available between 11:00 pm and 7:00 am were reported to the ASP the following morning. The ASP pharmacist or infectious diseases fellow on call for ASP (nights and weekends) reviewed the result and patient chart and notified clinicians if antibiotic changes were warranted. Infectious diseases consultation practices were unchanged after the intervention.
Data collection and study outcomes
Study enrollment began the day of first positive culture collection (day 0). Clinical and microbiological data were extracted from the EMR electronically and via structured chart review. Routine demographic and the following clinical characteristics were collected: International Classification of Disease, Ninth Revision (ICD-9) codes indicating complex chronic conditionsReference Feudtner, Christakis and Connell21 or immunocompromise,22 the need for intensive care (vasopressor or mechanical ventilation use days 0–2), prior adverse antibiotic reactions, admission and discharge dates, discharged alive or deceased, central venous catheter (CVC) on day 0, and infectious diseases consultation at culture positivity (defined by new or follow-up infectious diseases consultation notes the day of or following culture positivity).
The primary outcome was time to optimal antibiotic therapy, defined as the duration from culture positivity to the latter of (1) first administration of definitive antibiotic therapy (defined as the drug selected for targeted treatment of the bacteremia) or (2) last administration of any nondefinitive therapy (excluding antibiotics needed for prophylaxis or concurrent infections). Because BC-GP results are not known until after culture positivity, time to therapy was anchored on culture positivity to analyze the time frame in which BC-GP results could guide management decisions. If definitive therapy included multiple antibiotics, calculations were based on the “backbone” antibiotic (eg, β-lactam or vancomycin for prosthetic valve endocarditis). Isolates were classified as pathogens or contaminants based on the medical team’s documented assessment. For contaminants, time to optimal therapy was based on the last administration of any antibiotic, excluding those needed for prophylaxis or concurrent infections.
Secondary outcomes included time to first administration of effective antibiotic therapy (defined as any antibiotic to which the isolate was susceptible), time to first administration of definitive therapy, time to stopping vancomycin for patients who received vancomycin empirically but not for definitive therapy (defined by last vancomycin administration), length of stay (LOS) following first positive blood culture, and 30-day in-hospital all-cause mortality.
Statistical analysis
We compared pre- and postintervention outcomes, stratified by Gram stain status and individual organisms. Continuous outcomes were compared using Mann-Whitney U tests due to nonnormal distributions. Time-to-therapy outcomes were also compared with log-rank tests, censoring patients who died or were transferred after culture positivity but before definitive therapy was established.
We modeled BC-GP impact on time-to-therapy outcomes in 2 ways. First, we fit segmented Cox regression models that included variables for the pre- or postintervention period, enrollment month, and the interaction between the pre- and postintervention periods and enrollment month, to estimate the immediate impact and impact over time of implementing BC-GP testing, and to control for any confounding temporal trends. Models included patient characteristics associated with the outcome on bivariate log-rank analysis (P < .10). Proportional-hazards assumptions were assessed with Schoenfeld residuals, with time-varying covariates included to accommodate nonproportionality when needed. Patients with gram-positive and gram-negative infections were modeled separately.
Second, we used an interrupted time series (ITS) approach with segmented linear regression, modified to use censored survival data. This enabled the inclusion of patients with gram-negative infections in the model as a control, and it provided a more interpretable prediction of BC-GP impact on antibiotic outcomes. Outcomes were aggregated monthly, with monthly median estimates derived from a Kaplan-Meier curve using censored survival data for each month. Variables for the pre- and postintervention periods and enrollment month were included again to estimate the immediate impact of BC-GP testing and its impact over time. For the ITS analysis, we used the Prais-Winsten model, which incorporates adjustment for first-order autoregressive effects.
All analyses were restricted to patients who had not reached optimal therapy prior to culture positivity. Analysis of secondary time-to-therapy outcomes was further restricted to those who had not reached the corresponding end point prior to culture positivity. P < .05 was considered significant for all analyses. Statistical analyses were conducted using Stata version 15.1 software (StataCorp, College Station, TX).
Results
Study cohort
Of 678 positive blood-culture events, we excluded 75 for polymicrobial growth and 14 for Gram stain and isolate discordance. An additional 68 events were excluded for reaching optimal therapy prior to culture positivity. Of the remaining 521 events (456 unique patients), 264 events (50.7%; 191 gram-positive and 73 gram-negative) occurred before the intervention and 257 events (168 gram-positive and 89 gram-negative) occurred after the intervention. Among the gram-positive events, 36 (18.9%) that occurred before the intervention and 42 (25.0%; P = .158) that occurred after the intervention were contaminants. Also, 6 patients died and 1 patient was transferred after culture positivity but before definitive therapy was determined.
Clinical and microbiological characteristics
The median age of patients in these 521 events was 2.9 years (interquartile range, 0.3–10.1), 418 (80.2%) events involved children with >=1 complex chronic condition, and 218 (41.8%) events involved immunocompromised children. Patients with gram-positive and gram-negative infections were similar, except for more β-lactam allergies recorded among patients with gram-positive infections (47 [13.1%] vs 10 [6.2%]; P = .019) and trends toward more CVCs (109 [67.3%] vs 210 [58.5%]; P = .057) and intensive care (65 [40.1%] vs 116 [32.3%]; P = .083) among patients with gram-negative infections. Pre- and postintervention groups, stratified by Gram status, were also similar (Table 1).
Table 1. Characteristics of Children with Positive Blood Culturesa
Note. BC-GP, VERIGENE gram-positive blood culture test; IQR, interquartile range; LOS, length of stay; CVC, central venous catheter.
a Unadjusted comparison of hospitalized pediatric patients with positive blood culture events before and after BC-GP implementation, stratified by Gram stain result. P > .05 for all comparisons.
Gram-positive microbiological characteristics were similar before and after BC-GP testing was implemented. Nearly all isolates (95.5%) were organisms detected by BC-GP, and most (63.5%) were organisms for which resistance marker detection is performed. Staphylococci predominated (234 of 359 isolates, 65.2%), of which nearly half were S. aureus (n = 108, 46.2%).
Time-to-therapy outcomes
Among gram-positive patients, median time to optimal therapy decreased from 42.1 hours before the intervention to 26.0 hours after the intervention (log-rank P < .001 (Table 2 and Fig. 1). This decrease was observed among both pathogens and contaminants, but it was only statistically significant among pathogens. Among gram-negative patients, median time to optimal therapy did not significantly decrease after the intervention.
Table 2. Unadjusted Clinical Outcomes of Children With Positive Blood Culturesa
Note. BC-GP, VERIGENE gram-positive blood culture test; LOS, length of stay.
a Unadjusted analysis comparing clinical outcomes of hospitalized pediatric patients with a positive blood culture event before and after BC-GP implementation.
b P values are reported for log-rank tests (time-to-therapy outcomes), Mann-Whitney U test (LOS) or the Fisher exact test (mortality).
Fig. 1. Time to optimal antibiotic therapy for positive blood cultures before and after gram-positive blood culture test (BC-GP) implementation. Kaplan-Meier curves and log-rank analyses were used to compare time to optimal therapy among hospitalized pediatric patients with positive blood culture events.
Secondary time-to-therapy subgroups were nested within the primary analytic cohort (Supplementary Table 1 online), and characteristics were similar to those of the full cohort, with few exceptions (Supplementary Table 2 online). For patients with gram-positive infections, median time to definitive therapy and time to stopping vancomycin were significantly reduced after the intervention (Table 2). Similar to time to optimal therapy, when stratified by pathogens versus contaminants, time to stopping vancomycin was significantly lower among pathogens only. No significant postintervention decreases were observed for median time to effective therapy among patients with gram-positive infections nor any secondary time-to-therapy outcomes among patients with gram-negative infections.
Among individual gram-positive organisms, the most pronounced postintervention decreases in median time to optimal therapy, time to definitive therapy, and time to stopping vancomycin were observed for methicillin-susceptible S. aureus (MSSA; Supplementary Table 3 online). Median time to optimal therapy and time to stopping vancomycin were also significantly lower after the intervention for coagulase-negative staphylococci, particularly contaminants. Vancomycin-resistant enterococci (VRE) were only isolated once in each period; all time-to-therapy outcomes were dramatically lower after the intervention, but the limited sample size precluded robust analysis. Overall, postintervention median time to optimal therapy, time to definitive therapy, and time to stopping vancomycin were significantly lower among organisms for which BC-GP testing evaluated resistance markers. For organisms without potential resistance markers, time-to-therapy outcomes were nearly unchanged.
Secondary clinical outcomes
Excluding patients who did not survive until discharge, median LOS following culture positivity was similar before and after the intervention for all patients (Table 2). Among contaminants, there was a numeric decrease after the intervention, but this change was not statistically significant (10 vs 5.5 days; P = .312). The 30-day in-hospital all-cause mortality was low (4.0%) and was similar before and after the intervention for all patients (Table 2).
Multivariate and time trend analyses
Among patients with gram-positive infections, Cox regression demonstrated immediate associations between BC-GP implementation and faster time to optimal therapy and time to stopping vancomycin (Supplementary Table 4 online). Our ITS analysis revealed similar results (Fig. 2), predicting immediate decreases of 15.8 hours (95% CI, 1.1–30.5) in median time to optimal therapy and 23.1 hours (95% CI, 10.2–35.9) in median time to stopping vancomycin following BC-GP implementation. Both outcomes were stable over time before and after intervention in both analyses. Among patients with gram-negative infections, there was no immediate association between these outcomes and BC-GP implementation. However, in both analyses, time to optimal therapy was increasing slightly prior to BC-GP implementation (2.0-hour increase per month; 95% CI, 0.5–3.5 on ITS) and was stable thereafter.
Fig. 2. Interrupted time series analysis of time-to-therapy outcomes for gram-positive blood culture events. Time-to-therapy outcomes were aggregated by month of enrollment (1–12 preintervention and 13–24 postintervention) by calculating medians from Kaplan-Meier curves.
Among patients with gram-positive infections, ITS predicted a 28.9-hour decrease (95% CI, 9.65–48.21) in median time to definitive therapy after BC-GP implementation. This association was also observed in the Cox regression, but it did not reach statistical significance (P = .083) (Supplementary Table 4 online). No changes over time were observed with either analysis. Among patients with gram-negative infections, time to definitive therapy did not change immediately following BC-GP implementation. However, in both analyses, it was increasing prior to BC-GP implementation (2.7 hours per month; 95% CI, 0.8–4.6 on ITS) and the trend shifted significantly downward after the intervention, such that it was no longer changing.
Time to effective therapy was not significantly associated with BC-GP implementation among patients with gram-positive infections in the Cox regression. Among patients with gram-negative infections, time to effective therapy was significantly increasing prior to BC-GP implementation, and faster time to effective therapy was strongly associated with the postintervention period (Supplementary Table 4 online). The small subgroup size for time to effective therapy precluded calculation of medians for all months, and an ITS analysis could not be performed.
Several covariates were also associated with time-to-therapy end points according to our Cox regressions (Supplementary Table 4 online). Infectious disease consultation at culture positivity was the covariate most frequently associated with faster end-point achievement: faster time to definitive therapy, faster time to optimal therapy, and faster time to stopping vancomycin among patients with gram-positive infections, and faster time to effective therapy among patients with gram-negative infections. Unexpectedly, Asian race was associated with faster time to optimal therapy and faster time to definitive therapy for patients with gram-positive infections.
Discussion
In this study, BC-GP implementation coupled with real-time ASP result notification was associated with improved antibiotic management for hospitalized children with gram-positive blood culture isolates. Specifically, faster times to optimal therapy and stopping vancomycin were most convincingly associated with BC-GP implementation. Time to definitive therapy also decreased following BC-GP implementation, but an immediate association was only statistically significant in 1 of 2 models. The most pronounced improvements occurred in targeting and de-escalation of therapy for MSSA and de-escalation for coagulase-negative staphylococci.
Only 1 prior study has evaluated BC-GP in children, in which time to optimal therapy was also significantly decreased, with greatest impact among S. aureus and coagulase-negative staphylococci.Reference Felsenstein, Bender, Sposto, Gentry, Takemoto and Bard12 In addition to this outcome, we also assessed time to definitive therapy, the mechanism by which BC-GP is most likely to impact morbidity and mortality, given the superiority of β-lactams over vancomycin for MSSA definitive therapy.Reference Schweizer, Furuno and Harris23,Reference McDanel, Perencevich and Diekema24 We observed a substantial postintervention decrease, with ITS predicting a median decrease of 29 hours, although we could not conclusively attribute this to BC-GP implementation. Our study also adds an evaluation of BC-GP impact on vancomycin use, with ITS predicting nearly a full-day reduction in unnecessary vancomycin use. This finding aligns with those reported following implementation of the FilmArray blood culture identification panel in pediatric populations: Reuter et alReference Reuter, Palac and Kociolek13 noted significant decreases in vancomycin use for MSSA and in gram-positive agents for contaminants, and Veesenmeyer et alReference Veesenmeyer, Olson and Hersh17 noted significant decreases in vancomycin use for MSSA and coagulase-negative staphylococci.
Not surprisingly, we observed no significant change in time to effective therapy following BC-GP implementation. Time to effective therapy was short, and although small numeric decreases were observed, they were not statistically significant. Additionally, time to effective therapy was evaluable for relatively few patients, further limiting statistical power. These findings reflect the common practice of prescribing broad-spectrum therapy at blood culture collection when a serious infection is suspected. Additionally, opportunities to escalate therapy based on BC-GP results were limited, due to frequent empiric vancomycin use and rare vancomycin-resistant organisms. The impact on time to effective therapy is mixed in prior pediatric studies, with no studies focusing specifically on gram-positive isolates.Reference Reuter, Palac and Kociolek13–Reference Bhavsar, Dingle and Hamula16 Additionally, prior studies measured time to effective therapy from culture collection, with median or mean time to effective therapy shorter than typical times to culture positivity. This calls into question whether improvements were due to rapid diagnostics or rather, earlier empiric antibiotic administration over time. Even in adult studies, Timbrook et alReference Timbrook, Morton, McConeghy, Caffrey and Mylonakis11 noted heterogeneity in this outcome and found time to effective therapy most significantly impacted among patients with VRE. Thus, the impact of BC-GP on timely effective therapy is likely limited in pediatric populations where empiric vancomycin is frequent and VRE is rare.
We did not observe significant changes in length of stay or all-cause 30-day mortality. Similar to other pediatric populations,Reference Reuter, Palac and Kociolek13–Reference Malcolmson, Ng and Hughes15,Reference Veesenmeyer, Olson and Hersh17 mortality was low. This finding contrasts with those for adult populationsReference Leibovici, Shraga, Drucker, Konigsberger, Samra and Pitlik25 and may explain why mortality improvements seen in adults have not been observed in children. Notably, a consistent benefit of these tests in pediatric populations appears to be antibiotic de-escalation, and in particular, decreased vancomycin use. The clinical impact of this improvement has not been studied, and future studies should consider inclusion of vancomycin-related clinical outcomes, such as acute kidney injury.
A strength of this study is analyses that both account for temporal trends and adjust for potential confounders, as highlighted by our time to definitive therapy results. Median time to definitive therapy decreased by almost 27 hours after the intervention (P = .004). ITS supported this association, but Cox regression did not, suggesting that factors other than BC-GP are playing a role. Notably, infectious diseases consultation was strongly associated with faster time to definitive therapy on Cox regression. Therefore, increasing infectious diseases consultations over time, independent of BC-GP implementation, could have contributed to decreased time to definitive therapy over time.
Another strength is use of a nonequivalent dependent variable.Reference Schweizer, Braun and Milstone20 The immediate changes in time to optimal therapy and stopping vancomycin postintervention were observed among gram-positive but not patients with gram-negative infections. This is additional evidence that these improvements are associated with BC-GP implementation, rather than other potential factors, such as ASP efforts to reduce broad-spectrum antibiotic use. Furthermore, patients with gram-negative infections were included in the ITS model as a control, so results for gram-positive patients were adjusted for changes observed in patients with gram-negative infections.
Despite these strengths, our study has several limitations. We were unable to assess the impact of BC-GP testing alone because it was coupled with ASP notification to facilitate interpretation and uptake of results. This sometimes resulted in earlier ASP awareness of positive blood cultures, which may have had an impact on antibiotic management independent of BC-GP. Transfers were not excluded, so microbiological data from outside hospitals may have been available earlier and could have influenced antibiotic management. Use of the penicillin-binding protein assay before the intervention may have decreased the impact of BC-GP mecA testing for S. aureus. However, whether this assay was consistently effective in prompting de-escalation remains unclear; anecdotally, providers did not always understand the result, and it was not coupled with ASP notification. Finally, excluding patients who reached optimal therapy prior to culture positivity does not capture potential effects of anticipating faster results with BC-GP (eg, withholding antibiotics in a stable patient for whom contamination is suspected). Conversely, if BC-GP has no impact in these excluded patients, its overall impact may be less significant.
In conclusion, with the use of robust analytic methods, we found that BC-GP implementation with ASP result notification was associated with improved antibiotic management for hospitalized children with gram-positive blood culture isolates. In particular, BC-GP facilitated decreased use of vancomycin and faster de-escalation of therapy. BC-GP could be an important tool for microbiology laboratories and ASPs in ongoing efforts to optimize antibiotic management for children with positive blood cultures.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2020.191
Acknowledgments
None.
Financial support
This study was supported in part by the CDC Cooperative Agreement from the Epicenters for the Prevention of Healthcare Associated Infections (FOA no. CK16-004 to E.L. and J.S.G.) and by the NIH-NICHD Pediatric Pharmacoepidemiology Training Program (grant no. T32 HD064567-04 to A.C.T.).
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
