Prospective audit and feedback (PAF) is a core strategy used in antimicrobial stewardship programs (ASPs). 1 , Reference Dellit, Owens and McGowan 2 In general, PAF is defined as postprescription review of antimicrobials by a member of an ASP team who provides feedback to the patient care providers regarding opportunities for antimicrobial optimization.Reference Chung, Wu, Yeo, Chan and Hsu 3 The patient care team then decides whether to accept and implement the recommended changes. The PAF strategy has been successfully deployed in the pediatric setting and has been shown to reduce antimicrobial utilization and improve patient outcomes.Reference Hersh, De Lurgio and Thurm 4 – Reference Di Pentima, Chan and Hossain 8 Also, PAF programs have been recommended as a way to preserve prescribing autonomy because acceptance of PAF recommendations (PAFR) is voluntary. However, the roadmap to PAF success has not been clearly illustrated, and predictors of provider disagreement with ASP recommendations have not been fully elucidated.Reference Chung, Wu, Yeo, Chan and Hsu 3
Given the time-intensive nature of individual chart review and provider communication, an understanding of when and why providers disagree with PAFR is critically important to the efficiency and success of PAF. Prior studies in pediatric populations have identified the spectrum of antimicrobial activity, infectious problem, primary service, recommendation type, and role of the person receiving the PAFR as predictors of disagreement.Reference Di Pentima, Chan and Hossain 8 , Reference Goldman, Lee and Hersh 9 The generalizability of these findings are limited by the exclusion of specific antimicrobials in PAF, including antifungals and antivirals, unique programmatic factors (eg, methods of communication, parallel stewardship activities), and the inclusion of a limited number of prespecified variables in models used to predict PAFR disagreement. Therefore, we sought to advance the current knowledge base and to examine whether additional patient-level, antimicrobial, programmatic, and provider-level factors might also predict PAFR disagreement at our institution.
METHODS
Study Setting
Lucile Packard Children’s Hospital Stanford (LPCHS) is a 302-bed freestanding children’s hospital in Palo Alto, California. The level IV neonatal intensive care unit (NICU) has 40 beds; the level II intermediate care nursery (ICN) has 20 beds; the cardiovascular intensive care unit (CVICU) has 32 beds; and the pediatric intensive care unit (PICU) has 36 beds. In addition to stem cell transplant, the hospital has high-volume heart, kidney, liver, multivisceral, and lung transplant programs. Also, 2 community hospital settings with pediatric services are licensed and staffed by our institution: a 30-bed general pediatric unit and a 6-bed level II NICU.
Prospective Audit and Feedback Program
The PAF program at LPCHS began on March 30, 2015, with review of injectable antimicrobial orders active for ≥48 hours in the PICU. The program subsequently expanded to the NICU and ICN on July 20, 2015, the CVICU on December 7, 2015, the cardiology service on February 8, 2016, and the hematology-oncology service on May 31, 2016. These units were selected due to their high rates of antimicrobial utilization and presumed opportunities for improvement. Prospective audit and feedback were initially performed by the ASP medical director and ASP pharmacist 3 days per week before the addition of a second ASP pharmacist on September 13, 2016, allowed for the expansion of PAF to 5 days per week. The PAF program subsequently expanded to all inpatient pediatric units at LPCHS on January 17, 2017. Recommendations were generally discussed with the unit-based pharmacist (UBP), who subsequently communicated any recommendations directly with the patient care team (Figure 1). When the UBP was not available, recommendations were communicated directly to the patient care team by an ASP representative. When the infectious disease (ID) service was consulting, PAFR were communicated with the ID team rather than the primary service. The ASP team also tracked and documented whether the care team adhered to the recommendations within 48 hours of audit by noting whether the recommendations were followed, whether they were not followed (with a reason provided), or whether an alternative approach was agreed upon between the ASP and care team. In addition to the verbal communication, all PAFR, including whether the team did or did not follow the recommendation, were documented in the electronic medical record beginning June 2, 2016. In addition to PAF, our hospital has a limited restricted formulary of antimicrobials (ie, cidofovir, colistin, daptomycin, linezolid, micafungin, posaconazole, and tigecycline) that require approval from the ID team prior to being dispensed from the pharmacy.
FIGURE 1 Prospective audit and feedback process. NOTE. ASP, antimicrobial stewardship program; EMR, electronic medical record.
Study Design
All patients admitted to LPCHS with an audit between March 30, 2015, and April 17, 2017, were included in the study. Because patients could be on multiple antimicrobial regimens during their hospital stay and because multiple audits and recommendations can occur for a given antimicrobial, the unit of measure was each recommendation. The PAFRs not followed due to reason other than disagree (eg, patient was discharged or patient was transferred to a unit without active audit and feedback) were excluded. The Stanford University School of Medicine Institutional Review Board approved the study protocol.
Data were obtained from the LPCHS ASP team’s internal PAF tracking system and the LPCHS enterprise data warehouse. Potential predictors for PAFR disagreement were categorized as patient level, antimicrobial, programmatic, and provider level. Patient-level data included patient demographics, markers of infection (eg, procalcitonin, C-reactive protein, or fever), presence of mechanical ventilation, presence of ventricular assist device (VAD) or extracorporeal membrane oxygenation (ECMO) cannulation, and receipt of solid organ or stem-cell transplant. The documented infectious problem for antimicrobial therapy (online Appendix A), total hospital length of stay (LOS), LOS in an intensive care unit, and time from admission to audit were also captured. Antimicrobials were categorized based on their spectrum of activity (online Appendix B). Programmatic factors included PAFR type (online Appendix C), communicator of PAFR to patient care team (online Appendix D), and the time between PAF program commencement in a given unit and the date of audit. Provider-level characteristics included the medical service, attending gender, and number of years the attending physician had been in practice since completing medical training. The PAFR outcome data included whether the recommendation was followed, and if not, the reason. Recommendations that resulted in an alternative approach agreed upon between the patient care team and the ASP were considered to have been followed.
The primary outcome of interest for this study was disagreement with PAFR. Potential predictors of disagreement were assessed, including patient-level, antimicrobial, programmatic, and provider-level factors. Categorical predictors were compared with the outcome variable using the Pearson χ2 test of independence. A logistic regression model was used to estimate the probability that a recommendation was not followed based on the additive effects of a collection of categorical and numeric variables that were deemed potentially relevant either clinically or operationally. The relationship between numeric variables and the outcome variable was explored for nonlinearity. In cases in which a nonlinear relationship was discovered, a categorical variable was created from the numerical variable that would allow the model to fit the nonlinear aspect of the relationship. Initially, a backward stepwise selection was used where variables were dropped if the P value was<.05, where the P value was calculated from an F-distribution based on the likelihood-ratio test. After obtaining the initial model, variables were added and removed iteratively to find alternative models. Models were ranked by Akaike information criterion (AIC) to possibly account for other variables that could be considered more parsimonious. The new models that were developed all had comparable AIC values but had added variables that were not statistically significant. The final model was the best fitting model. Statistical analyses were conducted using R statistical package, version 3.2 (R Foundation for Statistical Computing, Vienna, Austria).
RESULTS
Of the 4,727 antimicrobial audits performed during the 2-year study period, 1,323 (28%) resulted in a PAFR (Figure 2). Vancomycin and piperacillin-tazobactam had the greatest number of PAFRs, accounting for 279 of 304 broad-spectrum gram-positive antibiotics (92%) and 140 of 519 broad-spectrum gram-negative antibiotics (27%), respectively. Of 1,323 PAFR, 525 (~40%) were for the infectious problem of suspected or proven sepsis. The most common type of recommendation was to stop the antimicrobial (46%) and the majority of recommendations (85%) were communicated to a UBP.
FIGURE 2 Total number of audits performed during the study period and study cohort identification based on exclusion of recommendations not followed for reasons other than disagreement.
Of the 1,323 PAFR, 1,046 were followed (including 34 recommendations with an alternative approach agreed upon), resulting in a 79% acceptance rate. After exclusion of PAFR that were not followed for a reason other than disagree (n=90), there were 187 PAFR with a recommendation not followed. Therefore, the incidence of PAFR disagreement was 15%. Univariate comparison identified the following statistically significant patient-level variables associated with PAFR disagreement: patient age, infectious problem, medical service, presence of central line, days from admission to audit, total intensive care unit length of stay, total length of stay, antimicrobial type, and recommendation type (Table 1).
TABLE 1 Patient, Antimicrobial, and Programmatic Characteristics at Time of Prospective Audit and Feedback Recommendations (PAFR)
NOTE. CRP, C-reactive protein; ECMO, extracorporeal membrane oxygenation; IAI, intra-abdominal infection; ICU, intensive care unit; ID, infectious diseases; IV, intravenous; LOS, length of stay; PAFR, prospective audit and feedback recommendation; PO, enteral; SD, standard deviation; UTI, urinary tract infection; VAD, ventricular assist device.
a Univariate analysis identified a statistically significant difference between groups (P<.05).
In the adjusted analysis, the following patient-level, programmatic, and provider-level variables were statistically significant predictors of PAFR disagreement: recommendation type, infectious problem, time from admission to audit date, medical service, and years of attending experience (Table 2). Compared to bacteremia, providers were 5- to 6-fold more likely to disagree with PAFR pertaining to intra-abdominal infection, febrile neutropenia, or skin and soft-tissue infection. Disagreement with PAFR was more likely when PAF was performed 31–90 days into a patient’s hospital admission compared to the first 30 days. The PAFR were more likely to be followed if the recommendation type was to clarify the antimicrobial plan or to optimize the antimicrobial dose or frequency compared to stopping the audited antimicrobial. The PICU had a 2.7-fold higher probability of not following PAFR compared to the NICU. For every year of experience following completion of postgraduate training, attending providers were 2.4% less likely to follow PAFR.
TABLE 2 Multivariate Logistic Regression Model of Predictors for PAFR Nonadherence 24
NOTE. Heme-onc, hematology-oncology; IAI, intra-abdominal infection; IV, intravenous; LOS, length of stay; PAFR, prospective audit and feedback recommendation; PO, enteral; UTI, urinary tract infection.
DISCUSSION
In the face of widespread antimicrobial utilization, emerging resistance, increasing regulatory requirements, and limited healthcare resources, an understanding of how to maximize both the efficiency and the impact of ASP strategies is critically important. Prospective audit and feedback have been shown to affect positive changes in antimicrobial utilization; however, to maximize the success of this strategy, the drivers of provider acceptance of recommendations must be understood. To our knowledge, this is the most comprehensive study evaluating the predictors of PAFR disagreement, including patient-level, antimicrobial, programmatic, and provider-level factors. Building on previously identified predictors of disagreement, we found that several additional factors were associated with provider disagreement with PAFRs at our hospital.
There is no standard approach to the PAF process, and the literature reveals a variety of strategies.Reference Chung, Wu, Yeo, Chan and Hsu 3 , Reference Newland, Stach and De Lurgio 5 , Reference Willis, Gillon, Xu, Slaughter and Di Pentima 7 , Reference Goldman, Lee and Hersh 9 , Reference Lee, Goldman and Yu 10 Unique aspects of our PAF program include the auditing of all injectable medications, communication of PAFR to the UBP in most cases, and documentation of PAFR in the electronic medical record. Despite these differences, our analysis reveals some remarkable similarities across pediatric PAF programs. As has been reported elsewhere, our recommendations were most commonly made for broad-spectrum antibiotics such as piperacillin/tazobactam and vancomycin.Reference Di Pentima, Chan and Hossain 8 Similar to prior studies, we also found that recommendations were most commonly made for patients with suspected or proven sepsis, and the most common recommendation was to stop the antimicrobial.Reference Di Pentima, Chan and Hossain 8 , Reference Goldman, Lee and Hersh 9 Given that this finding has been reported in other pediatric studies, presumed and proven sepsis should be areas of focus for future stewardship research. We found some interesting differences between our study results and prior studies. In our study, nearly 30% of our audits resulted in a recommendation, which is significantly higher than previous reports of a 16%–19% recommendation rate.Reference Newland, Stach and De Lurgio 5 , Reference Goldman, Lee and Hersh 9 , Reference Lee, Goldman and Yu 10 Potential explanations for this finding could include our review of all, rather than select, antimicrobials or clinical scenarios (eg, pathogen–drug mismatch, redundant therapy), worse overall antimicrobial prescribing practices at our institution, or a more aggressive approach to PAF by our ASP team. Our rate of disagreement of 15% was also slightly lower than previous reports of 18%–23%; however, the factors accounting for various rates of disagreement across institutions are likely multifactorial.Reference Newland, Stach and De Lurgio 5 , Reference Willis, Gillon, Xu, Slaughter and Di Pentima 7 , Reference Goldman, Lee and Hersh 9 , Reference Lee, Goldman and Yu 10
Our study adds to the literature on the types of patient conditions that are associated with provider disagreement with PAFR. One prior study showed that disagreement with PAFR was more common for specific patient conditions, including community-acquired pneumonia, suspected sepsis, febrile neutropenia, gastrointestinal disease, surgical infections, and diseases of the ear, nose, and throat.Reference Goldman, Lee and Hersh 9 Similarly, our study identified the presence of febrile neutropenia, gastrointestinal, and skin and soft-tissue infections as predictors of provider disagreement with PAFR. Given the existence of evidence-based consensus guidelines for the management of these infections, the degree of disagreement between the patient care team and ASPs is particularly interesting.Reference Solomkin, Mazuski and Bradley 11 , Reference Lehrnbecher, Robinson and Fisher 12 Our institution implemented a clinical practice guideline for the treatment of febrile neutropenia in June 2015, before the PAF program was initiated in the hematology-oncology unit. Although this guideline served as the foundation of ASP PAFR for patients with febrile neutropenia, we found significant provider disagreement with ASP recommendations. Thus, establishing institutional guidelines for an infectious problem, in isolation, may not resolve potential disagreement between the care team and the ASP. Additional opportunities to improve antimicrobial utilization may exist, and prospective monitoring of adherence to institutional guidelines may be essential to their success as stewardship interventions. The involvement of the ASP team in the formulation of institutional treatment guidelines is also essential if there is to be consensus on the approach to antimicrobial therapy.
We examined the relationship between several characteristics of a patient’s hospitalization and provider disagreement with PAFR. In our study, providers were twice as likely to disagree with recommendations for patients who had been hospitalized 31–90 days at the time of PAF compared to patients who had been hospitalized for ≤30 days. One potential explanation for this finding may be that patients with prolonged hospitalizations face unique infectious issues. This finding should be examined in future PAF research, as it may represent an important opportunity for ASPs to optimize appropriate antimicrobial prescribing in patients with extended hospital stays or who are receiving prolonged course of antibiotics (eg, endocarditis, osteomyelitis, meningitis) as inpatients. Given this finding, it may be helpful for ASPs to recommend formal ID consultations for patients with longer lengths of stay or prolonged antimicrobial courses. Because our program implemented PAF to the individual units on a rolling basis, we evaluated the time between PAF program implementation on a given unit to the time of a given audit. We were reassured to find that this was not a statistically significant predictor of PAFR disagreement, suggesting that the responsiveness of care teams to PAFR did not wane over time.
The only programmatic predictor of PAFR disagreement identified by our study was the recommendation type; providers were significantly less likely to agree with recommendations to stop antimicrobials than to clarify the indication or optimize the dosing. Similarly, prior studies have shown a willingness of providers to modify dosing of antimicrobials as recommended by the ASP.Reference Willis, Gillon, Xu, Slaughter and Di Pentima 7 Given that our program’s most common recommendation was to stop the antimicrobial, the finding that providers are less likely to follow this recommendation is concerning and merits further examination. Qualitative research that explores the decision process providers use to decide whether to accept or disagree with PAFR may clarify this finding.
Our study is the first to identify an association between the years of attending experience and PAFR disagreement. This finding may reflect the relatively recent national prioritization of antimicrobial stewardship and associated implementation of formal programs in hospitals in the last decade.Reference Hersh, De Lurgio and Thurm 4 As a result, many medical education and doctor-in-training programs have evolved to include principles of antimicrobial stewardship. Clinicians who have trained at hospitals with formal ASP programs may respond more favorably to interventions supporting the judicious use of antimicrobials. On the other hand, physicians who trained prior to the antimicrobial stewardship era may be less inclined to follow PAFR. This finding underscores the need for incorporating antimicrobial stewardship into continuing education strategies for all physicians.Reference Papoutsi, Mattick, Pearson, Brennan, Briscoe and Wong 13 , Reference Wong, Brennan, Mattick, Pearson, Briscoe and Papousi 14
Our study results indicated that PAFR for audits performed on patients in the PICU were less likely to be followed compared to PAFR for audits performed on patients in the NICU. This finding directly contrasts with that of another single-center study that identified NICU and hematology-oncology services as more likely to disagree with PAFR compared to hospitalist service but found no difference with the PICU.Reference Goldman, Lee and Hersh 9 These conflicting findings suggest that there may be important cultural differences in the reception of PAFR across units in different hospitals or that these units may have unique patient populations that are not comparable across institutions. These differences underscore the need for PAF programs to tailor their interventions to the specific needs and intensive care settings of their institution.Reference De Waele, Schouten and Dimopoulos 15 – Reference Nzegwu, Rychalsky and Nallu 17 Depending on the extent and maturity of additional ASP strategies (eg, antimicrobial restriction, clinical practice guidelines), there may be more or fewer opportunities to impact prescribing by PAF.Reference Nzegwu, Rychalsky and Nallu 17 – Reference de Araujo, da Silva and Diegues 21 Literature supporting ASP interventions in critically ill neonates and children are especially limited, and extrapolation of published PAF success in a specific institution’s ICU may be difficult due to differences in patient populations served and institutional microorganism resistance patterns.Reference Araujo da Silva, Albernaz de Almeida Dias and Marques 22 Further exploration of the factors impacting the success of stewardship in critically ill children are needed. Interestingly, we did not find that any specific marker of infection (eg, procalcitonin, CRP, fever), antimicrobial, or severity of illness (eg, LOS, ICU LOS, need for mechanical ventilation) impacted adherence to PAFR. To our knowledge, this is the first study to evaluate PAFR disagreement based on these clinical factors associated with infection.
Our study has several limitations. The generalizability of our study is limited by the unique design of PAF at our hospital, where PAFR are typically communicated to the UBP. Although not statistically significant in our final model, some of our success or failure may be related to working through the UBP as opposed to contacting the care team directly. As a quaternary care freestanding children’s hospital, our patient population and their infectious conditions may not mirror that of other institutions, especially nonfreestanding children’s hospitals.Reference Turner, Valcarlos, Loeffler, Gilbert and Chan 23
Our study adds to the literature by identifying important patient-level, programmatic, and provider-level factors that were associated with provider disagreement of PAFR. Given the time-intensive nature of PAFR, future research should explore ways to improve the acceptance of PAFR. Hospitals should incorporate antimicrobial stewardship into ongoing provider education and future studies should examine whether this improves acceptance of PAFR. ASP programs looking to implement PAF programs should be mindful of the programmatic and provider-level factors that may influence uptake in PAFR and track rates of disagreement to employ targeted interventions as needed.
ACKNOWLEDGMENTS
Financial support: No financial support was provided relevant to this article.
Potential conflicts of interest: All authors report no conflicts of interest relevant to this article.
SUPPLEMENTARY MATERIAL
>To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2018.85
