Improving the quality of antibiotic prescribing in nursing homes is both a resident and public health priority. 1–Reference Morrill, Caffrey, Jump, Dosa and LaPlante3 More than half of individuals who reside in a nursing home for >6 months will receive at least 1 course of antibiotics. Reference Crnich, Jump, Trautner, Sloane and Mody4 At the resident level, antibiotic use is a common driver of adverse drug events, Reference Field, Gurwitz and Avorn5 Clostridioides difficile infections, Reference Rotjanapan, Dosa and Thomas6 and increased risk of future infections caused by antibiotic-resistant organisms. Reference Emerson, Eyzaguirre, Albrecht, Comer, Harris and Furuno7–Reference Wang, Foxman, Mody and Snitkin10 At the population level, antibiotic overuse and misuse in nursing homes plays an important role in the concentration, amplification, and spread of multidrug-resistant organisms. Reference Lee, Singh and Bartsch11–Reference McKinnell, Singh and Miller13 In response, the Centers for Disease Control and Prevention (CDC) has advocated for the implementation of antibiotic stewardship programs (ASPs) in nursing homes. 14 As part of regulations recently released by the Centers for Medicare and Medicaid Services (CMS), 15 nursing home ASPs are required to develop protocols that standardize the assessment of residents with suspected infection and criteria for initiation of antibiotic therapy. 16
Standardized criteria for identifying and tracking infections in nursing homes, known as the revised McGeer criteria for infection surveillance, were first developed in 199117 and updated in 2012. Reference Stone, Ashraf and Calder8 Although the revised McGeer criteria are widely utilized in nursing homes, they were not designed to aid decisions on antibiotic initiation. Reference Nace, Drinka and Crnich18 The Loeb minimum criteria for initiating antibiotic therapy, Reference Loeb, Bentley and Bradley19 which were explicitly designed for this purpose, have been employed extensively in research studies, Reference Loeb, Simor and Landry20–Reference Pulia, Kern, Schwei, Shah, Sampene and Crnich25 but they are not widely employed in nursing homes. Reference Olsho, Bertrand and Edwards26 Whether the revised McGeer criteria can be utilized effectively to monitor antibiotic decision making in nursing homes is unknown, and comparative studies with the Loeb criteria, with the exception of one small study, Reference Eure, LaPlace and Melchreit27 have been lacking.
Harmonization of infection surveillance and monitoring of appropriate antibiotic prescribing in nursing homes could be achieved using a common standard if significant agreement between the revised McGeer and Loeb criteria were established. In this study, we applied the revised McGeer and Loeb criteria to a sample of antibiotic treatment courses initiated in Wisconsin nursing homes to establish the overall level of agreement between these 2 sets of criteria and to determine whether the level of agreement varied by infection type.
Methods
We conducted a prospective cross-sectional chart review study in 5 Wisconsin nursing homes, purposively selected for their geographic proximity to the University of Wisconsin–Madison, between January 2013 and September 2014. Trained research staff used REDCap electronic data collection tools hosted by the University of Wisconsin School of Medicine and Public Health, Department of Medicine Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde28,Reference Harris, Taylor and Minor29 to abstract data from health records in study nursing homes. Medication administration and resident health records were used to identify antibiotic treatment events, resident vital signs, exam findings, culture results, and imaging findings. Nursing progress notes and provider orders were reviewed to identify the treatment indication for the prescribed antibiotic. Brief interviews to clarify the indication for the antibiotic were conducted with staff caring for nursing-home residents prescribed an antibiotic when conflicts between data sources were encountered. Nursing staff and provider documentation starting 72 hours before and through the first day of antibiotic treatment were reviewed to identify pertinent resident symptoms, vital signs, and exam findings.
An antibiotic treatment course was included in this study (1) if it was initiated in the nursing home, during a clinic encounter, or at an emergency department (ED) without an associated overnight stay; (2) if the antibiotic prescribed was administered systemically; and (3) if the antibiotic was initiated for the treatment of a urinary tract infection (UTI), skin and soft-tissue infection (SSTI), or respiratory tract infection (RTI). Antibiotic treatment courses initiated during hospitalization and that continued after transfer to the nursing home, topically administered antibiotics, and antibiotics prescribed for an indication other than UTI, SSTI, or RTI were excluded from the analysis.
Appropriateness of antibiotic treatment courses meeting inclusion criteria were determined using the revised McGeer criteria Reference Stone, Ashraf and Calder8 and the Loeb criteria (1 online). Reference Loeb, Bentley and Bradley19 Both criteria require that specific signs (eg, fever) and symptoms (eg, pain with urination) be present for antibiotic treatment to be considered appropriate for a specific indication (eg, UTI). In addition, the revised McGeer criteria require that specific diagnostic test results (eg, positive urine culture) be present. Participating nursing homes were not routinely tracking antibiotic use at the time of this study, but they did employ the revised McGeer criteria for infection tracking and reporting purposes. The Loeb criteria were not routinely employed in any of the study nursing homes.
Descriptive statistics, including the total number of antibiotic treatment courses, number of antibiotic treatment courses by indication, and percentage of antibiotic treatment courses that were appropriate by the revised McGeer and Loeb criteria, overall and by indication, were calculated for the entire sample and by facility. Agreement between the revised McGeer and Loeb criteria was assessed using kappa (κ) statistics. We interpreted the strength of agreement using the approach of Landis and Koch. Reference Landis and Koch30 Data analyses were performed using SAS version 9.1 software (SAS Institute, Cary, NC) and Office Excel 2013 software (Microsoft, Redmond, WA).
Results
All facilities participating in this study were nonprofit, community-based skilled-nursing homes with a mixture of long-term stay and postacute-care beds. Participating nursing homes varied in size from 80 to 200 beds.
In total, 1,442 antibiotic treatment courses were identified during this study. 848 (58.8%) of the observed antibiotic treatment courses were initiated in nursing home, outpatient clinic, or ED and 594 (41.2%) were initiated in the hospital (Fig. 1). Overall, 734 of the nursing home–, clinic-, ED-initiated treatment courses were prescribed for a target infection (UTI, SSTI, or RTI), whereas 112 antibiotic courses were prescribed for other indications. An indication could not be identified for 2 additional treatment courses. UTI (363 events, 49.5%) was the most common indication for antibiotic treatment followed by RTI (206 events, 28.1%) and SSTI (165 events, 22.5%).
Fig. 1. Flow diagram of antibiotic events.
Eligible antibiotic treatment courses satisfied the Loeb criteria (372 [50.7%] of 734) more frequently than the revised McGeer criteria (211 [28.7%] of 734). Overall, 412 (56.1%) antibiotic treatment courses satisfied either the revised McGeer or Loeb criteria. However, only 171 antibiotic courses (23.3%) satisfied both criteria. When stratified by infection type, similar relationships were identified (Table 1A and 1B).
Table 1. Comparison of the Revised McGeer Criteria With the Loeb Criteria a
Note. RTI, respiratory tract infection; SSTI, skin and soft-tissue infection; UTI, urinary tract infection.
a Subtables (A–D) provide 2-by-2 comparisons of the revised McGeer and Loeb criteria of antibiotic events initiated for treatment of respiratory tract infection (RTI); skin and soft-tissue infection (SSTI); urinary tract infection (UTI); and all infections combined. Subtable (E) provides an overall summary of the level of agreement between the revised McGeer and Loeb criteria stratified by type of infection.
When stratified by infection type, satisfaction of either criteria was more common when an antibiotic treatment course was initiated for treatment of SSTI (119 of 165, 72.1%) than for RTI (134 of 206, 65.0%) or UTI (159 of 363, 43.8%). A similar relationship was identified with antibiotic treatment courses that satisfied both criteria: SSTI (36.4%) versus UTI (19.6%) versus RTI (19.4%). The percentage of antibiotic treatment courses satisfying the revised McGeer criteria was consistently lower than for the Loeb criteria: (1) UTI (27.5% vs 35.8%); (2) RTI (24.8% vs 59.7%); and (3) SSTI (36.4% vs 72.1%). The relationship between the indication for an antibiotic treatment course and satisfaction of the revised McGeer and/or Loeb criteria was similar across the study nursing homes (Appendix 2 online).
Kappa statistics demonstrated that overall agreement between the revised McGeer and Loeb criteria was fair (κ = 0.35) (Table 1E). When examining κ statistics by infection type, moderate agreement between the revised McGeer and Loeb criteria was identified for UTI (κ = 0.45) while fair and slight levels of agreement were identified for SSTI (κ = 0.36) and RTI (κ = 0.17) (Table 1E). Patterns of agreement between the 2 criteria by infection type did not vary substantially when stratified by study nursing home, except for facility 3, where the κ statistic was highest for SSTI (κ = 0.26) followed by UTI (κ = 0.20) and RTI (κ = 0.05), and facility 4, where the κ statistic was highest for RTI (κ = 0.38) (Appendix 2 online).
Discussion
Although more than half of the prescribing events in the nursing homes in this study satisfied either the revised McGeer or Loeb criteria, only one-fourth of eligible antibiotic treatment courses met both criteria. The likelihood of classifying an antibiotic treatment course as appropriate was consistently higher, whether overall or by infection type, when using the Loeb criteria as compared to the revised McGeer criteria. Similar discrepancies between the revised McGeer and Loeb UTI criteria have previously been identified. Reference Eure, LaPlace and Melchreit27 To our knowledge, ours is the first study to examine differences between the revised McGeer and Loeb criteria for assessing the appropriateness of antibiotic initiation for the treatment of RTIs and SSTIs. In our study, the appropriateness of antibiotics initiated for the treatment of RTI and SSTI was nearly twice as common when using the Loeb versus the revised McGeer criteria.
Overall agreement between the revised McGeer and Loeb criteria was fair and varied depending on treatment indication. The highest level of agreement between the criteria was observed for UTI. This may be partially explained by the considerable overlap in signs and symptoms used to establish the presence of UTI under the revised McGeer and Loeb criteria. The threshold to establish a RTI appears to be significantly higher using the revised McGeer as compared to the Loeb criteria. An imaging abnormality consistent with pneumonia in combination with at least 1 respiratory and 1 constitutional symptom or, in the absence of imaging, at least 2 respiratory symptoms in combination with at least 1 constitutional symptom is required to establish a RTI with the revised McGeer criteria. In contrast, residents aged >65 years with a history of COPD who develop a new cough would meet the threshold for a RTI using the Loeb criteria. The frequent coprevalence of COPD and heart failure in nursing-home residents may explain the over 2-fold higher frequency of RTIs satisfying Loeb versus the revised McGeer criteria. Not surprisingly, the lowest level of agreement between the revised McGeer and Loeb criteria was observed for RTI. The level of agreement between the revised McGeer and Loeb criteria for SSTI was in between that of UTI and RTI. Although the localizing signs and symptoms under both criteria are similar, a larger number of abnormalities are needed to satisfy the revised McGeer as compared to the Loeb SSTI criteria.
This study has several limitations. First, the data were collected in a relatively small number of nursing homes in a single state, which may limit the generalizability of our findings to other states. Second, this study was conducted in 2013–2014, when there was less focus on inappropriate antibiotic use. Consequently, it is possible that the absolute levels of prescribing events meeting criteria might be higher with data collected more contemporaneously, although it is not clear that the relative levels of agreement between the 2 criteria would differ. Third, we were unable to control for a variety of potential confounders, including resident case mix, prescriber training and experience, prescriber familiarity with the residents, and familial influences, Reference McElligott, Welham, Pop-Vicas, Taylor and Crnich31 which may have altered the prescribing threshold. Nevertheless, the frequency of inappropriate antibiotic prescribing using the Loeb criteria in our study nursing homes is largely consistent with those reported in other studies. Reference Vergidis, Hamer, Meydani, Dallal and Barlam22,Reference Eure, LaPlace and Melchreit27,Reference Phillips, Adepoju and Stone32,Reference Kistler33 Fourth, limitations in the quality of nursing-home documentation and problems with interpretation of diagnostic studies (eg, chest radiographs) Reference Loeb, Carusone and Marrie34 may have led to misclassification of the appropriateness of antibiotic treatment courses included in this study. Fifth, imminent plans to update the Loeb criteria Reference Rowe, Jump and Andersen35 may affect future relationships between the revised McGeer and Loeb criteria. Finally, it is important to recognize that this study focused only on appropriateness of antibiotic treatment initiation and did not address other important facets of antibiotic prescribing quality, including appropriateness of antibiotic choice and treatment duration.
In conclusion, our study shows that there is a general lack of agreement between the revised McGeer and Loeb criteria, and they appear to be measuring different constructs. Consequently, nursing homes should be discouraged from using the revised McGeer criteria, which are currently used for infection surveillance, as a means of tracking and reporting appropriateness of antibiotic prescribing. Modifications to the revised McGeer criteria, that exclude diagnostic study results (eg, urine culture and chest radiograph results), may enhance agreement with the Loeb criteria, and imminent updates to the Loeb criteria may have similar effects, but this will require additional study. Additionally, future investigation into surveys of nursing homes to determine how they are determining appropriateness of antibiotic use and the barriers encountered could provide useful information. Until then, our study results suggest that nursing homes should employ the revised McGeer and Loeb criteria separately for their respective intended purposes.
Acknowledgments
The authors are solely responsible for this document’s contents, findings, and conclusions, which do not necessarily represent the views of the AHRQ or the WI DHS. Readers should not interpret any statement in this report as an official position of the AHRQ, HHS or WI DHS.
Financial support
This project was funded, in part, by the Agency for Healthcare Research and Quality (AHRQ grant no. R18HS022465), US Department of Health and Human Services and by the Wisconsin Department of Health Services Civil Monetary Penalty Fund (grant no. FCC1043).
Conflicts of interest
None of the authors has any affiliation or financial involvement that conflicts with the material presented in this report.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2021.221
