Surgical site infections (SSIs) account for ∼20% of healthcare-associated infections, 1–Reference Coello and Gastmeier4 and they are associated with considerable morbidity, disability, and extra costs. Reference Del Pozo and Patel5,Reference Wilson, Charlett, Leong, McDougall and Duckworth6 Arthroplasty is a common surgical procedure Reference Del Pozo and Patel5,Reference Dal-Paz, Oliveira, Paula, Emerick, Pécora and Lima7 ; >150,000 total hip arthroplasties (THAs) and total knee arthroplasties (TKAs) are performed per year in France, according to the Technical Agency for Information on Hospital Care (ATIH) hospital discharge data (PMSI) for 2017–2019. Although infrequent (<2%), SSIs following these surgeries are devastating because of their heavy medical consequences. Reference Parneix8–Reference Zimmerli10 The financial burden, in addition to the emotional component emphasized by the media, have made these SSIs a key target for epidemiologic surveillance in recent decades. Reference Grammatico-Guillon, Baron and Gettner9,Reference Astagneau, Rioux, Golliot and Brücker11–Reference Muilwijk, van den Hof and Wille14 However, most SSI surveillance systems based on clinical data collection at the bedside are not cost-effective, mainly because of the human and logistic resources needed. Several recent studies have used hospital databases as a tool for automated surveillance. Reference Grammatico-Guillon, Rusch and Astagneau15,Reference Van Mourik, Perencevich, Gastmeier and Bonten16 In France, SSI surveillance is a priority target of the French national program for the prevention of healthcare-associated infections (PROgramme national de Prévention des Infections Associées aux Soins, PROPIAS 17 ). Guidelines for SSI prevention, skin preparation, antibiotic prophylaxis, and enhanced recovery are available. 17 Coding guidelines regarding THA, TKA, and SSI have also been made available by the Technical Agency for Information on Hospital Care (ATIH). 18 At the national level, the hospital discharge database (PMSI) has been shown to have with acceptable sensitivity and positive predictive value for THA and TKA infections. Reference Grammatico-Guillon, Baron, Gaborit, Rusch and Astagneau19,Reference Grammatico-Guillon, Perreau and Miliani20
A standardized indicator for SSI was needed as part of the benchmarking strategy for healthcare quality and safety as well as patient outcome improvement. Reference Fung, Lim, Mattke, Damberg and Shekelle21,Reference McKibben, Horan and Tokars22 Thus, the French National Authority for Health (Haute Autorité de Santé, HAS) decided to develop a national indicator, ISO-ORTHO, using a collaborative approach. 23,24 The HAS set up a multidisciplinary working group in 2017 to ensure consensus on the development processes as well as adherence of the field practitioners. In this project, we aimed to provide hospital healthcare professionals a valid outcome indicator designed for assessment of improvement and hospital benchmarking. Here, we present the steps of ISO-ORTHO indicator development, our evaluation of its feasibility for routine SSI assessment, and its different uses including hospital benchmarking. 25
Methods
Study population
Patients who had undergone a hip or knee arthroplasty according to the French Common Classification of Medical Acts (FCCMA) in France were selected from the national hospital discharge database (PMSI, January 1, 2017, through September 30, 2017) using specific validated codes for the implantation of a total hip or knee prosthesis, according to the revision of the inclusion and exclusion codes. 18,26 The inclusion criteria were based on the first hospital stay in the study period with a surgical code from the FCCMA for replacement of hip (NEKA010, NEKA012, NEKA013, NEKA014, NEKA015, NEKA016, NEKA017, NEKA019, NEKA020, NEKA021) or knee (NFKA007, NFKA008, NFKA009). Our exclusion criteria were designed to select a target population of adult patients who had had a THA or TKA and would be comparable among hospitals. We used SSI occurrence within 3 months because these infections could be related to in-hospital care management and not to the patient profile or to a previous surgery (for a complete list of the criteria, see Supplementary Data S1 online).
The hospital stays meeting the inclusion and exclusion criteria in the year 2017 were included in the study, and data for these patients from the previous year (2016) were used in a search of medical histories and patient health conditions in the 12 months before the THA/TKA hospital stay to identify comorbid conditions (see also the Supplementary Data S2 online).
Case definition of THA/TKA SSI
First, the case definition of THA/TKA SSI was based on the diagnosis and procedure codes used in the hospital discharge summary and the presence of specific codes up to 90 days after the joint replacement (Supplementary Data S3 and S4 online). In France, PMSI data and International Statistical Classification of Diseases, Tenth Revision (ICD-10) details are routinely collected. The PMSI data are based on the mandatory notification of each hospital stay, through coded summary, for all French hospitals, public or private. An algorithm was built for SSI detection using these databases in an investigation of >1,000 medical charts in which the positive predictive value (PPV) was 87% and the negative predictive value was 98%, along with a sensitivity of 97%. Reference Grammatico-Guillon, Baron, Gaborit, Rusch and Astagneau19,23,Reference Grammatico-Guillon, Baron and Rosset27 This algorithm was used to develop and consolidate the ISO-ORTHO indicator (Supplementary Data S3 online).
The HAS performed a quality indicator validation process and then set up a multidisciplinary expert group including orthopedic surgeons, infection control practitioners, physicians of medical information, national experts in coding, and patient representatives. This group refined and validated the target population and risk factors for adjustment based on literature analysis, validation by the clinicians of the working group, and the feasibility and the reliability of their detection in the hospital discharge databases. 23,24 Next, during a pilot release in France, feedback was gathered from the field practitioners who reviewed the SSI cases selected by the algorithm; they provided information concerning the false-positive cases as well as the true SSIs not linked to the THA/TKA, which allowed the agency and its working group to refine the algorithm.
The HAS then performed the last step in ISO-ORTHO validation: the SSI medical reports flagged by the consolidated ISO-ORTHO were reviewed by infection control teams and orthopedic surgeons to ascertain the occurrence of infection at a large scale, in real life, in the volunteer healthcare facilities across the country. Subsequently, the ISO-ORTHO indicator was revised according to these 3 recommendations:
-
1) Optimize the SSI detection algorithm by focusing on osteoarticular ICD-10 codes or using only the specific ICD-10 complication code (T84.5) recommended by the national agency of medical information (ATIH) and the procedure code for SSI diagnosis or treatment (surgery).
-
2) Include adult patients who had an elective THA/TKA during the study period and perform a fair assessment of hospitals to refine the target population of the indicator. In addition to data errors and/or linkage problems that did not allow a 3-month follow-up, exclude patients at very high risk of SSI (history of complex SSI…) and/or with SSIs that may not be related to hospital care (ie, previous intervention within 3 months, patients discharged against medical advice or escaped). Notably, exclusions did not exceed 9% of the entire THA/TKA stays in the study period.
-
3) Because adjustment factors are variables known to increase SSI rates independently from the care quality (potential confounding factors), optimize their detection in the hospital discharge database by expending their detection until 1 year before the THA/TKA stay. These factors are used to calculate the expected SSI number and the ratio of SSIs observed compared with expected SSIs in the target population.
Data management and data analysis
The SSIs were screened in the target population during the THA/TKA stay and in the 3-month follow-up records. If several hospital stays met the SSI criteria, only the first prosthesis-associated infection episode was analyzed. The different hospital stays were linked to each patient by a unique encrypted anonymized patient number (ANO). Medical conditions, comorbidities, and hip or knee surgery were searched in the previous 12 months before the replacement stay using specific ICD-10 codes (Supplementary Data S2 online). We assessed the following comorbid conditions and risk factors: sex, morbid obesity (BMI ≥35 kg/m2), malnutrition, diabetes, active malignant tumor, history of inflammatory polyarthritis, immune impairment or cirrhosis, history of osteo-articular infection, history of hip or knee surgery or arthroscopy, chronic renal failure, history of hospital stay ≥4 days, socioeconomic factor of precarity along with the location of the device (hip or knee) (Supplementary Data S2 online).
The preliminary step was to assess the impact of the consolidation process by comparing the initial indicator to the final ISO-ORTHO consolidated by the HAS and its multidisciplinary working group (Table 1). SSIs during THA/TKA stays were reduced as expected, and the proportion of exclusions from the eligible target population was reduced from 17% to 10%, which helped to validate the ISO-ORTHO indicator.
Table 1. Comparison of Results Before and After ISO-ORTHO Consolidation (2017 Data)
Note. THA/TKA, total hip arthroplasty/knee arthroplasty; SSI, surgical site infection; HCF, healthcare facility.
We excluded patients with data quality problems related to cause of infection (link to SSI) and patients not living in France from the target population to avoid missing data (N = 708 stays, representing 0.45% of the 154,614 THA/TKA during the study period). Moreover, we excluded hospitals with low THA/TKA target stays (<10) from the national assessment because of the uncertainty of SSI estimates associated with small sample sizes. These exclusions allowed robust analyses of these hospitals. With funnel plots, hospitals can compare their results to the reference (=1) and see their position within the SD limits.
A multivariate logistic regression model was used to predict the expected probability of SSI event by patient and to calculate the hospital standardized incidence ratios (SIRs). All covariates included in the model were retained, even if not statistically significant, to control potential confounders. Statistical performance of the logistic regression model was assessed using area under the receiver operating characteristics curve (ROC) or C statistic, and the Hosmer–Lemeshow test. No adjustment was made for the clustering of patients within healthcare facilities, and only the main effects for the variables were fitted. The sum of observed SSI events for each hospital (THA/TKA infection events detected using HDD algorithm) was compared with the sum of expected SSI events using the ratio of observed SSIs to expected SSIs (ie, SIR = O/E, where 0 is an integer value). We assumed that the observed number of SSI events was an observation from a Poisson distribution. Reference Dobson, Kuulasmaa, Eberle and Scherer28 To identify outliers, hospital SIR values were plotted against the expected SSI events for each healthcare facility with at least 10 THA/TKA target stays using a funnel plot. The funnel plot presents the hospital’s results regarding the control limits at 2 standard deviations (SD) and 3 SD, to be compared to the reference. In the case of SIR, the ‘target’ was the point at which the SIR = 1 (ie, the observed number of SSI events equals the expected number). Exact control limits were calculated using the link between the χ2 and Poisson distributions. Data analyses were performed using SAS version 9.1 software (SAS Institute, Cary, NC).
Results
During the study period, 139,926 hospital stays with at least 1 THA or TKA occurred in 790 French acute-care settings, of which 75,046 hospital stays for THA (54%) and 64,880 for TKA (46%) met the inclusion and exclusion criteria (Fig. 1). The median age was 70 years (interquartile range [IQR], 64–78), and 59% were female. The median length of THA/TKA stay was 6 days (IQR, 4–7; range, 0–87). When an SSI was coded, the median length increased to 27 days (IQR, 19–41). Most acute-care settings involved in the study of the target population performing THA or TKA (n = 743) were private hospitals (52%), followed by local hospitals (33%).
Fig. 1. Flow chart of the selection of the target population. Note. HKA, hip or knee arthroplasty; TH/KP, total hip/knee prosthesis; HCO, healthcare organization; PD, principal diagnosis; SD, secondary diagnosis; SSI, surgical site infections.
Overall, 1,253 SSIs occurred over the 3 months of surveillance after arthroplasty via ISO-ORTHO: 33 (2.6%) during the inclusion surgery stay and 1,220 (97.4%) at readmission. The crude SSI rate was 0.9% (95% CI, 0.85%–0.95%): 1.0% for THA, and 0.8% for TKA. The median time interval for SSI occurrence after the arthroplasty was 25 days (IQR, 17–37). In addition, 179 patients died after discharge over the 3-month period (0.1%), of whom 13 had a coded SSI during a readmission (7.3%). However, 308 patients died during their surgical stay without any SSI reported; hence, 487 deaths occurred among the study population, giving an overall death rate of 0.3%. The median duration between the joint replacement and death was 51 days (IQR, 25–72).
The SSI rate was significantly higher in males (1.2%), in patients with previous bone and joint infection during the year before arthroplasty (4.4%), in those with an admission in hospital for >4 days in the previous year (1.4%), and in those with active malignant tumor (2.3%), chronic renal failure (1.3%), morbid obesity ≥ 35 kg/m² (1.7%), diabetes (1.2%), and immune impairment and/or cirrhosis (3.3%) (Table 2). The adjusted odds ratio and their coefficients estimated in the complete multivariate logistic model are presented in Table 3.
Table 2. Characteristics of Patients (Univariate Analysis of Patients With or Without SSI Over the 3-Month Follow-Up After THA/TKA)
Note. SSI, surgical site infection; THA/TKA, total hip arthroplasty/total knee arthroplasty; THP, total hip prosthesis; BMI, body mass index.
a Pearson χ2 test or bilateral Fisher exact test when necessary, comparing patients with and without SSIs.
Table 3. Factors Associated With Surgical Site Infection (SSI) up to the 3-Month Follow-Up (Complete Multivariate Logistic Model)
Note. GOF, goodness of fit; 95% CI, 95% confidence interval; aOR, adjusted odds ratio; ROC, receiver operating characteristic. The Hosmer-Lemeshow test does not reveal any flaws in the fit of the model to the data (no significant difference between expected and observed SSI events; P = .2348). The area under the curve is 63.6%, which is at the limit of unacceptable. No. of replacement stays, 139,926; no. of HCOs, 790; no. of SSIs, 1,253; area under the ROC curve, 0.636; Hosmer-Lemeshow GOF test P value, .2348.
Concerning the funnel plot presentation, 89.9% healthcare facilities were within 2 SD limits; however, some hospitals in the targeted population were outliers (Fig. 2). During the study period, 12 healthcare facilities were classified as outliers at more than +3 SD (1.6% of the facilities), and 59 facilities (7.9%) were outliers between +2 SD and +3 SD.
Fig. 2. Funnel plots of the distribution of the healthcare facilities. Two sets of control limits are displayed at 95%; ‘alarm’ limits, and 99.8% ‘action’ limits roughly equate to ±2 and ±3 standard deviations, respectively. The control limits form a funnel shape around a target, which is presented as a horizontal line. Red dots represent 12 healthcare facilities classified as outlier at > +3 SD (1.6% of the facilities) and orange dots correspond to 59 healthcare facilities outlying between +2 SD and +3 SD (7.9% of facilities).
Discussion
The ISO-ORTHO is the first French outcome indicator based on hospital database used for SSI detection in THA/TKA patients. Our findings demonstrate that its implementation in routine is possible and currently feasible, despite the limits of medico-administrative data, which are not collected to a quality assessment purpose. This indicator can be used for quality improvement projects because its exclusion criteria and adjustment factors allow targeting patients without increased SSI risk and/or other complications not related to in hospital care management. Moreover, ISO-ORTHO calculates estimates of healthcare outcomes such as the expected SSI cases, which can be compared to observed SSI cases. It can compare the SSI rate to the national SSI rate to depict the position of each healthcare facility in a funnel plot with ±2 SD and ±3 SD limits. Benchmarking strategies aiming to improve hospital care quality and safety regarding their SSI occurrence can benefit from an automated system that provides reliable indicators from routine data at a country level. In the case of SSIs, the ISO-ORTHO indicator can be used as a complementary tool for the improvement of hospital care and for benchmarking, Reference Fung, Lim, Mattke, Damberg and Shekelle21,Reference McKibben, Horan and Tokars22,Reference Jackson, Leekha and Magder29 as well as risk management.
Worldwide, SSI monitoring requires active and patient-based surveillance. 30–32 However, in the methods usually performed to measure healthcare-associated infections, the selection of inpatient and/or outpatient procedures to monitor surgical patients for SSI is conducted for only 1 month in volunteer healthcare facilities. Reference Grammatico-Guillon, Rusch and Astagneau15,Reference Edwards, Peterson and Mu33 Data are collected on every patient undergoing a procedure within the selected procedure category, but this method has a high cost in human resources as well as potential biases of selection and information. 32
This project was led by the HAS, which consulted the key stakeholders and involved a working group of experts including patients and healthcare professionals to ensure the clinical relevance of the indicator and the adherence of the field practitioners. In the ISO-ORTHO model, every SSI detected should be investigated by the infection control team, the orthopedic surgeons, and other staff members. If the cause of the SSI is related to prevention practices, improvement actions should be implemented. Indeed, the data that are fed back to local healthcare staff are essential for improving awareness regarding infection risk and prevention. This awareness has been demonstrated to reduce SSI incidence through routine surveillance. Reference Edwards, Peterson and Mu33–Reference Geubbels and Groot36 In addition, comprehensive and ongoing interventions, including follow-up education and engagement of companion services, are needed to sustain decreasing SSI rates. Reference Wattier, Levy, Sabnis, Dvorak and Auerbach37
Monitoring SSI occurrence, along with a dedicated infection control team, could sustain the impact of ISO-ORTHO. For instance, dashboards with the funnel plots associated with audit and feedback to reduce overall SSI rates, along with a yearly meeting between orthopedic surgeons and infection control teams, could help maintain these results and improve awareness, as in antibiotic stewardship. Reference Cooke, Stephens and Ashiru-Oredope38,Reference Drees, Gerber, Morgan and Lee39
Because variability between hospitals can be explained in part by patient characteristics, funnel plots were constructed by plotting the observed SSI rates against the SSI standardized infection ratios using patient characteristics and risk factors to calculate the expected SSI rate. Therefore, the variability of ISO-ORTHO results due to different levels of surgical activity and patient characteristics were taken into account. This valuable method for estimating hospital performance has the ability to clearly identify hospitals with unexpected high or low ratios as outliers. Indeed, healthcare-associated infections, such as SSIs, could be used as pay-for-performance metrics. Reference Jackson, Leekha and Magder29 Thus, healthcare professionals in the field require adjustment of clinically relevant factors due to this potential use of the ISO-ORTHO for financial purposes. As recently recommended in the literature, comorbidity-based risk adjustment should be strongly considered to adequately compare SSI rates across hospitals. Reference Jackson, Leekha and Magder29 The model was able to identify the outliers at ±2 or ±3 SD to adjust the sensitivity in the detection of hospital outliers. Both are interesting for an alert tool, but the limits of ±3 SD were chosen for national routine implementation because they are related to the lower risk of statistical error (0.2%). Even if this reduced the sensitivity of the detection system, it facilitated the workload of the field infection control teams by decreasing the risk of false-positive cases.
The implementation process has several strengths. Funnel plots in the reuse of real-life data could allow the national healthcare politics to move forward using the hospital discharge database tool as a benchmark of quality improvement in SSI prevention and treatment. Healthcare facilities falling outside the control limits are potential outliers, and possible causes must be investigated. Thus, the HAS carried out a national implementation of indicator, ISO-ORTHO, yearly and automatically restituted to the healthcare facilities via a secured web platform QualHAS since 2019. The estimation of the final predictive positive value was even higher in routine use (91%). 25 The model performance will improve as the indicator is generalized. Because we already have selected the clinically relevant factors and optimized their detection, ISO-ORTHO will not need to be revised. On the contrary, as the indicator becomes known and is used, the quality and completeness of the risk-factor coding will improve. Notably, ISO-ORTHO is not calculated in hospitals with few THA/TKA target stays (ie, <10). In 2017, 57 HCOs had <10 THA/TKA stays during the study period; together, they had only 213 THA/TKA stays and 12 SSIs. The number of THA/TKA stays and SSIs are followed year after year. If the number of SSIs increases in a hospital, it will be possible to target the hospital with an alert message and/or to inform the accreditation program.
Postdischarge monitoring of SSIs is a challenge, especially in short-term/ambulatory surgery because the patient is no longer under direct medical supervision. In parallel, recommendations for improving arthroplasty management (eg, antibioprophylaxis, operating room hygiene measures, and/or orthopedic enhanced recovery programs) 25,Reference Haudebourg and Grammatico-Guillon40 and codes for hospital stays are proposed on the agency website. Thus, field practitioners have access to information regarding improvements to their practice and risk management as well as quality benchmarks.
This study also had several limitations. First, the use of administrative hospital databases introduced an inherent bias. Reference Jouan, Grammatico-Guillon, Espitalier, Cazals, François and Guillon41,Reference Laporte, Hermetet and Jouan42 The strengths and limitations of using healthcare databases for epidemiological and quality assessment purposes have been discussed extensively. Reference Carling, Fung, Killion, Terrin and Barza34,Reference Goossens, Ferech, Vander Stichele and Elseviers35,Reference Wattier, Levy, Sabnis, Dvorak and Auerbach37–Reference Drees, Gerber, Morgan and Lee39 For instance, the hospital discharge database is primarily used as a financial tool, and the reported codes could be mainly codes valued financially, whereas contextual codes may be neglected. Another point could be the lack of coding training among those in charge of coding the hospital stays. Incorrect or imprecise coding can occur, and SSIs in orthopedics are rare and difficult to define clinically (eg, true infection, soiling, inflammation, etc); thus, the coding along with the medical chart review could be inaccurate and unreliable. Reference Grammatico-Guillon, Baron, Gaborit, Rusch and Astagneau43,Reference Hermetet, Laurent and El Allali44 However, the expert group and the HAS conducted a specific and vast study of information and algorithm consolidation allowing decreasing its inherent risk. Accuracy of the patients’ THA/TKA and of the SSI was confirmed by a review of >700 THA/TKA patient charts with SSIs. Second, comorbid conditions were identified based on coding of the hospital stay and were defined as not present if the ICD-10 code was not coded. Thus, the corresponding risk factors in the study population may have been underestimated. However, to minimize that bias, patient medical conditions were searched within the arthroplasty stay and for the 12 months before. As the indicator is used, the quality and completeness of the risk-factor coding will likely improve. ISO-ORTHO was built in consultation with a multidisciplinary expert working group, and we expect the results to be relevant and useful in practice. However, ISO-ORTHO could not be used in all hospitals across France. The modeling compares healthcare facilities with the national rate and expected numbers of SSIs, which allows benchmarking. Thus, the outcome indicator for these factors must be adjusted to ensure fair assessment of hospitals Reference Jackson, Leekha and Magder29 ; therefore, ISO-ORTHO is not estimated in hospitals with very few arthroplasty procedures, both to avoid SSI estimates associated with small sample sizes and to ensure relevant and robust benchmarks. The HAS will perform a medical file review for the SSIs detected, directly through local French hospital teams. In this way, misclassifications can be reclassified as true-positive SSIs, coding errors can be corrected, and performance parameters can be adjusted.
Eventually, the collegiate development of this outcome indicator of SSIs in orthopedics, ISO-ORTHO, will be implemented routinely, and its results and benchmarks will be made available to the healthcare facilities. If the positive predictive value is at least 85%, other uses could be developed, such as open broadcast of results on the national website ScopeSanté (www.scopesante.fr) and/or for financial analyses.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/ice.2021.371
Acknowledgments
We thank all of the medical doctors and infection control teams involved in the development of surveillance networks in response to or despite particular circumstances. We thank the experts and patients of the working group and hospital administrators and the healthcare professionals involved in this development and validation of ISO-ORTHO, notably, Pr Thomas Bauer and Dr Anne Sophie Loth for their contribution to building the consolidated algorithm. We also thank the ATIH (Agence technique de l’information médicale) collaborators directly involved in the coding validation, indicator calculation, and restitution to the hospitals: Dr Catherine Le Gouhir, Dr Marie-Caroline Clément, Pascaline Lebreton, Pauline Renaud, Marc Mossand and Jean Paul Blanc. We also thank those who validated the preliminary 2014 algorithm: surgeons, medical doctors, infection control teams, doctors specializing in medical information systems and medical information technicians, especially in the Teaching hospital of Tours (Service d’information médicale, épidemiologie et economie de la santé, CHRU de Tours).
Financial support
No financial support was provided relevant to this article.
Conflicts of interest
All authors report no conflicts of interest relevant to this article.
