Introduction
Patients undergoing congenital heart surgery face long-term risks of premature mortality related to their CHD; however, there are currently no validated tools to provide a long-term risk assessment following congenital heart surgery. Reference McCracken, Spector and Menk1 The Risk Adjustment for Congenital Heart Surgery-1 and the Society of Thoracic Surgery–European Association for Cardio-Thoracic Surgery (STAT) Congenital Heart Surgery Mortality scoring systems have been used to predict early postoperative mortality following pediatric congenital heart surgery. Reference Jenkins, Gauvreau, Newburger, Spray, Moller and Iezzoni2,Reference O’Brien, Clarke and Jacobs3 Initially released in 2002, Risk Adjusted Classification for Congenital Heart Surgery-1 was updated in 2022 to be more applicable to ICD-10 administrative datasets. Similarly, the STAT scoring system, initially released in 2009, was optimised in 2020 using more recent operative data. Reference Jacobs, Jacobs and Thibault4,Reference Allen, Zafar and Mi5 While these scoring systems have been shown to perform well in predicting early postoperative mortality, their predictive value for long-term outcomes has never been tested. Thus, the purpose of this study is to determine whether commonly used and easily obtainable paediatric congenital heart surgery risk scores are also applicable to long-term outcomes.
Materials and methods
We performed a retrospective cohort study using data from the Pediatric Cardiac Care Consortium, a US-based registry of cardiac surgical interventions in 47 participating centres between 1982 and 2011. Reference Vinocur, Moller and Kochilas6,Reference Spector, Menk and Knight7 Since 1982, the Pediatric Cardiac Care Consortium has enrolled over 100,000 patients with CHD and has data for over 130,000 cardiac interventions. Reference Moller, Hills and Pyles8 Prior analysis has demonstrated the surgical case mix at Pediatric Cardiac Care Consortium to be representative of congenital heart surgery performed across the US. Reference Vinocur, Moller and Kochilas6 In-hospital mortality data are available for all procedures performed at a Pediatric Cardiac Care Consortium centre, while post-discharge deaths are captured by matching identifiers with the National Death Index through 2021. Reference Spector, Menk and Vinocur9 The study was reviewed and approved by the Institutional Review Board of Emory University, approval number IRB00080706.
We included all patients sequentially enrolled in the Pediatric Cardiac Care Consortium between 1982 and 2011 who were operated on for selected CHDs before 21 years of age and whose first operation had the highest surgical risk score when compared to any known subsequent surgeries required for complete repair. These included: coarctation of the aorta, complete atrioventricular canal, D-transposition of the great arteries, partial anomalous pulmonary venous connection, single ventricle physiology, total anomalous pulmonary venous return, tetralogy of Fallot, and ventricular septal defect. Patients with prior congenital heart surgery outside of the Pediatric Cardiac Care Consortium or patients with an operation that could not be assigned a surgical risk category by the Risk Adjusted Classification for Congenital Heart Surgery and STAT scoring systems were excluded. In addition, we excluded patients who operated in a non-US centre, who had non-US residence at time of surgery, and those with inadequate identifiers to match with the National Death Index records.
The ten selected cardiac lesions were chosen to represent the full spectrum of complexity of cardiac anatomy and match as closely as possible the list of benchmark procedures selected by the Society of Thoracic Surgeons for congenital heart surgery to compare the performance of surgical centres, as long as there was a sufficient number of surgeries for meaningful estimates of long-term survival. Reference Jacobs, Mayer and Pasquali10 This led to many more than ten surgical procedures being performed within the dataset and included all but one of the ten index surgery groups, Truncus Arteriosus. The goal was to utilise cardiac lesions for which the initial surgery would be the highest-risk surgery category and have relatively equal numbers of each respective lesion severity.
We used the first congenital heart surgery to determine the patient’s risk category by Risk Adjusted Classification for Congenital Heart Surgery-1, Risk Adjusted Classification for Congenital Heart Surgery-2, STAT, and STAT 2020. Reference Jenkins, Gauvreau, Newburger, Spray, Moller and Iezzoni2–Reference Allen, Zafar and Mi5 All scoring systems used five risk categorisation levels from one (lowest risk) to five (highest risk), except Risk Adjusted Classification for Congenital Heart Surgery-1, which uses six risk levels but five and six are combined due to the small number of risk category 5. For our analysis, we applied the same definition of postoperative mortality described in the Risk Adjusted Classification for Congenital Heart Surgery-2 and STAT 2020 methodology, which was defined as death prior to discharge or within 30 days of the index (highest risk) surgery if discharged before 30 postoperative days. Reference Jacobs, Jacobs and Thibault4,Reference Allen, Zafar and Mi5,Reference Overman, Jacobs and Prager11 For our inclusion of surgical era, we chose era tertiles created to represent approximately equal numbers of patients in each era as reported in a previous publication from this cohort. Reference Spector, Menk and Knight7
To validate the four scoring systems for our cohort, logistic regression was used to generate receiver operating characteristic curves to compare their ability to predict early postoperative mortality. Proportional hazard regression was used to assess the ability of the four systems to predict long-term survival up to 30 years after the index surgery. Patients who did not survive the postoperative period were not included in the assessment of long-term survival. A time-dependent receiver operating characteristic curve was generated using inverse probability of censoring weighting approach. To assess the overall performance of each system, C-statistics were estimated using Uno’s Concordance method. Reference Uno, Cai, Tian and Wei12 All analyses were conducted using SAS 9.4 (SAS Institute, Inc., Cary, NC).
Results
There were 20,753 patients with available identifiers who underwent their first operation in the Pediatric Cardiac Care Consortium at <21 years of age for an index cardiac lesion and met all inclusion criteria. Cohort demographic data, operative mortality, and late deaths for the long-term study cohort are summarised in Table 1. Patients were more likely to have surgery prior to 1 year of age and be male. Early post-operative mortality and post-discharge deaths declined across the surgical eras.
Table 1. Patient characteristics and outcomes
1 Operative mortality includes deaths that occurred within 30 days of the index surgery, including any death occurring within 30 days post operation independent of their in-hospital or post-discharge status beyond 30 days (i.e. including deaths prior to hospital discharge after the index surgery).
2 Post-discharge mortality includes deaths that occurred among those who were discharged after the initial postoperative hospitalisation and lived more than 30 days after the index surgery.
Operative mortality is presented by lesion in Table 2. The 30-year Kaplan-Meier survival plots of those lesions are presented in Figure 1 in groups of surgical risk category by one of the four risk stratification systems (Risk Adjusted Classification for Congenital Heart Surgery-1, STAT, Risk Adjusted Classification for Congenital Heart Surgery-2 and STAT 2020). There were 1,971 patients (9.5%) with an early postoperative mortality. Over a median follow up period of 24.4 years (IQR 21, 28.4), a total of 2,058 deaths occurred; 27 (1.3%) occurred after transfer to a non-Pediatric Cardiac Care Consortium hospital within 180 days after the index surgery and were excluded from the long-term analysis.
Figure 1. Long-term survival by RACHS-1 (A), RACHS-2 (B), STAT (C), and STAT-2020 (D).
Table 2. Cardiac lesion of each patient in patient cohort and the operative mortality by lesion, inclusive of surgical procedures and risk scores
1 Deaths that occurred within 30 days of the index surgery, includes both out of hospital mortality within 30 postoperative days and any in-hospital death that occurred prior to discharge.
2 Patients with the following cardiac defects were included:) CAVC = Complete Atrioventricular Canal; CoA = Coarctation of the Aorta; DTGA = D-Transposition of the Great Arteries; PAPVC = Partial Anomalous Pulmonary Venous Connection, Single Ventricle physiology including systemic, LV = left ventricle; RV = right ventricle and undetermined sidedness (other); TAPVC = Total Anomalous Pulmonary Venous Return; ToF = Tetralogy of Fallot; VSD = Ventricular septal defect.
3 Percent of total cohort represented by each cardiac lesion.
4 Percent of operative mortality within each cardiac lesion.
Long-term survival by each risk category in the respective risk assessment tools is shown in Figure 1(A–D). The original Risk Adjusted Classification for Congenital Heart Surgery-1 and STAT tools reached the highest C-statistic for early postoperative mortality (C-statistic 0.7872 for both) while the updated Risk Adjusted Classification for Congenital Heart Surgery-2 and STAT 2020 systems performed similarly well but with slightly lower c-statistic estimates (0.7668 and 0.7724, respectively). Each scoring system demonstrated a stepwise decrease in long-term survival by higher surgical risk category. This trend persisted after the redistribution of a large number of patients from the risk categories 3 for Risk Adjusted Classification for Congenital Heart Surgery-1 and 4 for STAT to different risk tiers in the newer Risk Adjusted Classification for Congenital Heart Surgery-2 and STAT 2020 systems, respectively.Reference Jacobs, Jacobs and Thibault4,Reference Allen, Zafar and Mi5
Comparison of the adjusted cumulative 30-year survival of all risk categories by stratification system is presented in Figure 2. The predictive ability of each system is expressed as area under the curve in the receiver operative curve created for each risk stratification system. Both Risk Adjusted Classification for Congenital Heart Surgery-2 and STAT 2020 demonstrated an improvement in long-term predictive ability over their predecessors with the c-statistic values ranging from 0.6741 for Risk Adjusted Classification for Congenital Heart Surgery-1 to 0.7156 for Risk Adjusted Classification for Congenital Heart Surgery-2 and from 0.6995 for STAT to 0.7114 for STAT 2020.
Figure 2. Time-dependent receiver operating characteristic curves and C-statistic values for long-term survival prediction ability by each risk-scoring tool.
Discussion
As the fields of congenital cardiac surgery and cardiology have evolved, so have the goals and expectations. The role of surgical scoring tools has been significant in leading to standardisation of practices and greatly improving surgical outcomes. Following this success, the aim has shifted to achieve long-term survival that will equal the life expectancy of the general population. However, our current scoring systems have been devised only to account for the short-term survival and do not provide significant comfort or insight to the families of children undergoing these procedures when they question what their child’s future may hold in the fourth, fifth, and sixth decades of life.
In this comparison study of congenital heart surgical risk scoring systems, we found an adequate, albeit attenuated, ability to predict long-term survival over 30 years after the first index surgery. Risk Adjusted Classification for Congenital Heart Surgery-2, a system validated for ICD-10 administrative data, performed equally well with the STAT 2020 system to predict both early and long-term postoperative mortality. While the Risk Adjusted Classification for Congenital Heart Surgery-2 tool is ostensibly designed for administrative data sets, this study suggests it also has validity in large, multicenter clinical registry datasets as well. The STAT risk assessment tool had a very slight edge over STAT 2020 for early postoperative mortality prediction, but the STAT 2020 performed better in the long term.
Prior work in the Pediatric Cardiac Care Consortium has highlighted long-term differences in mortality based on cardiac lesion severity. Reference Spector, Menk and Knight7 This study sought to apply the surgical risk-scoring tools to lesions representing the full spectrum of lesion severity. Often, long-term outcomes are discussed by physicians and families by lesions rather than specific surgery. In addition, the management plan of several included cardiac lesions can include several procedures of different surgical risks, which in combination impact the long-term outcome of the underlying heart defect.
The findings of this study suggest there may be opportunities to refine existing scoring systems for long–term prediction after congenital heart surgery. Additional risk factors worth considering include those previously reported to impact long-term mortality: presence of additional cardiac defects, patient’s sex, prematurity, chromosomal anomalies, or non-cardiac malformations. Reference Peterson, Setty, Knight, Thomas, Moller and Kochilas13,Reference Knowles, Bull and Wren14 Prior work utilising the Pediatric Cardiac Care Consortium has found both a modest centre’s surgical volume dependent and independent effect on early postoperative mortality. Reference Vinocur, Menk, Connett, Moller and Kochilas15,Reference Pasquali, Thibault and O’Brien16
The study has some limitations. First, neither the risk score nor the number of subsequent procedures were included in this analysis. The STAT and Risk Adjusted Classification for Congenital Heart Surgery scoring systems are by design applied to a single index surgery and the purpose of this investigation was to determine if those tools, without any modification, could predict long-term mortality. Further study will be necessary to determine if a new model accounting for the cumulative effect of subsequent surgical procedures throughout the lifespan would have improved predictive power. A second limitation is the primary outcome of all-cause mortality includes both cardiac and non-cardiac causes of death, while it does not consider heart transplant events. Restricting the analysis to only cardiac-related deaths along with inclusion of heart transplants as an equivalent of “cardiac death” may provide an important advance in understanding the true lifetime risk related to the CHD and surgical repair. However, there is no evidence suggesting that non-cardiac, late deaths would be differentially distributed by surgical risk category in a way that would dramatically change the predictability of the scoring tools. In fact, adults with even simple heart defects have been shown to have a significantly higher burden of comorbidities compared to the general population. Reference Rodriguez, Moodie and Parekh17 Lastly, the long-term analysis of surgical risk scores can only indirectly assess the outcome of an individual cardiac lesion. The operative mortality of each cardiac lesion is inclusive of the wide array of surgical palliations that may be offered based on the physiologic subtype of each heart defect as well as the surgical preferences of the centre and the surgical era. Consistent with the use of existing surgical tools, this study cannot draw conclusions regarding long-term risk purely by the diagnosed cardiac defect and thus surgical outcomes combine patients with several different cardiac defects and surgeries with the same risk score.
In summary, these findings suggest that the risk-scoring tools used for early postoperative survival do hold value in predicting long-term survival as well, but there is a clear opportunity for an improved scoring system to predict long-term survival. Future efforts are warranted to develop a tool with improved long-term survival prediction, ideally based on a primary CHD rather than operation.
Financial support
This study is supported by the National Heart, Lung, and Blood Institute (R01 HL122392) and Department of Defense (PR180683).
Competing interests
Andrew Dailey Schwartz, Krisy Kuo, Yanxu Yang, and Yijin Xiang declare none. Lazaros Kochilas and Matthew Oster received grants from the National Heart, Lung, and Blood Institute R01 (HL122392) and Department of Defense (PR180683).
