Variation in outcomes across hospitals

Over the past three decades, outcomes for patients undergoing congenital heart surgery have improved dramatically owing to improvements in diagnostic capabilities, refinement in surgical technique, and advances in peri-operative care. The majority of patients now survive to hospital discharge, with overall mortality rates <5% in national data from the United States.Reference Jacobs ¹ However, despite these overall improvements, several recent studies have shown that there is a marked variation in outcome across hospitals performing congenital heart surgery.

One of the largest studies included 18,375 patients from 74 hospitals undergoing eight benchmark operations in the Society of Thoracic Surgeons Congenital Heart Surgery Database from 2005 to 2009.Reference Jacobs, O'Brien and Pasquali ² The eight operations evaluated included ventricular septal defect repair, tetralogy of Fallot repair, complete atrioventricular canal repair, arterial switch operation, arterial switch operation plus ventricular septal defect closure, Fontan operation, truncus arteriosus repair, and the Norwood procedure. Overall in-hospital mortality ranged from 0.64% for ventricular septal defect repair to 19.3% for the Norwood procedure in this cohort.Reference Jacobs, O'Brien and Pasquali ² Both in-hospital mortality rates and post-operative length of stay varied across hospitals for the eight operations evaluated. However, despite pooling multiple years of data from this national registry, it was difficult to identify hospitals as statistical outliers because of the relatively small sample sizes and low mortality rates for most operations at any given institution. The exception to this was the Norwood procedure, where Bayesian analysis showed that risk-adjusted in-hospital mortality rates varied significantly across hospitals ranging from 7.0% to 41.6%, and 11 hospitals were identified as statistical outliers.Reference Jacobs, O'Brien and Pasquali ² Other recent studies have supported these results. In an analysis of 546 patients enrolled in the Pediatric Heart Network Single Ventricle Reconstruction Trial, the rate of in-hospital death or cardiac transplant was 18% overall and varied from 7% to 39% across 14 trial sties.Reference Pasquali, Ohye and Lu ³

Given the difficulties mentioned above in comparing outcomes across centres for most individual operations, investigators have also examined grouping operations into strata of similar risk or complexity in order to facilitate inter-institutional comparisons. An analysis of 58,506 operations from 2005 to 2009 in the Society of Thoracic Surgeons Congenital Heart Surgery Database evaluated variation in outcome across 73 hospitals for operations risk stratified into five categories based on the Society of Thoracic Surgeons – European Association for Cardio-Thoracic Surgery (STAT) methodology (STAT category 1 = operations with the lowest mortality risk; category 5 = operations with the highest mortality risk).Reference Jacobs, O'Brien and Pasquali ^{4 ,} Reference O'Brien, Clarke and Jacobs ⁵ Even when evaluating groups of procedures rather than individual operations, there was limited power to assess between-hospital variation in outcome for the lower risk categories (STAT categories 1–3).Reference Jacobs, O'Brien and Pasquali ⁴ For STAT category 4, the overall aggregate mortality rate was 8.0%, and in Bayesian analysis varied from 4.2% to 16.3% across hospitals. For STAT category 5, the aggregate mortality rate was 18.4%, and varied from 7.3% to 43.3% across hospitals.Reference Jacobs, O'Brien and Pasquali ⁴ Thus, these studies demonstrate substantial and clinically meaningful variation across centres in mortality rates, particularly for higher complexity or high-risk operations. However, it is also apparent that it is difficult to statistically quantify this variation for many individual operations, particularly those associated with low overall mortality.

Other studies have evaluated resource utilisation associated with congenital heart surgery. There was one recent analysis that assessed hospital charges among infants born with birth defects across the United States in 2003.Reference Robbins, Bird and Tilford ⁶ This analysis found that congenital heart defects were among those associated with the highest hospital charges, including an average of $199,597 for the Norwood procedure and $192,781 for truncus arteriosus repair.Reference Robbins, Bird and Tilford ⁶ Variation in resource utilisation across hospitals has also been evaluated. There was one large study that analysed 2124 patients across 20 hospitals undergoing four different operations of varying levels of complexity from 2001 to 2007 in a large administrative database.Reference Pasquali, Sun and d'Almada ⁷ This study found that, as expected, total hospital costs increased with the complexity of the operation, from a median of $12,761 for atrial septal defect repair to $55,430 for the arterial switch operation.Reference Pasquali, Sun and d'Almada ⁷ Significant variation in total costs across centres was found for all operations even after adjustment for patient and centre factors and length of stay. Interestingly, in this study there was greater variation across centres for the lower complexity operations – atrial septal defect and ventricular septal defect repair – compared with the higher complexity operations – tetralogy of Fallot repair and the arterial switch operation.Reference Pasquali, Sun and d'Almada ⁷ Other studies have also found significant variation in resource utilisation across different regions of the United States.Reference Connor, Gauvreau and Jenkins ⁸

Several studies have demonstrated that the vast majority of inpatient costs for patients undergoing congenital heart surgery are related to room and board charges, and that higher costs are associated with longer length of stay.Reference Pasquali, Sun and d'Almada ^{7 ,} Reference Parrish, Pollock, Gauthier, Park and Hobde ⁹ Variation in post-operative length of stay has been shown to be greater with increasing operation complexity. One large multi-centre analysis found that average post-operative length of stay after ventricular septal defect repair ranged from 5.8 to 10.4 days across centres, whereas the range for the Norwood operation was 17.5 to 67.5 days.Reference Jacobs, O'Brien and Pasquali ² Understanding the key drivers of variation in length of stay will be an important step towards reducing costs overall, and implementing targeted interventions to shorten length of stay at outlier institutions where costs are highest.

Underlying mechanisms

Although it has been shown that both outcomes and resource utilisation for children undergoing heart surgery vary across hospitals, relatively few studies to date have examined the mechanisms underlying this variation. One concept, known as “failure to rescue”, that has been studied in the adult surgical population has been evaluated recently in patients undergoing congenital heart surgery.Reference Pasquali, He, Jacobs, Jacobs, O'Brien and Gaynor ¹⁰ Historically, reducing post-operative complications as a means to improve outcomes and reduce variation across centres has been a general focus in patients undergoing a variety of surgical procedures. However, there is increasing evidence in the adult surgical literature that complication rates actually tend to be fairly similar across institutions, such that high-performing centres with low mortality rates do not necessarily have a lower rate of complications.Reference Ghaferi, Birkmeyer and Dimick ¹¹ Complications may be related more to patient comorbidities than to hospital characteristics or quality. In contrast, high-performing centres appear to have a lower rate of death in those who suffer a complication – or lower “failure to rescue” rate.Reference Silber, Williams, Krakauer and Schwartz ¹² Failure to rescue has been endorsed as a performance measure by the National Quality Forum. ¹³

A recent study evaluated the concept of failure to rescue in 40,930 children undergoing heart surgery at 72 centres participating in the Society of Thoracic Surgeons Congenital Heart Surgery Database from 2006 to 2009.Reference Pasquali, He, Jacobs, Jacobs, O'Brien and Gaynor ¹⁰ In this study, the overall in-hospital mortality was 3.7%, 39.3% had a post-operative complication (any of those defined in the Society of Thoracic Surgeons Congenital Heart Surgery Database), and the failure to rescue rate (number of deaths in those with a complication) was 9.1%.Reference Pasquali, He, Jacobs, Jacobs, O'Brien and Gaynor ¹⁰ When hospitals were characterised by in-hospital mortality rate, there was no difference across hospital mortality tertiles in the complication rate in adjusted analysis; however, hospitals in the highest mortality tertile had a significantly higher failure to rescue rate (12.4% versus 6.6%, p < 0.0001).Reference Pasquali, He, Jacobs, Jacobs, O'Brien and Gaynor ¹⁰ Similar results were seen when evaluating only “major” complications, and in stratified analysis across both low- and high-complexity operations. This study suggested that the higher mortality rate observed at poorer performing hospitals may be partly related to a higher rate of mortality in those with post-operative complications, rather than a higher rate of post-operative complications themselves. Further characterisation of failure to rescue rates across centres, time course, and relationship between multiple post-operative complications and mortality, and association of failure to rescue with structure and process measures impacting a hospital's ability to respond to and manage complications is needed.

Several analyses have also evaluated differences across hospitals in patient care practices, and have found differences in nearly all aspects of peri-operative care. Many of these studies have focused on the care of single-ventricle patients undergoing staged palliation.Reference Wernovsky, Ghanayem and Ohye ^{14 –} Reference Baker-Smith, Neish and Klitzner ¹⁶ There was one recent analysis that utilised prospectively collected data from 14 sites enrolling patients in the Pediatric Heart Network Single Ventricle Reconstruction Trial.Reference Pasquali, Ohye and Lu ³ Apart from randomisation to a right ventricle-to-pulmonary artery shunt or modified Blalock–Taussig shunt, patients received local standard of care. Although patient characteristics were similar across sites, virtually all aspects of pre-operative, operative, and post-operative care examined varied across the trial sites, including factors such as foetal diagnosis ranging from 55% to 85%, median total cardiopulmonary bypass time ranging from 74 to 189 minutes, proportion with an open sternum ranging from 35% to 100%, median intensive care unit length of stay ranging from 9 to 44 days, and enrolment in a home monitoring programme at discharge ranging from 1% to 100%.Reference Pasquali, Ohye and Lu ³ Although this analysis and other studies have found differences in many aspects of care across hospitals, few have evaluated the relationship of practice pattern variation with patient outcomes to determine “best practices”. Given the extent of variation in a wide variety of factors, it can be difficult to account for this in analysis and reliably identify which individual variables are independently associated with outcome.

Variation in hospital-level structure and process measures has also been evaluated in numerous studies. These include variables such as intensive care unit models of care – dedicated cardiac versus general paediatric intensive care unit – and hospital and surgeon case volume.Reference Burstein, Jacobs and Sheng ^{17 –} Reference Welke, O'Brien and Peterson ²¹ Numerous studies have shown that case volume – both hospital volume and surgeon volume – is significantly associated with outcome, and that the volume–outcome relationship is most apparent as case complexity increases.Reference Pasquali, Li and Burstein ^{19 –} Reference Welke, O'Brien and Peterson ²¹ For example, a recent analysis of 2557 infants undergoing the Norwood operation at 53 centres in the Society of Thoracic Surgeons Congenital Heart Surgery Database found that lower centre volume was associated with significantly higher in-hospital mortality (odds ratio in centres with ≤10 versus >20 cases/year 1.54, 95% confidence interval 1.02–2.32).Reference Pasquali, Li and Burstein ¹⁹ The volume–outcome relationship persisted across different levels of pre-operative risk. In addition, across all centre volume strata, lower volume surgeons had higher adjusted mortality rates.Reference Hornik, He and Jacobs ²⁰ However, although investigators found a significant association between volume and outcome, centre volume explained only 14% of the between-centre variation in mortality observed in the cohort, and significant between-centre variation in mortality remained after adjusting for volume.Reference Pasquali, Li and Burstein ¹⁹ Thus, volume alone cannot explain the differences in outcomes across centres.

Efforts to reduce variation and improve outcomes

In summary, studies to date have demonstrated significant variation across hospitals in outcomes and resource utilisation for patients undergoing congenital heart surgery. The mechanisms underlying this variation have yet to be fully elucidated. Rather than through a single study or trial, reducing variation across hospitals may best be accomplished through a collaborative effort focused on sharing of information across sites and continuous quality improvement to identify best practices, reduce variation in care, and improve outcomes, similar to initiatives in adult cardiac surgery.Reference Prager, Armenti and Bassett ^{22 ,} Reference Likosky ²³ The Northern New England Cardiovascular Disease Study Group pioneered work in this area in adult cardiac surgery in the 1980s, and their experience has shown that a precise assessment of variation in practice and outcomes across institutions is a critical first step.Reference Likosky ²³ Similar efforts have been undertaken more recently by other groups. Prager et alReference Prager, Armenti and Bassett ²² recently reported on The Michigan Society of Thoracic and Cardiovascular Surgeons quality collaborative. This group, composed of all adult cardiac surgery programmes in Michigan, meets regularly to evaluate variation in practice and programme outcomes. Through the adoption of practices utilised by high-performing sites, variation in care is reduced, outcomes improved, and hospital costs lowered. For example, following sharing of protocols to facilitate timely extubation, variation in duration of ventilation across sites was reduced and the overall rate of prolonged ventilation decreased from 19% to 14%.Reference Prager, Armenti and Bassett ²² In paediatric cardiology, the Joint Council on Congenital Heart Disease National Pediatric Cardiology Quality Improvement Collaborative has recently begun evaluating practice variation across sites in regard to feeding and home monitoring practices in the interstage period between the Norwood and Stage II procedures.Reference Schidlow, Anderson and Klitzner ²⁴ Founded in 2009, the Pediatric Cardiac Critical Care Consortium (PC⁴) aims to identify variation in paediatric cardiac intensive care practice and outcomes, develop quality metrics and benchmarking tools, create evidenced-based care guidelines, and analyse the value of care in the paediatric cardiac intensive care unit. Finally, efforts are also underway to develop a quality improvement collaborative in paediatric heart surgery.

Alternatively, regionalisation of care, or selective referral of patients to high-performing centres, may reduce variation and improve overall outcomes. A previous study has suggested that selective referral of patients in California from low and medium to high-volume hospitals could theoretically reduce mortality for children undergoing heart surgery, with an estimated 83 deaths avoided during a 3-year period.Reference Chang and Klitzner ²⁵ Regionalisation of care for paediatric heart surgery in Europe has already taken place in the United Kingdom and other countries. In Sweden, care was centralised to two centres with the lowest mortality in 1993, and 30-day national mortality rates were reduced from 9.5% to 1.9%.Reference Lundstrom, Berggren, Bjorkhem, Jogi and Sunnegardh ²⁶

Finally, analyses that take into account both outcomes and cost, which has been termed “value”, are necessary.Reference Porter ²⁷ Currently, there are limited data regarding the relationship between costs and outcomes in congenital heart surgery and potential mediating factors. Such studies may be facilitated through recently developed methodology, allowing linkage of clinical registry data with billing data from administrative data sets.Reference Pasquali, Jacobs and Shook ²⁸