Background

The Aristotle Basic Complexity Score defines the complexity of an operation through three factors:

• • potential for mortality,

• • potential for morbidity, and

• • technical difficulty of the operation.

When designed in 2000, the Aristotle Complexity Score was entirely based on subjective probability. This approach, based on the opinion of experts, was considered a good solution due to the limited amount of data available at that time. In 2008, two large multi-institution databases are available for studying outcomes of congenital cardiac surgery:

• • The Congenital Database of The Society of Thoracic Surgeons, and

• • The Congenital Database of The European Association for Cardio-Thoracic Surgery.

The goal of this paper is to discuss the feasibility of using data from these two registries to create new empirically-based mortality and morbidity components for the Aristotle Basic Complexity Score.

We previously reported that the Aristotle Basic Complexity Score was associated with both mortality as well as prolonged post-operative length of stay greater than 21 days.Reference O’Brien, Jacobs and Clarke¹ The strength of the association was quantified by calculating the C-statistic in a combined sample of nearly 35,000 patients from the congenital database of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery. The C-statistic was equal to 0.70 for mortality and 0.67 for prolonged post-operative length of hospital stay.Reference O’Brien, Jacobs and Clarke¹

When fixed hospital-specific intercepts were added to the logistic regression models along with the Aristotle Basic Complexity Score, the C-statistic was 0.74 for mortality and 0.72 for prolonged post-operative length of hospital stay. The C-statistics of the models containing hospital effects only were 0.63 for mortality and 0.62 for prolonged post-operative length of hospital stay. Thus, adding the Aristotle Basic Complexity Score to a model containing hospital effects appears to improve discrimination.Reference O’Brien, Jacobs and Clarke¹

In an effort to improve further the discriminative ability of the Aristotle Complexity Score, the next generation will take advantage of the available data from over 100,000 congenital cardiac operations currently in the congenital cardiac surgery databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery. The Congenital Database Taskforce of The Society of Thoracic Surgeons, working in collaboration with the Joint Congenital Database Committee of the European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons, is in the process of developing a new tool: the Aristotle Average Complexity Score. The mortality and morbidity components of the new score will be based primarily on objective data from the congenital databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery. Bayesian methods and subjective probability will be used where objective data is lacking. This tool will likely be incorporated into the Congenital Database of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery.

Mortality

The rate of mortality at discharge from the hospital was determined for 130 congenital cardiovascular procedures, using combined data involving more than 70,000 patients undergoing surgery in the years 2002 through 2005, inclusive, who were already present in the congenital cardiac surgical databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery. Procedures were excluded if they were classified as thoracic, extracorporeal membrane oxygenation, or ventricular assist device procedures. To ensure that 95% confidence intervals were no wider than ±5%, twelve additional procedures were excluded because they had less than 18 occurrences in the combined databases. The remaining 118 procedures represented 90.8% of the primary cardiovascular procedures of interest, and more importantly included 96.7% of the patients. The mortality rates per procedure were plotted and the sorted procedures were divided into five levels using subjectively determined break points (Fig. 1).

Figure 1 Mortality rate is plotted in increasing progression for 118 procedures. Light squares indicate data break points.

Each procedure was then assigned a mortality score between 0.1 and 5.0. Zero mortality was assigned 0.1 and the highest mortality given 5.0. Within each level, intervals of mortality rate were determined according to the slope of the mortality curve to correspond with 0.1 increments in index. The plot of the mortality score is illustrated in Figure 2 and the mean mortality for each level is shown in Figure 3.

Figure 2 Derived mortality scores for 118 procedures plotted in increasing order. Trend line reveals the linear approximation of the data array.

Figure 3 Average mortality rate for procedures within five mortality score levels.

This initial development of the mortality index, although interesting, needs refinement using more sophisticated mathematical and statistical methods. In the future, all 145–150 procedures will be included. Mortality rate estimates will be calculated using a Bayesian model that accounts for uncertainty due to rare occurrences and incorporates prior information from an expert panel. Procedures sorted by increasing risk will then be organized into the five data-driven groups of mortality. The groups will be optimized by minimizing the within-group variation and maximizing the between-group variation. The mortality index for each procedure will then be derived by methods similar to the ones used with the preliminary data.

Although individual procedure indices will allow better discrimination, the mortality component of the Aristotle Average Complexity Score will allow for the risk of operative mortality to be easily stratified into 5 levels. These levels may contribute to the planned unification of the Aristotle Complexity Score with the Risk Adjustment for Congenital Heart Surgery system.Reference Jenkins and Gauvreau²

Morbidity

Most evaluations of quality of care in cardiac surgery have been based on operative mortality.Reference Edwards, Grover, Shroyer, Schwartz and Bero^3, Reference Welke, Shen and Ungerleider⁴ De novo, the Aristotle complexity score, since 2002, has chosen to include morbidity in its definition of complexity.Reference Lacour-Gayet, Clarke and Jacobs⁵ Today, the operative mortality in congenital cardiac surgery, as observed in the congenital cardiac surgical databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery, is approximately 4%. No instrument currently exists to measure the quality of care delivered to the survivors, representing 96% of the patients. An objective assessment of morbidity is needed.Reference Lacour-Gayet, Jacobs, Jacobs and Mavroudis⁶ The following proposals were discussed in detail within the Society of Thoracic Surgeons Congenital Database Taskforce and also within The Multi-Societal Database Committee for Pediatric and Congenital Heart Disease.

Calculation of the morbidity core

The calculation of morbidity will be based on an algorithm that includes 3 components of measurable uses of resources of the hospital, and 1 component to represent the occurrences of major complications defined as either a temporary but with severe disability or a permanent disability (Table 1).

Table 1 Morbidity score components: ∑ (1+2+3+4).

*It is intentional that time on the ventilator is counted 2 times.

**It is intentional that assist device time is counted 3 times.

***The numerical value for each complication is not yet defined.

Hospital Resource components include:

• • Postoperative length of stay in hospital(s) prior to discharge to home

• • Postoperative time on the mechanical ventilator, from the start of surgery until the final extubation prior to discharge to home

• • Post-operative time on Extra Corporeal Membrane Oxygenation and/or Ventricular Assist Device, from initial cannulation to final decannulation.

Major complications include:

• • Re-operation during the same hospitalization (excluding sternal closure and procedures involving Extra Corporeal Membrane Oxygenation and/or Ventricular Assist Device)

• • Requirement for permanent pacemaker for atrio-ventricular block

• • Palsy of phrenic nerve, recurrent laryngeal nerve, or peripheral nerve

• • Permanent neurological lesions present at discharge from the hospital including stroke with clinical neurological deficit, choreoathetosis, epilepsy, blindness, deafness

• • Dialysis dependent chronic renal failure.

The numerical values for each component of the algorithm are under development.

Preliminary data

Only preliminary results of the morbidity assessment can be presented at this time because even though extra-corporeal membrane oxygenation procedures, ventricular assist device procedures, re-operations, and major complications are harvested by the congenital cardiac surgical databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery, a large amount of data is currently missing in these fields.

The sum of length of stay in the hospital and time on the ventilator was determined for 130 congenital cardiovascular procedures, using combined data involving more than 50,000 patients undergoing surgery in the years 2002 through 2005, inclusive, who were already present in the congenital cardiac surgical databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery. Procedures were excluded if they were classified as thoracic, extracorporeal membrane oxygenation, or ventricular assist device procedures. Thirteen additional procedures were excluded because of insufficient occurrences. The excluded procedures were represented by less than 17 cases in the combined database and exhibited 95% confidence intervals of greater than ±1 day. The remaining 117 procedures represented 90.0% of the primary cardiovascular procedures of interest, and more importantly included 94.9% of the patients. The morbidity values per procedure were plotted and the array was divided into five levels using natural break points and changes in slope (Fig. 4).

Figure 4 Median hospital length of stay plus median time on the ventilator in days is plotted for 117 procedures in ascending order. Light squares indicate data break points.

Each procedure was then assigned a morbidity score between 0.1 and 5.0. Zero morbidity did not exist but was assigned 0.1, and the highest morbidity value given 5.0. Within each level intervals were determined according to the slope of the morbidity curve to correspond with 0.1 increments in index. The plot of the morbidity score is illustrated in Figure 5 and the average morbidity value for each level is shown in Figure 6.

Figure 5 Derived morbidity scores for 117 procedures plotted in increasing order. Trend line reveals the linear approximation of the data array.

Figure 6 Average hospital length of stay plus time on the ventilator within five Morbidity Score levels.

Surgical difficulty

The evaluation of quality of care in congenital cardiac surgery needs a technical difficulty score so that the skill and effort of the surgical team is appropriately included in the evaluation of quality. It can be argued that although the time required to perform an individual operation is not a meaningful measurement; the overall average time to perform the same operation across the entire database may constitute an objective surrogate for the average technical difficulty of the operation relative to other procedures. This argument has not yet achieved consensus support.

It may be that the “skin to skin time” may need adjustment. For example, some procedures done without the use of cardio-pulmonary bypass are clearly more difficult than some simple open cardiac procedures which take as long or longer to perform. Perhaps additional time added to the duration of closed cardiac procedures would compensate for this perceived inequity. The authors are pursuing their investigation to find a valid technical difficulty index. Until a reasonable measurable parameter is settled upon, the technical difficulty score derived from subjective probability will serve as the surgical difficulty index.

Comments

The newly created assessments of mortality or morbidity associated with congenital cardiac surgery are compliant with the recommendations of The Society of Thoracic Surgeons and respect the criteria of the quality score published by the Quality Measurement Task Force of The Society of Thoracic Surgeons.Reference Shahian, Edwards and Ferraris^11, Reference O’Brien, Shahian and DeLong¹² These scores are also compliant with the recommendations of governmental agencies that assess and measure the quality of health-care. The National Quality Forum,⁹ the Agency for Healthcare Research and Quality, and the Institute of Medicine^{13, 14} recommend a standardized taxonomy for quality and safety and a precise methodology to evaluate performance. These recommendations were followed.

Evaluation of quality of care in congenital cardiac surgery is a work in progress and a learning process. This long enterprise has only been possible because of the creation of an International Nomenclature for Congenital Heart Surgery by Constantine Mavroudis and colleagues.Reference Mavroudis and Jacobs¹⁵ The administrative databases using the coding systems of the International Classification of Diseases and Current Procedural Terminology are insufficient to draw significant conclusions regarding mortality, morbidity, epidemiology, cost, and centre performance.Reference Welke, Peterson and Vaughan-Sarrazin¹⁶ The development of a cost index seems needed in the future. It will include all components of the morbidity calculation and an additional factors causing significant hospital cost increase.

The mortality score will be the first developed because of the concrete nature of the statistic and the availability of data about mortality prior to discharge from the hospital. A useable index will be available during 2008. The index will remain, however, a work in progress and will be updated when sufficient data about 30 day mortality is gathered.

The assessment of morbidity is a concept recently introduced in the evaluation of quality of care in our specialty. The Aristotle Complexity Score is the first benchmark of evaluation of quality of care in our specialty that includes morbidity.Reference Jenkins and Gauvreau^2–Reference Welke, Shen and Ungerleider⁴ It is no longer possible to ignore the quality of care delivered to the 96% of patients surviving surgery. With the prospect of pay-for-performance and cost evaluation looming, a quantification of morbidity is clearly needed. Since it is likely that it will be some time before the databases have enough data to produce the completed morbidity index, in the short term, in 2008–2011, we would propose use of available data in the registries such as post-operative length of stay and time on the ventilator, as a an objectively derived assessment, as shown in Figures 4, 5 and 6. Although not conceptually ideal, it may represent a reasonable surrogate for morbidity, and may well be a better quantitative indicator of true relative morbidity than the subjectively derived morbidity score currently in use.

Although the majority of the contribution of adverse events to morbidity is adequately reflected in the measurements of the different post-operative stages, some major complications with severe temporary disabilities, and permanent disabilities, represent a degree of patient suffering that is beyond the morbidity measured by hospital resources utilization. We chose to consider only major complications in order to simplify the score, following the example of the quality score of the adult cardiac surgeons that addresses coronary artery bypass surgery.Reference Shahian, Edwards and Ferraris¹¹ In reality, the less important complications are already measured through the utilization of hospital resource components. In calculating the hospital resource components, it is intentional that a period of time is counted two or three times, in an effort to weight the index relative to degree of patient suffering and utilization of resources. For example, the time on the ventilator is counted twice as it is also included in the hospital length of stay. Considering the cost of the intensive care unit and the suffering of the intubated patient, weighting this time twice seems justified. Similarly, the time of extra-corporeal membrane oxygenation and ventricular assist device will be counted three times; this calculation was considered justified for the same reasons. If a patient is extubated while on a ventricular assist device, then this time period will only be counted twice.

The mortality and morbidity scores will be part of the next generation of the Aristotle Complexity Score. The new complexity score will be named the Aristotle Average Complexity Score, which will be based on the sum of objective mortality, objective morbidity, and subjective technical difficulty scores.Reference O’Brien, Jacobs and Clarke^1, Reference Lacour-Gayet, Jacobs and Clarke^8, Reference Jacobs, Lacour-Gayet and Jacobs¹⁷