The year 2008 marks the beginning of the eighth decade of the history of congenital cardiac surgery as we know it. The infancy of this field of endeavour was proclaimed with the first successful surgical closure of a patent arterial duct by Dr. Robert Gross in 1938. Throughout most of the early history of our discipline, the process of acquisition of clinical data and the analysis of this data was an activity undertaken by individual surgeons or surgical units. Given the fact that surgery for congenital cardiac disease was an entirely new discipline, a great deal of innovation was involved, and previously untested therapies became the subject of published case reports, most often when they were associated with positive outcomes. This initial phase of reporting the early results of new surgical procedures and techniques was followed by a period which saw the development of many excellent single institutional surgical databases, some of which proved to be tremendously valuable, above and beyond serving the needs of the individual institution. Examples of such databases are those of the Cardiac Surgical Units of the University of Alabama at Birmingham and of the Green Lane Hospital in Auckland, New Zealand. The wealth of information in the first edition of the classic textbook Cardiac Surgery,Reference Kirklin and Barratt-Boyes1 edited by John Kirklin and Sir Brian Barrat-Boyes, is a direct result of the disciplined process of acquisition of data and the continual analysis of this data that was a priority of those surgical masters. Other individual units established and maintained databases of comparable quality.
For a discipline where the surgical procedures were, and still are, constantly evolving, the ability of practitioners to share data and compare outcomes with colleagues in other institutions using similar or different therapeutic approaches was of obvious importance. This need led to multi-institutional collaboration and the development of research databases such as that of the Congenital Heart Surgeons’ Society in North America. This subspecialty society, which has grown over nearly three decades from 16 to now close to one hundred surgeon participants, has since the middle of the decade of the 1980s maintained a multi-institutional database of several cohorts of patients that are each established based upon a fundamental anatomic diagnosis or specific surgical procedure.Reference Jacobs, Ungerleider and Tchervenkov2 The first landmark reports from this group compared the outcomes of arterial switch operations to atrial switch procedures for transposition of the great arteries at a time when anatomic repair of transposition of the great arteries was an innovative strategy not yet embraced by all surgeons and cardiologists.Reference Trusler, Castaneda, Rosenthal, Blackstone and Kirklin3, Reference Castaneda, Trusler, Paul, Blackstone and Kirklin4 A great deal has been learned from analysis of research databases, such as those of the Congenital Heart Surgeons’ Society. But, as clearly articulated by Dr. William Williams of Toronto, past president of the Congenital Heart Surgeons’ Society, the scope and objectives of these research databases are defined by the principle of establishing an inception cohort, based on a particular diagnosis or procedure, and aggregating as much information as possible, concerning a relatively small universe of patients. The acquisition of more generalizable information concerning the efficacy of surgical strategies of treatment, their impact on the survival and status of patients, and the evaluation of the performance of surgeons and centres in the management of patients with the full spectrum of congenital cardiac diseases, is the goal of “Registry Databases” which are distinguished in principle from “Research Databases” in that they are designed to catalogue essential information, in less voluminous detail per patient than is practical in a research database, but with the goal of having this information on all patients. Thus, a “Research Database” strives to obtain “all of the data about some of the patients”, while a “Registry Database” strives to obtain “some of the data about all of the patients.”Reference Williams and McCrindle5
During the 1990s, both The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery initiated the establishment of “Research Databases” to document the outcomes of patients undergoing surgery to treat congenital cardiac disease. In 1998, the first reports of the Society of Thoracic Surgeons National Congenital Heart Surgery Database Committee were published6, Reference Mavroudis, Gevitz, Rings, McIntosh and Schwartz7 and included data from 24 centres that joined the program at various dates of entry between 1994 and 1997. There were 19,894 enrolled patient records, from which 8149 patient records were used to compile relevant clinical features of patients in 18 categories of congenital cardiac disease. Data about outcomes included multiple fields such as operative death, complications, and length of stay. Analyses of outcomes were segregated by age and weight at operation where appropriate, which varied from diagnosis to diagnosis. The data analysis was largely descriptive in character. During the same time period, the European Congenital Heart Defects Database was founded through the European Congenital Heart Surgeons Foundation, which has subsequently been renamed as the European Congenital Heart Surgeons Association. By 1995, the European Congenital Heart Defects Database had collected data from 31 centres in 18 countries. Like the first report of the Society of Thoracic Surgeons National Congenital Heart Surgery Database Committee, the data analysis in the initial reports of the European Congenital Heart Defects Database was largely descriptive. With respect to both of these early efforts, the data sets were difficult to manage and the software was rudimentary by current standards, lacking the flexibility to facilitate creative approaches to analysis.
Nevertheless, important lessons were learned from these early efforts at establishment of national or regional multi-institutional “Registry Databases”. Most important among these lessons was the eventual recognition of five fundamental elements that are essential to success and accuracy in achieving meaningful multi-institutional outcomes analysis:
• a common language or nomenclature, acceptable and familiar to all participants,
• a mechanism of data collection, a “Registry Database” with an established uniform core dataset,
• a mechanism of evaluating the complexity of the operations,
• a mechanism to ensure and verify the completeness and accuracy of the data, and
• a platform that lends itself to collaboration between medical and surgical subspecialties.
These important lessons were learned in a timeframe where events around the world drew considerable attention to the pressing need for fair and accurate analysis of outcomes of congenital cardiac surgery. Events at Bristol, England,8 Denver, Colorado, United States of America,9 Winnipeg, Canada,10 and elsewhere, led to investigations of the quality of care of select groups of patients who had undergone surgery for congenitally malformed hearts. While each series of events was unique with respect to the others, investigation of each circumstance led to the common finding of a need for accurate multi-institutional databases to quantitate outcomes of care rendered to patients with congenital cardiac diseases, and to facilitate programs of quality-assessment and quality-improvement. Furthermore, these events, and the sometimes misleading reporting of selected data of uncertain quality, further emphasized the importance of physicians, through their professional societies, taking up the mantle of responsibility with respect to the analysis and reporting of outcome data concerning the treatment of patients with congenital cardiac diseases. As stated by Dr. William Williams at the 2004 meeting of the Society of Thoracic Surgeons, “Outcomes for cardiac surgery are closely scrutinized and expectations are very high. It is timely that we as a profession develop a report card for congenital heart surgery. The report card must be timely, freely available, and fairly represent case mix… .of the wide spectrum of congenital heart disease that we treat.”Reference Williams11 But, in fact, a need exists for much more than simply a report card, and the opportunity exists to accomplish so much more in the realms of education, research, the allocation of resources, the analysis of outcomes, and the improvement of quality. These objectives are among the goals of the development, maintenance and continual improvement of databases to promote the analysis of outcomes of the treatment of patients with congenital cardiac diseases.
Common language = Nomenclature
The International Congenital Heart Surgery Nomenclature and Database Project was initiated in September 1998. The efforts of this group led to the publication in April 2000Reference Mavroudis and Jacobs12 of a common nomenclature and a common core minimal data set that were enthusiastically accepted by both the Society of Thoracic Surgeons and the European Association for Cardiothoracic Surgery, which was by this time, in collaboration with the European Congenital Heart Surgeons Association, responsible for the European Congenital Heart Defects Database. The system of nomenclature is based upon several guiding principles:
• the identification and classification of fundamental cardiac phenotypes
• the definitions and recognition of similar terms, or synonyms, in the lexicons of taxonomy of congenital cardiac defects and procedures,
• a hierarchical system of Diagnostic and Procedural Long Lists, whereby the user can achieve a desired level of specificity,
• the simultaneous utilization of Diagnostic and Procedural Short Lists, which facilitates the easy categorization and grouping of lesions according to major anatomic and diagnostic categories.
Thus, for the first time, congenital cardiac surgical programs at centres in both North America and Europe, as well as participating centres in Asia and elsewhere, established, through their respective professional societies, a policy endorsing the use of a common system of nomenclature to describe congenital cardiac diseases or defects and the therapeutic procedures used to treat them.
The Association for European Paediatric Cardiology also published in 2000 a suggested nomenclature system named the European Paediatric Cardiac Code.Reference Franklin, Anderson and Daniëls13 The nomenclature system developed by The International Congenital Heart Surgery Nomenclature and Database Project, and the European Paediatric Cardiac Code, each include both a Short List and a Long List. The Short Lists facilitate the creation of multi-institutional outcomes registries. The Long Lists support the creation of echocardiography software, academic databases, and the electronic record, among other applications. The acknowledgement that the two systems of nomenclature developed separately but nearly simultaneously should be complementary, rather than competitive, led to the development of an International Working Group for Mapping and Coding of Nomenclatures for Paediatric and Congenital Heart Disease, which has systematically taken on the task of cross-mapping first the Short Lists and then the Long Lists from each nomenclature system, one to the other. This process of bidirectional cross-mapping led to the development of a single “super-tree” of nomenclature, known as the International Paediatric and Congenital Cardiac Code.Reference Béland, Franklin and Jacobs14 This code, and the cross map of its Short Lists and Long Lists, are available at no charge via the Internet at [www.IPCCC.NET]. Two dominant versions of the International Paediatric and Congenital Cardiac Code are available:
• The International Paediatric and Congenital Cardiac Code derived from the European Paediatric Cardiac Code of The Association for European Paediatric Cardiology;
• The International Paediatric and Congenital Cardiac Code derived from the International Congenital Heart Surgery Nomenclature and Database Project of The Society of Thoracic Surgeons, The European Congenital Heart Surgeons Association, and The European Association for Cardio-Thoracic Surgery.
Both versions have been utilized by numerous professional societies, and government agencies in Europe and North America for the establishment of “Registry Databases” and an increasing number of research studies. The version derived from the International Congenital Heart Surgery Nomenclature and Database Project has been used to analyze the outcomes of over 120,000 patients undergoing surgery for congenital cardiac diseases in the congenital databases of The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons. The version derived from the European Paediatric Cardiac Code has been used within European national registries for purposes of clinical governance, such as described below for the United Kingdom.
Mechanism of collection of data
The common nomenclature and common minimum data set derived from the work of the International Congenital Heart Surgery Nomenclature and Database Project have served as the platform for development of the fields of data that are essential elements of the software available to users of the congenital databases of The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons. To use these databases, institutions or surgical groups enter into an agreement of participation with their respective professional societies, and utilize, as a platform for storage and retrieval of data, computerized software that is made available either through the professional society, or is commercially or privately available but certified as compatible with the applicable standards of the respective professional society.
On an annual basis, the task force or committee responsible for the database of each Society issues to each participating institution a report consisting of aggregate data from all participating groups and institutions, de-identified with respect to source, and of data specific to the participant. Data is organized according to the diagnoses and procedures that make up the Short Lists. These Short Lists consist of 164 diagnoses and 204 procedures. The dataset is also segregated by age group into neonates, infants, children, and adults with congenital cardiac disease. The feedback report for the individual participating institution also includes a detailed analysis of outcomes of patients in the most populous diagnostic groups, and comparable analysis of these diagnostic groups in the aggregate of all participants. Thus, each participant is in possession of a report of outcomes encompassing all of their annual activity, as well as cumulative activity over the years of participation. The participant, therefore, has sufficient information to identify trends in their own practice, with respect to prevalence of diagnoses and procedures, and measures of outcome such as mortality, complications, length of stay, and utilization of resources. Moreover, they have sufficient data to make comparisons of their own institutional data to the aggregate data from all participants. The scope of these activities is illustrated by the observation that the most recent report of the Congenital Database of The Society of Thoracic Surgeons, released in 2007, included 61,014 operations submitted from 58 centres in North America. The most recent report of the congenital database of The European Association for Cardio-Thoracic Surgery, meanwhile, included 61,750 operations from units in 62 countries.
The Congenital Database of The European Association for Cardio-Thoracic Surgery
In 1992, in the spirit of cooperation and critical evaluation of the outcomes in congenital cardiac surgery, the European Congenital Heart Surgeons Foundation, later renamed the European Congenital Heart Surgeons Association, decided to establish the European Congenital Heart Defects Database. The goal of this initiative was to compare results of surgery between countries, units, and individual surgeons, and define the areas of weakness to enable continuous improvement of outcomes. The European Congenital Heart Surgeons Foundation initially located the operations of its database at The Great Ormond Street Hospital for Children, in London, United Kingdom, under supervision of Martin Elliott. Data on 16,000 congenital heart surgery procedures had been collected by 1998, when the registry was moved to the Department of Cardiothoracic Surgery at the Children’s Memorial Health Institute, Warsaw, Poland, under responsibility of Bohdan Maruszewski. In 1999, during the Annual Meeting of The European Association for Cardio-Thoracic Surgery in Glasgow, Scotland, it was decided that European Congenital Heart Defects Database would become the congenital database of The European Association for Cardio-Thoracic Surgery, under the joint leadership of The European Association for Cardio-Thoracic Surgery and the European Congenital Heart Surgeons Foundation.
Following two years of efforts by congenital heart surgeons and cardiologists representing The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons, in 2000, the International Congenital Heart Surgery Nomenclature and Database Project published the nomenclature and database standards used in the “twin” congenital databases of these organizations on both ends of the Atlantic Ocean. This published system of nomenclature includes a minimal dataset that includes five Short Lists that have periodically been revised and upgraded by agreement of both The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons:
• Noncardiac Abnormalities
• Preoperative Risk Factors
• Diagnosis
• Procedures
• Complications.
By January 1, 2008, the congenital database of The European Association for Cardio-Thoracic Surgery contains 61,750 operations performed in 53,402 patients including 12,109 operations in neonates, 20,487 in infants, 25,102 in children and 4,052 in adults. The registry grows continuously and recently shows between 5 and 10 thousand new operations each year. 274 Units from 62 countries are registered and have access to over 300 on line reports. Since 2003, the Aristotle Basic Complexity Score, developed by Francois Lacour-Gayet and an international panel of experts, as discussed below, has been utilized as a tool to stratify the complexity of operations. The “bubble graphs”, developed and published earlier, use the Aristotle Basic Complexity Score for comparison of data about volume and outcome between surgeons and institutions. In these graphs, the diameter of the bubble is directly proportional to the volume of operations performed by the programme or surgeon under analysis. In 2004, the Source Data Verification Project was started.Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy15 During 24 site visits, 18.26% of data collected from 2003 to 2006, involving 5,892 operations, underwent verification procedures. In 2007, the legal document defining the Database Rules and protecting both the data and the management of the database, was finalized and published on the website of the database. Figs 1, 2, 3, 4, 5, 6 provide representative data from the congenital database of The European Association for Cardio-Thoracic Surgery. Similar data can also be obtained from the congenital database of The Society of Thoracic Surgeons.
Figure 1 Growth in the congenital database of The European Association for Cardio-Thoracic Surgery.
Figure 2 Standard report of the congenital database of The European Association for Cardio-Thoracic Surgery for all patients.
Figure 3 Trends in the congenital database of The European Association for Cardio-Thoracic Surgery for mortality prior to discharge from the hospital for neonates.
Figure 4 Mortality in the congenital database of The European Association for Cardio-Thoracic Surgery for the Norwood Stage 1 operation.
Figure 5 Bubble graph showing mortality versus volume versus Aristotle Basic Complexity Score for Institutions submitting data to the congenital database of The European Association for Cardio-Thoracic Surgery.
Figure 6 Results of verification of data in the congenital database of The European Association for Cardio-Thoracic Surgery.
The Congenital Database of The Society of Thoracic Surgeons
As of July 7, 2008, the three databases of The Society of Thoracic Surgeons have a cumulative participation of 1097 sites, including 919 sites participating in the Adult Cardiac Database, 106 sites participating in the General Thoracic Database, and 72 sites participating in the Congenital Database. This current level of participation in the Congenital Database of The Society of Thoracic Surgeons represents more than half of the centres in the United States of America performing surgery for congenital heart diseases. (The Report of the 2005 Society of Thoracic Surgeons Congenital Heart Surgery Practice and Manpower Survey, undertaken by the Society of Thoracic Surgeons Workforce on Congenital Heart Surgery, documented that 122 centres in the United States of America and 8 centres in Canada perform paediatric and congenital heart surgery.Reference Jacobs, Mavroudis, Jacobs, Tchervenkov and Pelletier16) The congenital database of The Society of Thoracic Surgeons was founded in the early 1990s under the leadership of Gus Mavroudis and is now the largest multi-institutional registry of patients undergoing surgery for congenital cardiac disease in North America. The Spring 2008 annual report of the Congenital Database of the Society of Thoracic Surgeons included data from 68 of the 130 centres in North America. The Congenital Database of the Society of Thoracic Surgeons has grown annually since its inception, both in terms of the number of participating centres submitting data, and the number of operations analyzed (Figs 7 and 8). The report from the 2008 harvest of data from the Congenital Database of The Society of Thoracic Surgeons17 includes 72,002 operations performed in 68 centres in North America, 67 from the United States of America and 1 from Canada. One Japanese centre also submits data; however, these Japanese data are not included in the aggregate report produced by the Society of Thoracic Surgeons.
Figure 7 Growth in the Society of Thoracic Surgeons’ Congenital Database – participating centres.
Figure 8 Growth in the Society of Thoracic Surgeons’ Congenital Database – operations.
The feedback reports of the Congenital Database of the Society of Thoracic Surgeons allow individual programs to compare their outcomes to the aggregate of outcome in the database. Figures 9 and 10 are taken from the 2007 Report of the Congenital Database of the Society of Thoracic Surgeons18 and demonstrate a technique that allows one to identify outliers without creating a “League Table” that ranks programs when no true difference exists. These graphs also provide a real world example of the importance of the adjustment for case-mix. The feedback reports of the Congenital Database of the Society of Thoracic Surgeons present this type of programmatic data about mortality for all patients in the last calendar-year and all patients in the last four calendar-years. The data is further presented in charts and graphs that break the analysis into all 4 Levels of the Aristotle Basic Complexity Score and all 5 functional levels of the Risk Adjustment in Congenital Heart Surgery-1 method, as described below, for both the last calendar-year and the last four calendar-years. Finally, this data is also broken down into 4 age groups: neonate, infant, child, and adult with congenital cardiac disease, again presented for all patients and broken down into all 4 Levels of the Aristotle Basic Complexity Score and all 5 functional levels of the Risk Adjustment in Congenital Heart Surgery-1 method, for both the last calendar-year and the last four calendar-years.
Figure 9 Discharge mortality.
Figure 10 Discharge mortality – by RACHS-1 Score 5-6.
A casual examination of Figure 9 might suggest that Hospital number 37 achieved the highest level of performance, because it has the lowest overall mortality; however, Figure 9 documents raw mortality without any adjustment for case-mix. Applying stratification for complexity, Figure 10 reveals that Hospital number 37 has no patients in the category of highest risk. Thus, their very low overall mortality must be considered in the context of a case-mix that includes zero Norwood Stage I palliations, or operations of comparable complexity.
Together, the congenital databases of The Society of Thoracic Surgeons and The European Association for Cardio-Thoracic Surgery now contain data on 122,764 operations. These two “sister databases” use the same nomenclature, database standards and definitions, tools for stratification of operative complexity, and methods of verification of data. Multiple publications generated from these two databases have reported outcomes after treatment for congenital cardiac disease in general, as well as outcomes for specific lesions.Reference Jacobs, Mavroudis and Jacobs19–Reference Jacobs, Jacobs and Maruszewski21 The concepts and principles discussed in the preceding section of this article titled “The Congenital Database of The European Association for Cardio-Thoracic Surgery” also apply to the congenital database of The Society of Thoracic Surgeons and will therefore not be repeated.
The Pediatric Cardiac Care Consortium
Under the leadership of James H. Moller, MD, and based out of the University of Minnesota, the Pediatric Cardiac Care Consortium (PCCC) was one of the first large scale multi-institutional databases for congenital heart disease. Founded in 1982, the PCCC is a collaborative, voluntary effort of pediatric cardiologists to gather and analyze data regarding operative results.Reference Moller, Powell, Joransen and Borbas22 The PCCC collects information on each child who undergoes cardiac catheterization, electrophysiological study, or a cardiac operation, or dies with a cardiac malformation. The data are analyzed annually and individual reports are created for each centre. Representatives from the centres meet annually, and data on the major operative procedures, including risk factors, patient profiles, and variations in adjusted mortality-are presented. By July 1, 2003, the PCCC collected data about congenital cardiac surgery and interventional cardiology from 47 centres across North America.23 An advantage of the PCCC is that it provides data about surgery and transcatheter interventions within the same database. One weakness of the data from the PCCC is the lag time from the actual clinical event to the release of the data. For example, the 2007 report of the Pediatric Cardiac Care Consortium only contained data through 2005. For comparison, the 2007 report of the Congenital Database of the Society of Thoracic Surgeons contained 2006 data and is released in the summer of the year so that it includes data from within the past 6 months. Going forward in 2008, the harvest cycle of the Congenital Database of the Society of Thoracic Surgeons will be shortened from one year to six months, so that two reports will be produced each year.
The Central Cardiac Audit Database in the United Kingdom
The Central Cardiac Audit Database in the United Kingdom is used to assess outcomes after therapeutic procedures in the United Kingdom and represents an excellent example of what can be achieved at a national level to monitor surgical and transcatheter cardiovascular interventions undertaken on patients with congenitally malformed hearts. This database was established in 1999 by the British Cardiac Society, the Society of Cardiothoracic Surgeons, and the British Paediatric Cardiac Association, after the Kennedy report on the results of infant congenital cardiac surgery in Bristol, United Kingdom underscored the need for a national system to analyze the outcomes after cardiovascular surgery and therapeutic catheterization in the young.8 The development of the Central Cardiac Audit Database involved the establishment of a team of experts to set up computerized registries with access to sophisticated analyses of anonymised data, inclusive of robust protocols for the protection and validation of data. Part of the methodology was to audit and compare levels of performance, so as to set standards of optimal care as a benchmark for individual hospitals. Units found to be relatively underperforming would receive constructive feedback, which might focus, for example, on surgical techniques, intensive care support, or shortcomings in the ‘system’ or infrastructure.
The initial system consisted of tracking of mortality using only a standardized minimum dataset of 20 fields. This system was unchanged for the first two years. A gradual expansion of fields then occurred, to include whether there was a fetal diagnosis in 2003, and the monitoring of outcomes related to morbidity a year later. Outcomes now include 30 day, in hospital and 1 year “alive or dead status”, the length of stay in the hospital, and the time to extubation. The acquisition of local data at the point of delivery was found to be essential to ensure timely and comprehensive collection of data on all cases, as was the presence of an “audit facilitator” to encourage the clinicians and to validate the quality of data before submission. Currently data are submitted electronically in an anonymous encrypted format with prospective tracking of mortality and re-intervention using up to a 40 field minimum dataset. The database is centrally funded by the Department of Health and data submission is compulsory for all centres undertaking interventions on patients with congenital cardiac malformations. Patients give informed consent for data submission.
As detailed above, a common clinical language is fundamental for success and the Short List of the European Paediatric Cardiac Code has been employed since 2003.Reference Franklin, Anderson and Daniëls24 The European Paediatric Cardiac Code is mapped to the 10th revision of the International Classification of Diseases, as provided by the World Health Organisation, for diagnoses, and the 4th revision of the United Kingdom specific procedure codes, as provided by the Office of Populations Censuses and Surveys, for central government returns and ‘billing’. Independent validation of the status of the patient as alive or dead is achieved by central mortality tracking using the linkage of the National Health Service number of the patient to the Office of National Statistics, where the death of every resident in England and Wales is registered. A separate, similar system is used in Scotland. In addition, annual visits for the validation of data are undertaken to each hospital submitting data to ensure accuracy of the data and that all procedures undertaken have been captured, as detailed elsewhere in this Supplement.Reference Clarke, Breen, Jacobs, Franklin, Tobota and Maruszewski25
Figure 11 documents the overall numbers of surgical and transcatheter cardiovascular procedures undertaken in the United Kingdom from 2000 to 2006, with 30 day and 1 year survival.26 Over 26,000 surgical procedures have been amassed at a current rate of over 4,500 each year. The number of new surgical procedures done each year is fairly constant and reflects the number done in the United Kingdom. The increase in surgical volume over time reflects capturing more data about adults with congenital cardiac disease over recent years.
Figure 11 Cardiovascular procedures for congenital heart disease undertaken in the United Kingdom from 2000 to 2006 with 30 day and 1 year survival. Data Source: [http://www.ccad.org.uk/002/congenital.nsf/WMortality?Openview]. Accessed 17 August 2008 for Figure 11 (reproduced with permission). The authors thank Dr. David Cunningham of the National Health Service Information Centre in the United Kingdom for permission to reproduce the data from the Central Cardiac Audit Database shown in Figure 11 and the funnel plot in Figure 12 from the Central Cardiac Audit Database.
The results to date have been gratifying in that there has been no statistical difference found in 30-day, in-hospital and one year post-procedural outcomes between the 14 centres undertaking paediatric congenital cardiac procedures in the United Kingdom.26–Reference Spiegelhalter29 This information has recently been published on the world wide web, with free access to families and the media, providing details of the outcomes after 29 major surgical procedures and 10 transcatheter procedures for the years 2000 through to 2006.26, 27 It is presented with procedure and centre specific outcomes both in tabular and graphical format, using funnel plots, which plot rates of death against the number of cases with superimposed ‘control limits’ of two and three standard deviations below the mean, as exemplified in Figure 12.
Figure 12 Funnel survival plot of the 757 arterial switch procedures for isolated transposition of the great arteries undertaken in the United Kingdom between 1st April 2000 and March 31st 2005. Data Source: [http://www.ccad.org.uk/002/congenital.nsf/0/16AFD27D0A18BD8D802573D3005CEE6D?OpenDocument?Benchmark]. Accessed 9 February 2008 for Figure 12 (reproduced with permission). The authors thank Dr. David Cunningham of the National Health Service Information Centre in the United Kingdom for permission to reproduce the data from the Central Cardiac Audit Database shown in Figure 11 and the funnel plot in Figure 12 from the Central Cardiac Audit Database.
Figure 12 documents the funnel survival plot of the 757 arterial switch procedures for isolated transposition of the great arteries undertaken in the United Kingdom between 1st April 2000 and March 31st 2005. This graph shows the national average survival as a horizontal grey line. Two control limits are shown: a warning limit
• a green line for the 98% confidence limit, representing a “warning limit”, and
• a red line for the 99.5% confidence limit, representing an “alert limit”.
Unit performances are shown as identifiable coloured symbols. If the symbol for a unit is above the green line, then the performance is no different from the national average. If the symbol for a unit is below the warning limit, their performance will be closely monitored in subsequent years. If the symbol for a unit is below the alert limit, an investigation into possible reasons and remedial actions will be launched by the appropriate professional and regulatory bodies.27, Reference Spiegelhalter29
Using this methodology, the spurious ranking of the centres is avoided, whilst procedural complexity and the volume of cases is taken into account.Reference Spiegelhalter29 Hospitals falling outside these limits would then incur further investigation. This reporting to the public of data about outcomes, combined with the knowledge that central tracking of mortality will externally monitor performance, provides added incentive to provide accurate and complete data.
Mechanism of evaluating the complexity of cases
In evaluation of the outcome of surgical management of patients with cardiac diseases, analysis and reporting based on measures of raw mortality alone, without stratification of the complexity of cases, is inadequate. The field of cardiac surgery for adults with acquired diseases of the heart involves a relatively small number of different operative procedures, primarily coronary artery bypass grafting and repair or replacement of diseased heart valves. Risk-modelling, based empirically on observed outcomes, and application of conventional statistical methods, has led to the acceptance of processes of “risk-adjustment,” which facilitate comparison of “observed results” to “expected results,” based on a number of risk-factors of established statistical significance. For the field of surgical treatment of congenital diseases of the heart and great vessels, the problem is considerably more complex. While the universe of patients is considerably smaller than that of adults with acquired diseases, the list of individual anatomic diagnoses and the number of individual or combined operative procedures used to treat them is, by comparison, immense. And, the lists include scores of diagnoses and procedures that are relatively rare. Case-mix can vary greatly from centre to centre, and from one time period to another for a given centre. As the outcomes of extremely complex cases are likely to be less consistently favourable than those of cases of lesser complexity, the reporting of outcomes based upon raw mortality, without a quantitative measure of the relative complexity of cases, is inevitably incomplete and will be misleading.
The recognition of this problem led to the development, nearly simultaneously, of two different systems of stratification of operative procedures for congenital cardiac diseases: the Risk Adjustment in Congenital Heart Surgery-1 method,Reference Jenkins, Gauvreau, Newburger, Spray, Moller and Iezzoni30, Reference Jenkins31 and the Aristotle Complexity Score.Reference Lacour-Gayet, Clarke and Jacobs32, Reference Lacour-Gayet, Clarke and Jacobs33 Under the leadership of Kathy Jenkins M.D, from Children’s Hospital Boston, the Risk Adjustment in Congenital Heart Surgery-1 method was developed to adjust for baseline case-mix differences when comparing discharge mortality for groups of patients undergoing surgery for congenital cardiac diseases. The system was created using a combination of judgment- based and empirical methodology. An eleven-member panel of pediatric cardiologists and cardiac surgeons grouped surgical procedures into six risk categories based on expected discharge mortality. Categories were then refined using empirical data from two large datasets, one from the Pediatric Cardiac Care Consortium and the other generated from state-wide administrative hospital discharge databases. The Risk Adjustment in Congenital Heart Surgery-1 method has been demonstrated to be a useful tool in numerous studies in both Europe and North America and represents one of the first widely accepted risk adjustment tools developed in our field of clinical endeavour.
In 1999, under the leadership of François Lacour-Gayet M.D., the Aristotle Committee was created to address the issue of adjustment for operative complexity in congenital cardiac surgery. This group was composed of experts, made up of 50 surgeons from 23 countries, and representing multiple professional societies. The purpose of the project was to develop a tool for stratification of complexity, which could be used to compare equitably outcomes and assess performance of centres performing surgery for congenital cardiac diseases. The Aristotle Basic Complexity Score allocates a score between 1.5 and 15, to each primary operative procedure of the Short List of the International Congenital Heart Surgery Nomenclature and Database Project. The Aristotle Basic Complexity Score for each procedure is calculated based upon three factors: the potential for mortality, the potential for morbidity, and the technical difficulty of the operation. Each procedure is also assigned to an Aristotle Basic Complexity Level, ranging from 1 to 4. The Aristotle Basic Complexity Level provides a broad generalization of complexity, while the Aristotle Basic Complexity Score allows more precise stratification of complexity. Beyond this, an additional tool, the Aristotle Comprehensive Complexity Score, adds to the Basic Score by incorporating additional points for two categories of specific complexity modifiers:
• procedure-dependent factors, including anatomical factors, associated procedures, and age at procedure; and
• procedure independent factors, including general factors such as weight and prematurity, clinical factors such as preoperative sepsis or renal failure, extracardiac factors such as duodenal atresia and imperforate anus, and surgical factors such as reoperative sternotomy.
Like the Risk Adjustment in Congenital Heart Surgery-1 system, the Aristotle Complexity Score has been widely accepted and been used extensively as a tool in clinical research and the analysis of outcomes.Reference Jacobs, Lacour-Gayet and Jacobs20 At the present time, analysis of outcomes based upon both these systems, the Risk Adjustment in Congenital Heart Surgery-1 system and the Aristotle Complexity Score, is incorporated in the yearly reports of the congenital databases of both The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons.
Mechanism to assure and verify the completeness and accuracy of the data
As we move forward with efforts to measure outcomes and improve the care of our patients, assurances of the completeness, accuracy, and quality of the data are central issues. Collaborative, and parallel, efforts, involving The European Association for Cardio-Thoracic Surgery and The Society of Thoracic Surgeons, are underway to achieve these goals through processes of periodic on-site evaluation of data to ascertain accuracy and completeness, and to identify areas for improvement in management of the database.Reference Maruszewski, Lacour-Gayet, Monro, Keogh, Tobota and Kansy15 These processes, while costly and labour-intensive, are of utmost importance. This importance is amply demonstrated by a recent prospective, observational national cohort survival study from the United Kingdom Central Cardiac Audit Database, whose methodology is described above. The analysis focused on outcomes of surgical procedures and therapeutic cardiac catheterizations at thirteen centres from 2000 and 2001. Thirty day mortality was identified both by volunteered life status from the hospital databases and by independently validated life status through the Office of National Statistics. In the final analysis, hospital-based databases underreported 30-day mortality by 21.6%, even though the hospitals were aware that the data would be independently verified. The authors concluded that “independent data validation is essential for accurate survival analysis” and that “one-year survival gives a more realistic view of outcome than traditional peri-operative mortality.”Reference Gibbs, Monro, Cunningham and Rickards28 A combination of site visits with on-site verification of data, and external verification of data from independent databases such as national registries of death, may ultimately be required to achieve optimal verification of data.
Collaboration between medical and surgical subspecialties
A great deal can be learned from careful evaluation of accurate data concerning the operative experience and the “traditional or conventional” period of post-operative recovery. This “traditional or conventional” operative period of time includes both of the following time periods:
• within 30 days after surgery or intervention in or out of the hospital, and
• after 30 days during the same hospitalization subsequent to the operation or intervention.
But the continuum of care stretches far beyond these arbitrary windows in time. In addition, the management of patients undergoing treatment for congenital diseases of the heart is truly multi-disciplinary. In the final analysis, the assessment of outcomes and the efforts to improve quality-of-care that follow from this assessment should be hampered neither by arbitrary limits of time nor by barriers that impede transfer and sharing of information between the multiple categories of specialists involved in the care of patients. The formation of the MultiSocietal Database Committee for Pediatric and Congenital Heart Disease represents an initial effort by an international consortium of clinicians and investigators from several disciplines to begin collaborative efforts to approach the goal of creating databases for the various subspecialties that can “talk to each other.” During 2006 and 2007, the group focused on the establishment of mutually acceptable definitions of peri-operative complications. That effort will be followed by a similar collaboration to create common ground for the evaluation and reporting of preoperative risk-factors.
The future
The past decade has been marked by a great deal of progress in the development of national and multi-national databases for congenital cardiac surgery. These registries have been acknowledged and accepted as essential tools for efforts directed at the assessment and improvement of quality. This experience has also led to the discovery of several key areas where there is a need for further development. Completeness and verification of the data, and multi-societal collaboration, are discussed above. The development of mechanisms to track the course of a given patient through multiple interventions, and multiple admissions to the hospital, even at multiple institutions, has been recognized as another important objective. The accomplishment of this goal, while respecting all of the governmental regulations that protect confidentiality and privacy of the patient, is challenging but feasible. And it is recognized that evaluation of outcomes, traditionally based upon reporting of mortality, looks at only a small part of the big picture. While reduction of operative mortality is an important goal, it is fortunately the case that approximately 95% of operations for congenital cardiac diseases are attended by survival of the patient. The elements of a database that facilitate the analysis of complications, short and long term morbidities, resource utilization, and ultimate quality of life of the patient deserve as much attention and development as those that focus on operative mortality. Finally, the goals of a “Registry Database” are truly accomplished only when participation is complete. At the present time, more than one half of the centres performing surgery for congenital cardiac diseases in the United States participate in the Congenital Database of The Society of Thoracic Surgeons. In Europe, comprehensive national databases are being established within the congenital database of The European Association for Cardio-Thoracic Surgery based on the requests of several countries. The United Kingdom has its own independent national database, named the United Kingdom Central Cardiac Audit Database. The organizers of each of these enterprises are working constantly to improve the quality and utility of their Congenital Heart Surgery Databases. Efforts are ongoing to further standardize definitions within the databasesReference Jacobs, Mavroudis and Jacobs34, Reference Jacobs, Jacobs and Mavroudis35 and to improve the tools for adjustment of complexity.Reference O’Brien, Jacobs and Clarke36 Programs of verification of data are being expanded. Collaborative efforts with individuals in other subspecialties, such as cardiology, cardiac anaesthesia, and critical care, expand the breadth and relevance of the surgical databases. And the quality of the feedback reports to participant centres is constantly being upgraded with more data, expanded graphics, and refined statistical analysis. Improvement of the quality of care is virtually impossible without accurate and timely data to serve as the platform upon which our efforts to better the outcomes for our patients are constructed.
Acknowledgement
We thank The Children’s Heart Foundation (http://www.childrensheartfoundation.org/) for financial support of this research. The authors thank Dr. David Cunningham of the National Health Service Information Centre in the United Kingdom for permission to reproduce the data from the Central Cardiac Audit Database shown in Figure 11 and the funnel plot in Figure 12 from the Central Cardiac Audit Database.
