Background
The mission of most medical societies is to provide the highest quality care to patients through education, research, and advocacy. With the advent of “web-based” medicine and research about outcomes, this mission has expressed itself in many societies by the formation of databases or registries. The advantages of such initiatives are numerous and obvious such as assessment of therapies, education of patients, and interactions with third parties including industry, payers, and government. Databases for societies with procedure-based specialties such as surgery will help with the care of patients by predicting prognosis, defining risk-factors, and aiding with the selection of patients who are the best candidates for these procedures. These databases eventually will also help to establish standards of care. The latter is rapidly growing in importance as governments attempt to create “pay-for-performance” programs in many of the surgical specialties. If societies do not create their own accurate databases that they can direct, then the government will use available databases from which they will infer. These inferences are often not representative of the population of patients or their outcomes.
A common issue with the registries of surgical specialties is that after the short-term post-operative period of care, the surgeon is no longer the primary provider of care for the patient. The ability for surgeons to track their patients over the long-term is limited; and therefore, the ability of surgeons to create databases with long-term follow-up has been limited as well. This limitation represents a significant deficiency to otherwise robust databases such as “The Society of Thoracic Surgeons Database”, which has over 2 million cases registered in its three components, namely Adult Cardiac Surgery, General Thoracic Surgery, and Congenital Heart Surgery. Although the database is quite comprehensive in its capture of patients undergoing surgery of the chest, it lacks follow-up data past discharge from the hospital and 30 days after surgery. The addition of long-term follow-up data will be useful in multiple areas:
• comparing novel surgical techniques
• performing surveillance of implanted medical devices such as prosthetic cardiac valves
• educating patients about their long-term prognosis, and
• creating Kaplan Meier Survival curves from the data.
The Society of Thoracic Surgeons is a not-for-profit organization representing more than 5,400 surgeons, researchers, and allied health professionals worldwide who are dedicated to ensuring the best possible heart, lung, oesophageal, and other surgical procedures for the chest. Founded in 1964, the mission of The Society of Thoracic Surgeons is to enhance the ability of cardiothoracic surgeons to provide the highest quality patient care through education, research, and advocacy. By sending data about outcomes to The Society of Thoracic Surgeons Database, surgeons are committing to improving the quality of care that their cardiothoracic surgical patients receive. The Society of Thoracic Surgeons Database Workforce has been charged with investigating how to strengthen the database by extending its long-term follow-up. The introduction of “personal health information” into the Society of Thoracic Surgeons database will allow it to be linked to other databases such as the databases of the American College of Cardiology, those of the Centers for Medicare and Medicaid Services of the United States of America, and various national registries of death. This accomplishment will greatly increase how The Society of Thoracic Surgeons Database can help patients, physicians, research, and initiatives to improve quality. Mortality is the most fundamental and still the most important long-term result in which patients, clinicians, and third parties, including government, and industry, are interested. Therefore, The Society of Thoracic Surgeons Database Workforce has been doing due diligence regarding accessibility and accuracy of national registries of death. This article provides a summary and comparison of the two most comprehensive registries of death in the United States of America, namely, the: National Death Index and the Social Security Administration Death Master File, otherwise known as the Social Security Death Index. The lessons learned form this analysis may provide useful information to those in other countries attempting to obtain follow-up data by linking to similar national registries of death. This task has been successfully accomplished elsewhere. The Central Cardiac Audit Database of the United Kingdom already has a mechanism in place to verify mortality from a national registry of death. In this database, data about mortality is independently validated from records of deaths obtained by the Office for National Statistics, using the unique National Health Service number of the patient, or the general register offices of Scotland and Northern Ireland. This successful use of a national registry of death to verify data about mortality is described elsewhere in this Supplement by Clarke and colleagues in their article about verification of data.
The National Death Index of the United States of America1
Certificates of death are written in the United States for every death, whether or not the deceased is a citizen and regardless of age. Only the state government, in which the death occurred, has the authority to collect these certificates of death, and through the Office of Vital Statistics of each state, create a registry of death from these certificates. Therefore, before the establishment of the National Death Index, if someone wanted to verify the true mortality in the United States within a specific registry or database, it would have been necessary to send a standard form to each of the offices of vital statistics in all 50 states. The office for the state would cross-reference the “personal health information” given by the investigator with the registry of death of the state and determine which subjects in the registry of the investigator had died in that state. This process would take at least 8 months to 1 year to complete, but was the most accurate and only mechanism to determine true mortality in the United States regardless of citizenship or age. These statewide registries of death were a source of revenue for the states since various businesses, government agencies, and researchers of health-care, would pay to access them.
In the 1970s, a work force committee of the National Center of Health Statistics recommended a national registry of death, or “national death index”, be created by working with all 50 states to centralize all state registries of death into one national registry of death. Since only the states have the authority to collect the certificates of death, the “National Death Index” could not mandate the collection of their registries. The state governments agreed to participate if their data would only be used for research. With this understanding, monetary contracts were negotiated with all 50 states to obtain their registries of death yearly.
The National Death Index was begun in 1979 and activated for use in 1982. Initially, the National Death Index was partially funded by the National Institute of Health, but it became self-supportive in 1992. The National Death Index is part of the Division of Vital Statistics, which falls under the authority of the National Center of Health Statistics, a division of the Centers for Disease Control and Prevention. Yearly harvests of the registries of death from the states are done 12 months after the end of each calendar year. The National Death Index then takes approximately 1 year to organize, verify, audit, and insert the data into the database. This entire process leads to a two-year delay in the availability of the information. Therefore in 2007, all the deaths from 1979 to 2005 are available.
To access the database, one must first submit an application stating how the information learned from the National Death Index will be utilized. There is a list of acceptable uses and criteria found on the website of the National Death Index.1 The overriding criterion is that the National Death Index can only be used in medical or healthcare research. It cannot be used for legal or administrative purposes. Once the application is submitted, it takes approximately 2–3 months to be processed and approved. Submission of records to the National Death Index is quite specific. One submits as many of the 9 National Death Index pre-determined items of “personal health information” into the format of an “American Standard Code for Information Interchange text file” as described on the National Death Index website:1
• social security number
• day/month/year of birth
• first name, middle initial, last name
• father’s surname
• gender
• marital status
• state of birth
• state of residence
• race.
The National Death Index retrieval program is the software used to determine if a particular National Death Index death record qualifies as a possible match with a submitted user record. To qualify as a match, both records must share at least 1 of 7 matching criteria listed:
• Social Security Number
• Exact month and year of birth, first and last name
• Exact month and year of birth, first and middle initials, last name
• Exact month and day of birth, first and last name
• Exact month and day of birth, first and middle initials, last name
• Exact month and year of birth, first name, father’s surname
• If the subject is female: exact month and year of birth, first name, last name and father’s surname.
If a user record satisfies 1 of the 7 matching criteria with 1 or more National Death Index records, the National Death Index records are retrieved. The software then analyzes each of the matching National Death Index records with the entire “personal health information” in the user record. It determines not only how many “personal health information” data points match but the “quality” of the matching “personal health information” data. For example, matching social security numbers are of higher quality than matching first names. The retrieval program uses this analysis to determine a probabilistic score for each matching record in the National Death Index. The score represents the probability that the record of death in the National Death Index is a true match with the submitted record of the user. The report of the National Death Index for a particular submitted record lists in descending order all matching records in the National Death Index with the probabilistic score of each record.
The program also determines if the matched record in the National Death Index with the highest probabilistic score meets criteria so that the user record is marked “probably dead”. This determination is based on the probabilistic score and the type and quantity of “personal health information” given by the user for this particular record. Per internal audit, the sensitivity of the National Death Index to capture death of a submitted record is 92–98%, and if it determines that the patient is “probably dead”, the specificity is 98%.2
Use of the National Death Index is expensive because the National Death Index is self-sufficient and gets no federal funding. The National Death Index pays every state for their registry of death yearly, and also pays to each state a fee every time one of the records in the registry of that state matches with a submitted record. The user must pay a US$350 service charge at the initial submission of records for a newly approved protocol and US$100 for each subsequent submission of records under that protocol. The cost of the National Death Index service is US$0.15 per subject per year searched.
The National Death Index also provides an optional service called National Death Index Plus, which will give the user an overall cause of death. All causes of death are given to the user, as listed on the certificate of death, in the form of a code using the 9th or 10th Revision of the International Classification of Diseases. The National Death Index also uses software from the National Center of Health Statistics to examine these codes and generate an overall or underlying cause of death. Thus, the report of the National Death Index Plus will list the overall underlying cause of death, as well as all causes of death listed on the certificate of death. This process is done only for the record from the National Death Index with the highest probabilistic score. The cost of this National Death Index Plus service is US$0.21 per subject per year searched.
Table 1 demonstrates the calculation of the cost one would incur in order to use the National Death Index Plus service to establish mortality for 1000 hypothetical patients who underwent a given operation starting in 2001, with 200 patients operated on in 2001, 200 patients in 2002, 200 patients in 2003, 200 patients in 2004, and 200 patients in 2005. The total cost would be US$630.00. The cost of the National Death Index increases exponentially with longer follow-up and more patients. The same investigation as the example above with 200,000 people per year for 7 years will cost US$1,176,000. This total is calculated in the same manner as is shown in the example in Table 1 except that one uses 200,000 people per year and extends the time two years.
Table 1 Calculation of the cost one would incur in order to use the National Death Index Plus service to establish mortality for 1000 hypothetical patients over a 5 year period of time.
Currently, the National Death Index is the most accurate and uncontaminated registry of death in the United States. Unlike all other registries, the National Death Index is based on actual certificates of death, which are filled out for every death in the United States, regardless of whether the deceased is a citizen or non-citizen. Also, the National Death Index does not rely on reporting from third parties, require any one particular “public health information” data point, or need to glean information from other databases. Therefore, the National Death Index is the “gold standard” for determining true United States mortality.
The Social Security Administrating Death Master File, otherwise known as the Social Security Death Index of the United States of America3
The “Social Security Death Master File”, otherwise known as the “Social Security Death Index”, was created in 1980 by the Social Security Administration as a result of a lawsuit brought forth by an individual under the Freedom of Information Act.4 The Death Master File contains information on anyone assigned a Social Security Number whose death was reported to the Social Security Administration anytime after 1962. The Death Master File is part of the Social Security Administration’s Numerical Identification database called “NUMIDENT”. Each entry in the Death Master File contains the following variables:
• name of the person
• social security number
• date of birth
• month and year of death
• date of death if the record is from 2000 or later
• state or country of residence
• zip code of last residence
• zip code where “death benefit payment” was sent.
The Social Security Administration is not permitted to release to the public, in any form, data about death obtained from a state under the auspices of section 205(r) of the Social Security Act, unless given permission by the state. All of the states do give their data from their registries of death to the Social Security Administration to be used for internal government functions. However, half the states do not give permission to the Social Security Administration to use that data for the Death Master File or any other dissemination to the public.
Therefore, the data in the Death Master File is retrieved from other sources. Approximately 90% of the reported deaths come from funeral homes and family members. The remaining 10% come from various financial agencies and/or the postal service. The majority of deaths are reported to the Social Security Administration in order to receive monetary benefits such as a burial benefit, for which most are eligible. In October of 1981, Law P.L. 97-35 greatly reduced the number of people eligible for such benefits and thus the number of deaths reported to the Social Security Administration significantly decreased. This problem was remedied in 1989 by the Death Benefit Enunciation, which restored the death benefit to even more families than before. This legislation is thought to have made the Death Master File more robust than it had ever been.
The Death Master File is available to the public under the Freedom of Information Act. Public access to this information was transiently discontinued in 1989 by the Supreme Court in the United States Department of Justice versus Reporters Committee for Freedom of the Press. Public access to this information was reinstated in the Social Security Independence and Program Improvements Act of 1994. A number of websites exists that allow one to search the Death Master File for free, one “user file” at a time. The National Technical Information Service sells the Death Master File for the government in an array of options to suit the various needs of users. A single issue of the Death Master File, which is published quarterly, costs US$1730. This option is the most economical for someone who needs to sample the National Vital Statistics at a given time or only yearly. This file includes data from 1937 up to 2 months prior to the request date. Subscriptions may be updated via download on a monthly or weekly basis for US$2600 and US$7500 per year, respectively. Alternatively, a user may choose to pay a one time fee of US$6900 for the complete Death Master File plus one year of quarterly updates, which one can download. It is important to realize that the Death Master File is a large but simple database with over 80 million deaths reported in the format of an “American Standard Code for Information Interchange text file” with no search engine. Therefore to cross-reference the Death Master File with the purchaser’s database, the user must have their own matching, retrieval, and reporting program created. Therefore, the price of data management and reporting must be factored when calculating cost.
In determining whether the Death Master File is an appropriate alternative to the National Death Index, accuracy must be considered. When compared to the National Death Index, only by total number of adult deaths, not knowing if they are the same deaths, for the years 1990, 1995, and 1999, the Death Master File showed an impressive correlation to the “gold standard” at 95%, 97.8%, and 99.6% respectively.5 The specificity of the Death Master File is less clear. The Social Security Administration only began auditing the deaths in the Death Master File in 1996. The specificity over the past five years is 93% according to internal reviews. External reviews, often including known periods of poor collection of data for the Death Master File, show sensitivities of approximately 73% to 83%.Reference Schisterman and Whitcomb6–Reference Buchanich, Dolan, Marsh and Madrigano8 However, sensitivity and specificity have been shown to improve significantly with increasing age. A study involving a more elderly Veterans Affairs population reported sensitivity as high as 92.1%.5 Thus, the age of the subjects in the database must be considered when thinking about using the Death Master File. The Death Master File is based upon specific parameters:
• that the deceased was issued a Social Security Number
• that the deceased did not change their name
• that his or her death was reported to the Social Security Administration.
The specificity of the Death Master File therefore drops precipitously with age. The data on younger patients, and especially infants, is more likely to change and/or be incomplete more frequently than with older patients. Schisterman and colleagues stated, “Among certain populations [children and foreign born patients] use of these databases (Death Master File) alone may provide incomplete information.”Reference Schisterman and Whitcomb6
The Social Security Administration Vital Status Service
The Death Master File is not the only source of vital statistics, which the Social Security Administration offers; it also offers the Vital Status Service to Epidemiological Researchers.9 This service is available only for use in research that has been determined to contribute to a national health interest. This service searches the following three government databases for the vital status of an individual; thus, the service informs the user not only of probable death but of probable living:
• NUMIDENT which contains information from Social Security cards and includes the Death Master File
• the Master Beneficiary Record, which contains information on all individuals receiving any governmental benefits, and
• the Master Earnings File, which contains information about earnings on all persons that both work and have a residence.
The Vital Status Service requires the following six parameters for each submitted record:
• the first name of the patient
• the middle name of the patient
• the last name of the patient
• date of birth
• social security number, and
• sex.
If any of these is not provided, this service cannot be used. This absolute data requirement may prove difficult for the paediatric population and when analyzing a database retrospectively. The cost of using the Vital Status Service is less than 10% the cost of using the National Death Index, especially for large cohorts. If, however, there is no concern for whether a person is “presumed living” or not, the Vital Status Service does not offer any benefit over the Death Master File. The Death Master File is what is used by the Vital Status Service to determine if a person is dead. The other two databases used by the Vital Status Service determine if a person is alive. If the goal is confirmation of the presence or absence of death, therefore, the only benefit of the Vital Status Service over the Death Master File is that the user does not have to create a software program to search the Death Master File.
Discussion
The National Death Index and the Social Security Death Index have been compared in the past.Reference Lash and Silliman10 When a society is choosing with which national registry of death to link, there are many considerations including accessibility, cost, and accuracy. Most societies should have the ability to partner with managers of data to be able to search an “American Standard Code for Information Interchange text file” in a method particular to the interests of that society. Therefore, the Vital Status Service does not provide any advantage over the Death Master File. The comparison in this discussion will thus be limited to the Death Master File and the National Death Index (Table 2).
Table 2 Comparison of national registries of death.
In regards to accessibility, the National Death Index is limited in that there is a two-year lag between collection of data and access of the user to that data. This delay makes “real-time” yearly reporting of mortality not possible with the National Death Index. However, one can update a prospective database monthly using the Death Master File. In 2007, the Death Master File captures over 80% of deaths within the first month of a death and 99.5% of deaths within a year of the death.11
The National Death Index is to be used for research only and can not be used for administrative functions or interactions with payers. The Death Master File has no restriction in the use of its information. Although the primary goals of a medical society for a database are to improve the care of patients, educate patients and physicians, and improve the performance of physicians, the database will eventually be used to interact with industry, third party payers, and the government. It can be argued that the primary function of the databases of The Society of Thoracic Surgeons is “healthcare operations”, which is not research. This limitation of the National Death Index must therefore be considered.
In regards to cost, the National Death Index is much more expensive when compared to the Death Master File. Table 3 demonstrates the cost of using the National Death Index to review retrospectively 10 years of records from The Adult Cardiac Surgery Database of The Society of Thoracic Surgeons. Table 4 demonstrates the cost of using the National Death Index to review retrospectively 5 years of records from The Adult Cardiac Surgery Database of The Society of Thoracic Surgeons. In comparison to the cost of US$1,881,638.25 and US$529,735.35 to perform these analyses with the National Death Index, the cost to use the Social Security Death Master File (otherwise known as the National Death Index) for both the 10 and 5-year retrospective studies is US$1730.00 dollars plus the cost of creating a program to cross reference the database of The Society of Thoracic Surgeons with the Death Master File. This program can be written to the preferences of the Society in terms of what data should be weighted stronger for matches, how much data is needed to be considered a match, and how matches are reported. The sensitivity and specificity of the results can therefore mostly be determined by the programmer. Also, once this program is created, it can be used to harvest mortality from the Death Master File in any increment of time and for any patient cohort. The cost of creating reports retrospectively becomes independent of the number of patients and years to be reviewed as well as to the number of reports one wishes to create. This cost-savings alone is a tremendous advantage of the Death Master File over the National Death Index. It is also not possible to collect data prospectively and give yearly “real-time” reports about mortality in a growing database using the National Death Index because of the 2-year delay in data collection. Therefore the report in 2007 would contain data about mortality through 2005. If one uses the Death Master File, one can give a yearly report on the mortality of the entire database, or for any particular procedure or patient population, within months of the end of the year. Once the matching program is created, the cost would be US$1730.00 per year.
Table 3 The cost of using the National Death Index to review retrospectively 10 years of records from The Adult Cardiac Surgery Database of The Society of Thoracic Surgeons.
Table 4 The cost of using the National Death Index to review retrospectively 5 years of records from The Adult Cardiac Surgery Database of The Society of Thoracic Surgeons.
In regards to accuracy, the National Death Index is the most accurate and reliable way to determine mortality in the United States because it is based on certificates of death. Age, citizenship, or socio-economic class does not affect the issuing of a certificate of death, and thus does not affect the National Death Index. Therefore, the National Death Index is the “Gold Standard”. The Death Master File is known to be most accurate for older patients who are beneficiaries of Social Security. The limitations in people who are not beneficiaries of governmental funds, children, and non-citizens, are well known.3, 5, 11 There are also eras in which reporting of deaths to the Social Security Association was limited. However, since 1995, the sensitivity of the Death Master File has been greater than 97%, reaching 99% in the past few years. Also, over the past 5 years, internal auditing has demonstrated a specificity of 93%. As an increasing number of governmental databases are integrated, the accuracy of the Death Master File will continue to increase.
The Society of Thoracic Surgeons thinks that linking to a national death index is essential at this time for many reasons including some that we may not even be aware of at this time. Therefore, linking the database to a death index like the Death Master File, that allows the Society to use this powerful information about mortality in any manner it feels is beneficial to its patients and physicians, is essential. The flexibility to create multiple reports on specific procedures, populations of patient, or of particular eras, with no additional cost, is extremely beneficial. Yearly reports of mortality in a variety of different cohorts of patient and follow-up periods with no delay in time would be a tremendous service to our specialty. Also, the majority of data in The Society of Thoracic Surgeons database is after 1995 and consists of adults. Therefore, multiple reasons support the position that the Death Master File will probably be the primary index of death for the Society of Thoracic Surgeons. However, if there is a known part of a database that is easily identifiable and known to be grossly under represented in the Social Security Death Master File, such as the Society of Thoracic Surgeons Congenital Heart Surgery Database, one can consider using the National Death Index for that particular group.
The importance of linking one’s registry to an accurate national registry of death should not be underestimated. When choosing an index of death with which to link, it is important to examine multiple factors in one’s own database including
• its patient populations
• its eras of data collection
• its available “patient health information”, and
• its goals in regards to cost, reporting, and services provided to members and perhaps payers.
Although this analysis of the registries of death was specific to databases of The Society of Thoracic Surgeons, the information gathered, and the processes used to determine the best combination of indices of death for our Society, should be translatable and hopefully useful for other societies and registries who wish to empower their databases with long-term national data about mortality.
Acknowledgement
We thank The Children’s Heart Foundation (http://www.childrensheartfoundation.org/) for financial support of this research.
