Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-02-11T10:29:46.376Z Has data issue: false hasContentIssue false
  • English
  • Français

Measuring cost-effectiveness of secondary health care: Feasibility and potential utilization of results

Published online by Cambridge University Press:  02 March 2005

Pirjo Räsänen ,
Harri Sintonen ,
Olli-Pekka Ryynänen ,
Marja Blom ,
Virpi Semberg-Konttinen  and
Risto P. Roine
Pirjo Räsänen
Affiliation:
Helsinki and Uusimaa Hospital Group
Harri Sintonen
Affiliation:
University of Helsinki
Olli-Pekka Ryynänen
Affiliation:
University of Kuopio
Marja Blom
Affiliation:
Academy of Finland
Virpi Semberg-Konttinen
Affiliation:
Helsinki and Uusimaa Hospital Group
Risto P. Roine
Affiliation:
Helsinki and Uusimaa Hospital Group
Rights & Permissions [Opens in a new window]

Abstract

Objectives: Whether cost-effectiveness of secondary health care can be measured in a simple, yet commensurate way was studied.

Methods: Approximately 4,900 patients' health-related quality of life scores before and after treatment were measured. Used were a combination of quality of life data with diagnostic and financial indicators routinely collected in the hospital.

Results: Seventy percent of patients returned the first questionnaire and the informed written consent to participate. Of these patients, 80 percent also returned the second questionnaire sent out 3 to 12 months after treatment, depending on clinical specialty and diagnostic category. The routine of sending out questionnaires could be automated in such a way that data collection required only a limited amount of extra work. Patients were generally satisfied with the fact that the hospital was interested in their well-being also after treatment. No physician offered the chance to participate refused data collection in the patient group he or she was responsible for. The attitudes of the nursing staff were generally positive toward data collection, although it caused some extra work for some of them. The possibility of relating already routinely collected financial performance indicators with a relevant measure of treatment effectiveness, opened prospects for refined analysis of cost-effectiveness of secondary health care.

Conclusions: Routine collection of health-related quality of life data as an indicator of treatment effectiveness is feasible, requires only a small amount of extra work, and is potentially very useful when combined with existing measures of hospital performance.

Keywords

Quality of lifeQuality-adjusted life yearsCost-effectivenessTreatment outcome
Type
GENERAL ESSAYS
Information
International Journal of Technology Assessment in Health Care , Volume 21 , Issue 1 , January 2005 , pp. 22 - 31
Copyright
© 2005 Cambridge University Press

Society invests large amounts of money in secondary health care without knowing the return on this investment, as the health gains produced are not systematically assessed. In the light of the scarcity of resources, this strategy can be considered irresponsible. Every effort should be made to ascertain that health care is effective and that the resources available are allocated to treatments shown to be cost-effective. To reach this goal, secondary and tertiary health care need novel ways of measuring cost-effectiveness that allow the comparison of costs and health gains of various treatment alternatives.

Traditionally, the providers of the treatments have assessed the effectiveness by using provider-oriented outcomes. Such an approach is, however, prone to lead to subjective and biased assessments. Furthermore, the clinical outcomes used to evaluate effectiveness are usually disease- or at best specialty-specific and do not allow comparison of treatment results across different clinical specialties.

During recent years, it has been acknowledged that, in addition to the length of life, also life's quality is of importance. Consequently, there have been several attempts to develop new, generic (non–disease-specific) methods for the estimation of treatment results that would take into account patient preferences also. These approaches have mainly relied on the use of health-related quality of life (HRQoL) instruments such as the EQ-5D (EuroQol), SF-6D (based on RAND-36/SF-36), HUI 3 (Health Utilities Index Mark III), AQoL (Assessment of Quality of Life), and 15D (2;6).

HRQoL reflects the physical, psychological, and emotional dimensions of health (3). It can be used to describe the effects of an illness on the quality of life and the effect of clinical interventions on health and general well-being (5). In addition to the disease and its treatment, HRQoL is affected by the general condition of the individual in question, other health problems and sickness experiences he may have, his phase of life, as well as the tasks and goals he has. The different dimensions of health can be described with various generic or disease-specific instruments. The generic instruments allow the comparison of a variety of patient groups, whereas the disease-specific instruments only give information on the effect of a certain disease on health and, thus, are not suited for comparison of treatment results across different diseases.

In Finland, the idea of measuring the cost-effectiveness of secondary health care by using a standard generic HRQoL instrument (15D) was presented in 1996 (10). Pilot projects since have demonstrated that it is feasible (1). A similar approach to the measurement of effectiveness has been used at least in Canada, where outcomes of elective surgery in a large group of surgical patients were successfully evaluated using the SF-36 (13). However, they did not have cost-effectiveness data.

In Helsinki University Hospital, a large referral hospital providing secondary and tertiary health-care services for a population of approximately 1.4 million, data on the cost of treatment of the individual patients have been collected routinely for years. These data, however, have not allowed the estimation of cost-effectiveness of the treatments as indicators of effectiveness have been missing. Our aim, therefore, was to test the feasibility of collecting effectiveness data using the generic 15D HRQoL instrument and combining these data with the available cost data to create means for estimating the cost-effectiveness of secondary and tertiary health-care services routinely.

PATIENTS AND METHODS

Patients

Data collection began in March 2002 and has so far covered patients receiving treatment according to normal hospital routines in over thirty distinct medical entities (Table 1).

Health-Related Quality of Life

HRQoL is measured with the 15D. It is a generic, standardized, self-administered instrument that can be used both as a profile and a single index score measure. Its health state descriptive system consists of 15 dimensions: moving, seeing, hearing, breathing, sleeping, eating, speech, eliminating, usual activities, mental function, discomfort and symptoms, depression, distress, vitality, and sexual activity. For each dimension, the respondent must choose one of the five levels that best describes his/her state of health at the moment (the best level = 1; the worst level = 5). The single index score (15D score) on a 0–1 scale, representing the overall HRQoL, is calculated from the health state descriptive system by using a set of population-based preference or utility weights. An index value of 1 represents full health and 0 is equivalent to being dead (11). Validation studies have shown that the 15D compares favorably with the other instruments of that kind in most of the criteria and properties set to such instruments (4;8;9;11;12).

The 15D questionnaire is mailed together with the invitation to come to the hospital or given personally during an ambulatory visit before hospitalization or start of ambulatory treatment. The patients are encouraged to fill in both the questionnaire and the accompanying informed consent form and to return both in a prepaid envelope by mail or to hand them over to the staff when they enter the ward for treatment. Those patients not responding to the first questionnaire are considered unwilling to participate and are not approached again.

After treatment, another questionnaire is sent, depending on the medical entity, usually 3 to 6 months after the first questionnaire to patients having responded to the first questionnaire. A reminder is sent once, if the patient does not return this repeat questionnaire within 3 weeks. The patients are asked to return the repeat questionnaire in a prepaid envelope by mail.

Disease Specific Questionnaires

In some medical entities, a disease-specific questionnaire has been used in addition to the generic 15D questionnaire to allow comparison. Among the disease-specific questionnaires used are, for example, the Health Assessment Questionnaire, the Eating Disorder Inventory, the Harrison Hip Score, the Knee Society Score, and the Depression Scale.

Costs and Service Organization

Direct costs of secondary health care are available from the Ecomed patient administration system (Datawell Ltd., Finland) routinely used in the hospital. In addition to cost data, the system includes data on patient variables (diagnosis, therapeutic procedure, where discharged) and organization of services such as urgency, waiting time, service provider (unit and physician in charge), ad so on.

Cost-Effectiveness

To assess the cost-effectiveness of treatments, the 15D data are combined with the cost data. At this stage, all information allowing the identification of individual patients is removed and the data are stored using patient-numbers only.

Ethical Considerations

Ethical approval for the study was obtained from the surgical and coordinating ethical committees of the Helsinki and Uusimaa Hospital district. Every patient participating in the study was asked to fill in an informed consent form.

Statistical Methods

The SPSS for Windows statistical software (SPSS, Inc., Chicago, IL, USA) was used to derive the preliminary results. Outcome variables are reported as mean values or medians. For comparison, the age and gender distribution of the general population was matched with that of the patient groups with weighting.

RESULTS

Data Collection and Recording

To date (by March, 2004), 4,900 patients have returned the first questionnaire and 2,680 the repeat questionnaire (the repeat questionnaire has not been due to all patients yet). The average response rate has been over 70 percent, but there is considerable variation between different clinical specialties (Table 2).

All data concerning personal identification and address details of the patients, clinical specialty in which they were treated, and the data collected by the 15D questionnaires are first entered into a Microsoft Access (Microsoft Co., Redmond, WA, USA) database specifically tailored for the project. The mailing of the repeat questionnaires is based on data automatically generated by this database.

From the Microsoft Access database, the data are later transferred to the Ecomed database, where all cost data concerning medical treatments given in the hospital are routinely stored. At this stage, all data are encrypted so that individual patients cannot be identified from the database during further analyses.

Feasibility

Attitudes of the hospital personnel—both physicians and nurses—have generally been very positive toward the project. For instance, the total number of patients entering treatment during data collection was 419 in hip and knee replacement surgery. Of these, 385 (92%) were offered the questionnaire, indicating good compliance of the personnel with the data collection. No physician who was offered the chance to participate has refused data collection in the clinical specialty he/she is responsible for. In all cases, initiation of data collection has required personal communication with those responsible for distribution of the first questionnaires, and in some cases, continuous attention and motivation has been needed to ensure successful continuation of the data collection.

Costs of Data Collection

The total cost of the project (consisting of salaries, printing, copying, mailing, data recording, statistical consulting, meeting and travel costs, development of necessary software for patient administration) has been 73,000 € by the end of year 2003. This amount equals approximately 16 € per patient recruited.

Examples on the Use of Effectiveness Data

In most clinical specialties, data collection is still ongoing and even in those, in which data collection has come to an end, final analyses are still to be performed. Therefore, the following results are preliminary and should be regarded as mere examples of how, and for what, the data can be used, not as definite estimates of the relative cost-effectiveness of various interventions.

One of the most important possibilities the data offers is the comparison of effectiveness (i.e., the change in HRQoL) of treatments across different clinical specialties (Fig. 1). Another significant possibility is to compare the 15D profiles, that is, the scores on the fifteen different dimensions of health before and after treatment. Figure 2 shows that knee replacement surgery patients are clearly better off on the dimensions of moving, usual activities, discomfort, and symptoms as a consequence of treatment, but worse off on sexual activity. Effectiveness of treatments can also be examined as a function of, for example, gender or age (Fig. 3).

Health-related quality of life (15D score on a 0–1 scale) before and after treatments in different specialties or medical entities.

The 15 dimensions and mean 15D score of health in knee replacement surgery patients before and 6 and 12 months after knee surgery.

Effect of gender and age on effectiveness (i.e. change in health-related quality of life score) of disc herniation surgery. Square symbols represent the median, and the error bars represent the range.

In diagnostic groups such as cancer treatment, the HRQoL data can be used to distinguish between the advantages and disadvantages of radical and palliative treatment. Figure 4 shows that, although palliative treatment may not be effective in prolonging life, it can have a positive effect on the quality of life of the patients. Comparison of the 15D profiles of patient groups and the age/gender-matched general population can reveal on which dimensions the patients have more severe problems than the general population and, thus, help target treatment to tackle those problems (Fig. 5).

Health-related quality of life (15D score on a 0–1 scale) in esophageal carcinoma patients before and 6 months after radical or palliative treatment.

The 15D dimensions and mean 15D score of esophageal cancer patients and age/gender-matched general population.

Examples on the Use of Cost-Effectiveness Data

Combining effectiveness data with cost information allows the estimation of short-term cost-effectiveness of various interventions across different clinical specialties (Table 3; Fig. 6). When the service provider (e.g., a ward or an individual physician) is known, cost-effectiveness data can also be used to monitor the effectiveness of various service providers and to possibly discover service problems at an early stage (Fig. 7).

Short-term cost-effectiveness of two surgical interventions (total hip replacement surgery and disc herniation surgery).

Effect of treatment ward on short-term cost-effectiveness of hip replacement surgery.

By taking into account the length of life (the years a patient can be expected to survive after a successful intervention) and the HRQoL change and its duration, we can estimate the cost utility (cost/quality-adjusted life years [QALYs] gained) of the intervention (Fig. 8). This finding can significantly affect judgments about treatments in certain subgroups as can be seen when comparing the short-term cost-effectiveness (Fig. 6) and cost utility (Fig. 8) of total hip replacement in men and women.

Cost-utility (i.e. cost/quality-adjusted life year gained [QALY]) of two common surgical interventions (total hip replacement surgery and disc herniation surgery) assuming that the short-term health-related quality of life gain lasts for the rest of the lifespan.

DISCUSSION

The basis for our study has been the need for reliable effectiveness and cost-effectiveness data to guide future decisions on investments in health care. Such data should be available with as little extra investment as possible and cover, on a routine basis, at least all the most common treatments the hospital provides. As our hospital, like most other hospitals, has long collected rather meticulous data on costs, all that is needed for the estimation of cost-effectiveness is an indicator of effectiveness of treatments. To allow comparison of different interventions across many clinical specialties, this indicator of effectiveness needs to be generic, that is, non–disease-specific.

For measuring effectiveness, we chose the 15D HRQoL instrument due to its many desirable properties. The 15D questionnaire can easily be filled in by the patients themselves. This ease is evidenced by the fact that the patients seldom contacted the project personnel to ask for help concerning the questionnaire, although contact details were readily available on the project information sheet. In some clinical specialties, HRQoL data were supplemented with disease-specific outcome data. However, preliminary analyses show that the disease-specific outcomes do not correlate especially well with HRQoL. We acknowledge that disease-specific outcomes are in many cases important for a given clinical specialty to monitor their treatment outcomes and to control the quality of the treatment given. Disease-specific outcomes, however, rarely are suited for the estimation of cost-effectiveness of treatments as they do not allow the comparison of different clinical specialties with each other. Furthermore, they rarely involve the viewpoint of the patient, that is, the patient's conception of whether a particular treatment has been helpful or not.

In addition to the specific disease and its treatment, HRQoL can be affected by other health problems and sickness experiences a patient may have. In most cases, these conditions will be reflected in the 15D score. However, in some cases more sophisticated analyses will be needed. These analyses can be based partly on unexpected or illogical changes in the 15D profile. Furthermore, our database also includes secondary diagnoses, which can be used to create a system that takes into account comorbidity by using validated comorbidity scores or other similar approaches.

Collection of effectiveness data proved to be fairly simple and did not require major investment. A single person can fairly easily handle the mailing and data collection provided that the mailing is at least partly automated using appropriate software. Both the health-care personnel and the patients appreciated the efforts to produce effectiveness data. This finding was reflected in that none of the specialties refused the offer to participate in our project, even though the distribution of the first questionnaires did in some cases cause a little extra work for those sending out the invitations to the patients. Staff of some specialties even approached us spontaneously with a request to join the project.

Also, patients appreciated the unusual event that the hospital was interested in their well-being afterward. An indication of this appreciation is the unusually high response rate to the first questionnaire compared with surveys in general. This finding despite that the patients were not approached a second time if they had not returned the first questionnaire. Also, the response rate to the repeat questionnaire was high, around 80 percent, showing the willingness of the patients to participate.

The handling of patient data required the design of some new software solutions in the beginning of the project. As the Microsoft Access database was modified to meet our needs, the mailing of questionnaires was sufficiently automated to be easily managed by one person alone. However, if the measurement would take place on an even larger scale in the future, efforts should be made to also automate the data entry, which in our case was manual. One possibility is to make the data collection, at least in part, Internet-based so that patients can enter the data directly into a database. Further automation of the mailing process could be achieved by improving the database, for example, to take into account that the covering letters of the questionnaires need to be tailored to suit the individual needs of different clinical specialties. Furthermore, combining the mailing system with the Finnish population register to automatically ensure that the patient is alive and the address is correct would further reduce the amount of human resources needed for large scale routine data collection.

The merging of effectiveness data with cost data and information on the treatments posed no problems in the Ecomed database. However, when individual patients had been treated multiple times during the follow-up period, it was sometimes difficult to ascertain to which treatment the effectiveness data were connected. Similarly, it is also virtually impossible to tell the effect of a single intervention on the change in HRQoL if the patient has received treatment for two or more illnesses during the follow-up. Some of these problems can be overcome by manually checking that the effectiveness data are linked to the right treatment during the follow-up period. This process does not require much work when the patient material is small but does not solve the problem of how much each treatment has contributed to the change in HRQoL. In large patient materials, hand checking is exceedingly difficult but, on the other hand, may not be that important, as some deviant patients easily disappear into the mass when thousands of patients are studied and, thus, do not bias the results in a significant way.

Whether the cost-effectiveness data produced by our system is able to really influence decision making remains to be studied when enough data are available to make reliable comparison across various interventions. When Wright and colleagues studied outcomes of elective surgery using the SF-36, their results showed some interesting differences in the effectiveness of various surgical procedures. However, only approximately half of the surgeons giving feedback had positive comments about the project and the information it produced. The rest believed it was of little value and did not want to receive such information on their patients in the future (13). It may indeed be, as Wright et al. (13) speculate, that some surgeons may have difficulty accepting that self-reported HRQoL is a more valid outcome than their own impressions and, thus, may be prone to deny the usefulness of such data (13). This finding could be an even bigger problem in our project, where opposed to Wright et al., we also report cost-effectiveness data, which may be seen by some physicians as an attempt to limit their sovereign right to decide what is right for each patient regardless of costs. On the other hand, a limited number of other hospital decision makers so far interviewed considered our preliminary cost-effectiveness data very useful.

The approximate cost of data collection, €16 per patient, may not be considered quite insignificant and is higher than the $12 Canadian per patient reported by Wright et al. (13). One has to bear in mind, however, that our costs include all the initial investments in the development of the system, and we are confident that, once the system is up and running, the cost per patient will be significantly lower. Further cost reductions can certainly be achieved by the automation of the data collection so that, compared with the value of the information received, costs of data collection eventually can be expected to be trivial.

Measurement and interpretation of the cost-effectiveness data is especially suited for comparison of fairly straightforward interventions such as various surgical procedures. We have attempted, however, to collect data also on more vague disorders, where the intervention does not consist of a single, well-defined procedure. These entities include, for example, rheumatology, intensive care, and treatment of obesity. Whether HRQoL data can be used to assess the effectiveness of also such treatments remains to be seen, as data collection in these fields is still in progress. However, preliminary results concerning the treatment of, for example, esophageal cancer, showing clear HRQoL benefits from palliative treatment, indicate that our approach may well work in less clearly defined treatment entities as well.

In conclusion, our project shows that routine collection of effectiveness data is feasible and does not require major extra investment. Data on effectiveness can be easily used to complement the already available cost data to produce estimates of cost-effectiveness of various interventions. Attitudes of both clinicians and patients have been favorable toward data collection, indicating that estimation of cost-effectiveness of treatments is generally considered necessary. Preliminary experiences also allude that other decision-makers appreciate cost-effectiveness data. Whether the data produced really affect allocation of resources, however, must be established in future studies.

POLICY IMPLICATIONS

Our approach enables routine collection of cost-effectiveness data based on QALYs, which, for example, the National Institute for Clinical Excellence responsible for providing national guidance on treatments and care for those using the NHS in England and Wales uses as the principal measure of health outcome (7).

CONTACT INFORMATION

Pirjo Räsänen, MSc, Researcher (), Group Administration, Helsinki and Uusimaa Hospital Group, P.O. Box 100, 00029 HUS, Finland

Harri Sintonen, PhD, Professor of Health Economics (), Department of Public Health, University of Helsinki, P.O. Box 41, 00014 Helsinki, Finland

Olli-Pekka Ryynänen, MD, PhD, General Secretary (), Department of Health Policy and Management, University of Kuopio, P.O. Box 1627, 70211 Kuopio, Finland

Marja Blom, PhD, Senior Fellow (), Academy of Finland, Stakes, P.O. Box 220, 00531 Helsinki, Finland

Virpi Semberg-Konttinen, MSc, MJD, Officer for Audit Board (), Risto P. Roine, MD, PhD, Chief Physician (), Group Administration, Helsinki and Uusimaa Hospital Group, P.O. Box 100, 00029 HUS, Finland

The invaluable contribution of Professor Pekka Laippala (deceased May 13th, 2003) during the early phases of the project is gratefully acknowledged. Anna-Maija Koivisto is sincerely thanked for help in statistical analyses, Mikko Rotonen and Petri Mikkola for expert help in the construction and management of databases, and the clinical staff of the Helsinki and Uusimaa for their sympathetic attitude and help in the distribution of the questionnaires. The study was funded by research grants from the Helsinki and Uusimaa Hospital Group.

References

Blom-Lange M, Sintonen H. 1998. Measuring the effectiveness of hospital treatment. Paper presented at the 19th Nordic Health Economists' Study Group Meeting in Oslo, Norway, August 21-22, pp. 1– 10. SINTEF Unimed; 1998.
Drummond MF, O'Brien BJ, Stoddard GL, Torrance GW. 1997. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press;
Gold MR, Russell LB, Siegel JE, Weinstein MC. 1996. Cost-effectiveness in health and medicine. New York: Oxford University Press;
Hawthorne G, Richardson J, Day NA. 2001 A comparison of the Assessment of Quality of Life (AQoL) with four other generic utility instruments. Ann Med. 33: 358– 370.Google Scholar
Kauppinen R. 1999. Cost-effectiveness of intensive patient education for guided self-management of adult novel asthmatic patients. Doctoral dissertation. Kuopio: University Printing Office;
Kopec JA, Willison KD. 2003 A comparative review of four preference-weighted measures of health-related quality of life. J Clin Epidemiol. 56: 317– 325.Google Scholar
Rawlins MD, Culyer AJ. 2004 National Institute for Clinical Excellence and its value judgments. BMJ. 329: 224– 227.Google Scholar
Sintonen H. 1994. The 15D-measure of health-related quality of life. I. Reliability, validity and sensitivity of its health state descriptive system. National Centre for Health Program Evaluation, Working Paper 41, Melbourne;
Sintonen H. 1995. The 15D measure of health-related quality of life. II Feasibility, reliability and validity of its valuation system. National Centre for Health Program Evaluation, Working Paper 42, Melbourne;
Sintonen H. 1996; Kustannusvaikuttavuus erikoissairaanhoidossa (Cost-effectiveness in specialised health care). In: Mikkola T, Outinen M, eds. Kohti ensi vuosituhatta. Näkymiä erikoissairaanhoidon laadunhallintaan. Helsinki Stakes Raportteja 205: 32– 37.
Sintonen H. 2001 The 15D instrument of health-related quality of life: Properties and applications. Ann Med. 33: 328– 336.Google Scholar
Stavem K. 1999 Reliability, validity and responsiveness of two multiattribute utility measures in patients with chronic obstructive pulmonary disease. Qual Life Res. 8: 45– 54.Google Scholar
Wright CJ, Chambers GK, Robens-Paradise Y. 2002 Evaluation of indications for and outcomes of elective surgery. CMAJ. 167: 461– 466.Google Scholar
Figure 0

Clinical Specialties and Medical Entities Studied in the Project

Figure 1

Number of First and Repeat Questionnaires Returned and the Corresponding Response Rates in Clinical Specialties Where Data Collection Has Already Been Terminated

Figure 2

Health-related quality of life (15D score on a 0–1 scale) before and after treatments in different specialties or medical entities.

Figure 3

The 15 dimensions and mean 15D score of health in knee replacement surgery patients before and 6 and 12 months after knee surgery.

Figure 4

Effect of gender and age on effectiveness (i.e. change in health-related quality of life score) of disc herniation surgery. Square symbols represent the median, and the error bars represent the range.

Figure 5

Health-related quality of life (15D score on a 0–1 scale) in esophageal carcinoma patients before and 6 months after radical or palliative treatment.

Figure 6

The 15D dimensions and mean 15D score of esophageal cancer patients and age/gender-matched general population.

Figure 7

Short-Term Cost-Effectiveness of Various Interventions

Figure 8

Short-term cost-effectiveness of two surgical interventions (total hip replacement surgery and disc herniation surgery).

Figure 9

Effect of treatment ward on short-term cost-effectiveness of hip replacement surgery.

Figure 10

Cost-utility (i.e. cost/quality-adjusted life year gained [QALY]) of two common surgical interventions (total hip replacement surgery and disc herniation surgery) assuming that the short-term health-related quality of life gain lasts for the rest of the lifespan.