Many biomedical innovations follow the law of diminishing returns: they improve the expected health outcome, but it is necessary to devote an increasing amount of resources to obtain an additional unit of benefit (for example, an additional life year). In recent years, advances in breast cancer treatment have been a typical illustration of such a trend (19;20;33). High-dose chemotherapy (HDC) supported by blood stem cell transplantation for advanced breast cancer patients is currently a matter of intense controversy in the scientific and medical communities. Several clinical trials comparing HDC to conventional chemotherapy have failed to demonstrate that these more-intensive therapies significantly improve these patients' survival. However, breast cancer patients treated with HDC seem to have a lower likelihood of relapse and a significantly better disease-free survival (29).
In the current context in which all Organisation for Economic Co-operation and Development governments implement policies to control the escalation of health-care expenditures, cost-effectiveness considerations play a growing role in public and professional decisions to adopt therapeutic innovations that may improve the health status of patients but for increasing costs. One key parameter in the decision to adopt an innovation is whether or not its incremental cost-effectiveness ratio (ICER) compared with the standard treatment is below or above some implicit or explicit threshold (16;35). Therefore, the handling of uncertainty in the estimation of the cost and outcome parameters used in economic evaluation has become an important issue. There are several potential sources of uncertainty in any cost-effectiveness analysis (CEA): uncertainty over the choice of parameter estimates and modeling assumptions (4) and uncertainty due to the variations in the sample data (25). Sensitivity analyses are commonly used to explore uncertainty resulting from assumptions made on the variables used. In addition, providing stochastic data on both costs and effects has led to a growing interest in uncertainty due to sampling fluctuations (5;7;37;38), and statistical methods, therefore, have been developed to obtain confidence intervals for cost-effectiveness ratios. In the framework of potentially controversial decisions about the adoption of costly innovations, such as HDC in advanced breast cancer, providing decision-makers with more accurate information about the cost-effectiveness of innovative procedures may influence the decision process and increase the contribution of economic analysis to this process (12;13;31).
This article presents the results of the CEA included in the PEGASE 01 protocol, a randomized controlled trial designed to determine whether adding a single course of HDC may improve disease-free and overall survival for breast cancer patients with axillary lymph node invasion. This example illustrates how both the choice of alternative clinical outcomes and the use of statistical techniques to take sampling uncertainty into account can significantly affect the results of a CEA and its interpretation for policy recommendations.
METHODS
Clinical Trial
Patients included in the study had a histologically confirmed nonmetastatic breast cancer with axillary lymph node invasion (8 positive nodes). They were 18 to 60 years old, with a World Health Organization performance status lower than 2. Each patient as required by national guidelines signed written informed consent forms, and the protocol was approved by an ethical committee (Consultative Committee for Protection of Persons Participating to Biomedical Research).
After initial surgery, 314 patients were randomized between 1994 and 1998. In arm A, patients received four cycles of FEC 100 (fluorouracil, epirubicin, cyclophosphamide). In arm B, patients received the same adjuvant chemotherapy (four cycles of FEC 100) followed by one course of CMA (cyclophosphamide, mitoxantrone, Alkeran). In this intensive arm, peripheral blood stem cells were collected during FEC 100 cycles under 5/,±g/kg filgrastim stimulation and reinfused after intensification. Chemotherapy was followed by 5-week radiotherapy regimen for all patients.
Cost-Effectiveness Analysis
Costs
Resource Utilization
Economic evaluation was undertaken from the French Hospitals perspective and is consequently restricted to direct medical costs. The estimation of these costs consisted of a measurement of resource utilization in physical quantities combined with a monetary valuation using unit cost data. With the exception of costs related to relapse, which will be presented below, physical quantities related to medical resource utilization were collected prospectively.
Resource items collected for calculation of costs were as follows: Hospital stays, length of inpatient stays and number of day-clinic visits; Pharmacy, quantities of drugs administered; Blood products, number of transfusion episodes; Laboratory, tests and medical examinations specified by the clinical protocol as well as additional tests due to febrile events; Surgical procedure, mastectomy or breast-conserving surgery. Cost of radiotherapy was deliberately excluded from the cost estimation, as modalities of irradiation were identical in both treatment groups.
Monetary Valuation
Monetary values were attributed to the physical quantities consumed. Because of the well-known problem of differences between hospital charges and real costs (15), especially in the context of publicly funded health-care system such as the French one, the hospital charges were not used to assess the costs of hospitalization. The per diem “real cost” was calculated from a detailed observation of the annual expenditures of one center (Institut Paoli-Calmettes). This cost included consumable supplies, cost of staff, food cost, and depreciation of equipment (taking a depreciation rate of 20 percent). Step-down methodology was used to add overheads to these unit costs (14).
Drug prices were the purchase prices nationally negotiated by the Federation of French Cancer Hospitals. For prices of blood products, the official 1999 prices, published each year by the French Government, were used (28). The costs of laboratory tests, diagnostic examinations, and surgical interventions were determined using the official tariffs of the French National Health Service (36). Costs of the peripheral stem cell collection procedure (including the costs of labor, supplies, equipment, and laboratory tests) were obtained from a parallel French study (23).
Economic data were collected until 6 months from the entry in the study, and the total cost of treatments, therefore, was calculated during this 6-month period. This limitation is partly justified as posttreatment follow-up procedures and investigations beyond this 6-month period were the same between treatment groups.
However, to take into account a possible difference in relapse rate between treatments, a cost of relapse was also introduced into the CEA using the study by Bercez et al. (3) that evaluated the direct medical costs of breast cancer recurrence in France. For patients who relapsed, we applied this average cost of recurrènce, depending on the type of recurrence involved.
Sensitivity Analysis
Several sensitivity analyses were carried out to examine the robustness of the cost results to a variation in some key parameters. First, we assessed the impact of a change in the unit prices of the three main cost factors: hospitalization, drugs, and blood products. In addition, some differences between the management of patients in a clinical trial and current clinical practice are likely to exist. In particular, the actual practice for intensification is leading to a decrease in hospitalizations, the major component of total cost. We, therefore, evaluated the impact on cost results of changing the length of hospitalization.
Because our cost data were strongly skewed, the normality hypothesis could not be assumed and the Student t-test was not appropriate. Parametric and nonparametric bootstrap tests, therefore, were performed to test the difference in mean costs between treatments.
Effectiveness
The Kaplan–Meier method was used to estimate the mean survival times of each treatment group, and the log rank test was applied to test the difference in these mean times. Two outcome measures were successively considered in our CEA: disease-free survival (DFS) and overall survival (OS).
Cost-Effectiveness Ratio and Impact of Uncertainty
The main clinical end point of the study was DFS. However, the ultimate goal of any health program is to improve OS. Thus, we realized the CEA for both DFS and OS outcomes, and ICERs were successively calculated for these two end points, giving the cost per progression-free life year gained and the cost per life year gained.
The impact of uncertainty on these estimated ICERs, due to the sampling fluctuations, was analyzed through the building of 95 percent confidence region for the ICER. The “truncated Fieller's method”, based on the joint distribution function of the pair (mean costs difference, mean survival times difference), is stable, even if the difference in mean survival time approaches zero statistically (32). Consequently, we used this method to estimate the confidence interval for the ICER.
Estimators of the variance of the effects as well as the covariance between costs and effects were obtained using bootstrap methods, because we had no simple available expression for them. The bootstrap consisted in resampling with replacement, simultaneously, of costs and survival time from the sample (so as to preserve their correlation). This procedure was repeated 5,000 times: variance of survival differences as well as covariance between costs and effects differences, therefore, were obtained.
RESULTS
Costs
Calculation of costs was carried out on 300 patients, as 14 intensive patients did not receive HDC (data relative to the treatment effectively received were not available for these patients). Resource consumption was measured for these 300 patients from entry in the study up to 6 months after treatment, and the costs calculated are summarized in Table 1.
Intensive treatment gave an average cost per patient of 17,837€ (range, 17,286€–18,389€), versus 5,188€ (range, 4,883€–5,492€) for the standard treatment. HDC was, therefore, 3.4 times more expensive than standard chemotherapy. This cost difference, statistically significant (p<.001, using the parametric and nonparametric bootstrap tests), was mainly due to differences in the length of hospital stays (43.5 percent of the additional cost) and granulocyte colony-stimulating factor administration (13 percent of the extra cost). Distribution of costs per categories was different in each arm: quantities of drugs represented the most important part of cost for standard treatment with a mean cost of 1,532€ (29.5 percent of total cost), whereas hospitalization cost represented the most important cost component for HDC with a mean cost of 6,537€ (36.7 percent of total cost).
Including the cost of relapse, the difference between the mean costs of the two treatments decreased. Mean costs were, respectively, 28,262€ for the intensive arm versus 20,859€ for the standard treatment; that is almost a 42 percent decrease in the cost difference.
Sensitivity Analysis
The first sensitivity analysis was performed by varying the price of the three main cost factors: hospitalization, drugs, and transfusion. An increase of 10, 15, or 25 percent in these prices did not reverse the conclusion of the analysis. Conclusions were similar considering a decrease in these unit prices. The second sensitivity analysis, designed to evaluate a potential change in the routine management of hospital stays showed that a change in the number of days per inpatient would not reverse the results of the cost comparison.
Effectiveness
Survival curves for DFS and OS are presented in Figure 1. After a median follow-up period of 61.2 months, a significant difference in DFS was found with mean DFS durations (95 percent confidence interval [CI]) of 49.5 months (CI, 44.1–54.8) and 61.1 months (CI, 56.1–65.8) for standard and HDC patients respectively (p <.001). The 5-year DFS rates were 40.7 and 60 percent, which represented 65 relapses for the standard arm versus 44 for the HDC arm.
However, OS after 5 years was the same in both treatment groups with mean durations of 70.27 months (CI, 65.47–75.08) for standard arm and 71.78 months (CI, 67.83–75.73) for HDC arm. At the time of analysis, 91 patients died: 50 and 41 in standard and intensive arms, respectively.
Cost-Effectiveness Ratio and Impact of Uncertainty
The incremental cost-effectiveness ratios were calculated for both DFS and OS outcomes, and descriptive statistics are presented in Tables 2 and 3. Because there was no difference in OS between the two treatment groups, the CEA in that case turns out to be a cost-minimization analysis. On this basis, because the intensive treatment was more expensive but resulted in a similar survival, it can be considered as a dominated strategy that should be rejected. However, CEA using DFS as the end point showed that adoption of HDC would lead to an incremental cost per progression-free life year gained of 13,074€. Figure 2 gives a graphic representation of the sampling uncertainty associated with this ICER. This figure shows 5,000 replications of the pairs constituted by the differences in average total costs and mean survival times between the two treatments, corresponding to an approximation of the joint distribution of costs and survival outcomes. The rays whose slopes correspond to the upper and lower limits of confidence regions for the ICER, for various confidence levels (99 percent; 95 percent, and 90 percent), obtained using the truncated Fieller's method.
Confidence regions for the incremental cost-effectiveness ratio with the truncated Fieller's method with disease-free survival as clinical end point. DC.DE, pair, mean costs difference, mean effects difference; CI, confidence interval.
The bounds' values of the confidence regions calculated at the 95 percent CI with the truncated Fieller's method are presented in Table 2. These results show that the upper limit of the 95 percent CI is equal to around 38,000€ per disease-free life year gained (which is around three times more than the point estimate of the ICER). In this case, if a cost of at least 38,000€ to obtain the improvement of quality of life associated with a disease-free life year in comparison with a life year with cancer is considered to be socially acceptable, then the intensive treatment should be adopted (with a 5 percent error rate).
DISCUSSION
In cancer care, as in many other fields, medical decisions are often confronted with situations in which innovative treatments, which are evaluated through randomized clinical trial, fail to demonstrate a significant advantage in terms of overall survival but may offer other benefits for patients by reducing or delaying the risk of relapse. When this additional benefit is obtained while minimizing costs, adoption of the new treatment should be straightforward. However, in most cases, as in the case of HDC in our study, therapeutic innovations are more costly than standard treatment and devoting additional resources to obtain a limited benefit in quality of life, which does not translate into increased life expectancy, raises complex social issues. In such cases, handling of uncertainty in economic analysis becomes especially important to guarantee the robustness of the estimations and minimize the risk of devoting too many resources for a limited improvement.
In this study, we present the results of the CEA of the PEGASE 01 protocol designed to evaluate the benefit of adding a single course of CMA to a conventional 4 FEC100 chemotherapy regimen for node-positive patients with breast cancer. Our uncertainty analysis was threefold. First, we considered in the analysis two different clinical end points: disease-free survival and overall survival. The use of two different possible end points is a way to evaluate to which extent the choice of the clinical end point can reverse the results of CEA when definite conclusions are not evident. Second, we handled for sampling uncertainty in the cost minimization analysis (using bootstrap tests for cost comparisons) and in the CEA with the building of confidence interval around the ICER obtained. Third, as there are many other potential sources of uncertainty in economic analysis, which derive from the assumptions that need to be made, a sensitivity analysis on the key modeling parameters was performed.
Cost data were based on a detailed observation of medical records, because the goal was to calculate the real cost of the procedures. They were available from the entry in the study until 6 months after the end of treatment for 300 patients included in the protocol. Our cost assessment showed an additional cost of 12,650€, HDC being 3.4 times more expensive than standard-dose chemotherapy. These cost estimates were robust to various sensitivity analyses carried out on the key components of the total cost (changes in the unit price of the three major cost factors, decrease in the number of hospitalization stays). This cost assessment appeared consistent with those recently published on HDC and peripheral blood stem cell support (2;11;27). In particular, De Rosa et al. (11) evaluated the cost of HDC with blood stem cell support for breast cancer patients at 20,816€, the larger part of this cost being due to hospitalizations during the posttransplant phase.
Survival times were assessed directly from the clinical trial and complete clinical results have been described elsewhere (unpublished data, 2004). Survival results did not show any benefit in terms of OS, whereas DFS was significantly increased for patients intensified. These results confirmed those obtained by Rodenhuis et al.(30), who reported a DFS advantage relative to HDC for very poor prognosis patients.
Although all health economists agree on the necessity of accounting for the uncertain nature of economic evaluation through appropriate statistical tests and the use of sensitivity analyses (17;22), two authors have noted recently that, in practice, very few published studies used methods to examine uncertainty (1;6). Uncertainty due to sampling fluctuations can be handled through the building of confidence regions for the ICER and various methods for calculating such confidence regions have been explored in the literature such as Taylor's method (25;33), parametric and nonparametric bootstrap methods (7), and Fieller's method (10;38). Recently, a study (32) has shown that the truncated Fieller's method is quite perfect and stable in all situations, even with skewed data and in the case of the pair (mean costs difference, mean effects difference) approaching zero. This method, therefore, was used here to calculate ICER confidence regions. However, several types of uncertainty are likely to be present in any economic analysis and the ICER confidence interval is a way of addressing a particular form of uncertainty, the one deriving from sampling variation. Consequently, it is not a total substitute for any other way of trying to estimate uncertainties, in particular sensitivity analyses that are required in any CEA.
The first end point considered in our CEA was DFS, as this was the main end point for the clinical evaluation of this protocol. The ICER was evaluated to approximately 13,074€ per disease-free life years gained.
We have shown that the upper bound of the 95 percent CI was equal to around 38,000€ per disease-free life years gained; that value is around three times more than the point estimate of the ICER. Consequently, we have to be cautious about the findings of this analysis, which are no longer obvious and depends on the maximum acceptable value (or ceiling ratio) that society is willing to pay. If society is willing to pay at least 38,000€ to gain 1 disease-free life year, then intensive treatment is a cost-effective strategy with a 5 percent error rate. Otherwise, questions have to be asked, and the choice will depend on the preferences and priorities of society, including preferences of patients.
Generally, cost-effectiveness analyses present results in terms of cost per life year saved, using the usual criterion of OS. In our analysis, no difference in OS was found between treatments, but the significant cost difference (extra cost = 7,403€) indicated that HDC is a dominated strategy and should not be retained on the basis of this criterion. This finding is in contradiction with the conclusions we obtained with DFS as effectiveness criterion (of course, in this case, cost of relapse was not included in the total cost). This illustrates the very general difficulty of evaluating treatments that slightly improve survival while significantly increasing DFS, what commonly occurs in the field of cancer. In practice, this situation arises in many empirical studies, as clinical trials are often designed to detect small differences between treatments and medical innovations often involve a deterioration of the cost-effectiveness ratio to obtain a limited improvement in efficacy when compared with standard treatments. The question of the choice of the efficacy criterion therefore arises, as it is crucial for decision making. Is the usually used criterion of 5-year OS a satisfactory and definitive criterion, because DFS at 5 years is significantly different? A 10-year OS may be a more appropriate criterion, especially in the context of therapeutic strategies where treatments of relapse are able to maintain patients alive. However, in this case, the criterion is problematic, as we cannot wait for 10 years to make a clinical and economic evaluation of an innovation: the use of an intermediate criterion, therefore, clearly is required. These questions certainly will be the ones addressed in the future, as medical evaluations are more and more confronted with situations where there is not survival benefit. Moreover, even if the DFS is not a definite efficacy end point, it seems to be particularly relevant in the context of treatments where quality of life (QOL) is now largely recognized as an important element in decision making. Facing the choice of the appropriate effectiveness criteria, QOL becomes an important argument as several studies (8;18;34) have proved that the recurrence disease is associated with a poorer QOL. The recurrent phase of cancer has been shown to affect a very large number of QOL dimensions: physical, functional, emotional, pain, distress, and in particular, psychological sequels are a major effect related to recurrence (9;21). In that context, a gain in DFS is clearly related to a benefit for patients, at least in terms of QOL, and the criterion of DFS, therefore, is of great relevance. So, even if no difference in OS is observed, questions about the role of HDC have to be addressed. This question can be considered as a problem of trade-off between economic considerations and a gain in DFS, that is, a gain in welfare. Is society ready to pay for a treatment that would certainly contribute to patients' well-being by reducing the risk of relapse? However, even if DFS is a relevant criterion, choosing this end point means considering implicitly that living after relapse has no value. This hypothesis being quite important, we recalculated the ICER after weighting the time spent after relapse, thus doing an implicit hypothesis on QOL. We noticed that, calculating the ICER with a weight of 0.9 (which means that the time spent after relapse is equivalent to 90 percent of the time spent without) yields a practically identical value (41,052€/QALYs) to that of the upper bound of the confidence interval calculated for handling uncertainty. Conversely, the ICER with a weight of 0.1 for time after relapse led to a close value (9,760€/QALYs) to the lower bound of the confidence interval. Thus, handling uncertainty for decision making with calculation of a confidence interval for the ICER can be related to an implicit choice on the value of life after relapse, that is, implicit QOL hypotheses that differ completely, depending on which bound we are locating.
Lastly, with regard to the generalization of our results, we must remember one major problem often associated with economic evaluations carried out alongside the diffusion process of innovations. In all evaluation of an innovative treatment, a change in medical practice between the trial study and the today's practice often occurs. Substantial cost savings may be achieved in the present case by changing practice patterns to reduce hospital stays of patients receiving high-dose treatments. Even larger cost savings are now conceivable with the development of outpatient treatment modalities, largely tested in various trials (24;26). Some substantial cost savings will presumably occur for the use of intensive chemotherapy in today's practice compared with the trial practice. In addition, care in clinical trials often requires more resources than routine clinical practice. In particular, the total cost of intensification was clearly overestimated in our study. These arguments, therefore, weigh in favor of intensification, if this strategy is adopted in routine practice as a standard treatment.
CONCLUSION AND POLICY IMPLICATIONS
The results of the present CEA suggest that the use of HDC should not be excluded, although a basic cost minimization study would provide arguments in favor of standard treatment as being less costly for the same efficacy in terms of survival. However, if one adopts DFS as end point, the value of the ICER may seem more reasonable to some decision-makers and this decision would encourage the development of intensification for breast cancer patients. One has to be careful, however, about how we interpret the point estimate of the ICER, as handling uncertainty indicates an upper bound for this ICER, making a definite conclusion about the economic acceptability of HDC difficult to reach.
Further research obviously now is required in the field of intensive therapeutics, considering complementary criteria such as quality of life. An evaluation of toxicity, QOL, and quality-adjusted survival analysis of this protocol is ongoing. It will certainly provide other arguments in the debate over the benefits of intensification of therapy for breast cancer patients.
CONTACT INFORMATION
Patricia Marino, PhD in process (marino@marseille.inserm.fr), Carole Siani, PhD (siani@marseille.inserm.fr), University of Méditerranée; INSERM Research Unit 379, 232, Bd de Sainte Marguerite, 13273 Marseilles Cedex 9, France
Henri Roché, PhD, Professor of Medicine (Roche@icr. fnclcc.fr), Institut Claudius Régaud, 20–24 rue du pont St. Pierre, 31052 Toulouse, France
Jean-Paul Moatti, PhD, Professor (moatti@marseille. inserm.fr), University of Méditerranée; INSERM Research Unit 379, 232, Bd de Sainte Marguerite, 13273 Marseilles Cedex 9, France
The PEGASE program was supported by the Ligue Nationale Contre le Cancer. The authors are grateful to the Fédération Nationale des Centres de Lutte Contre le Cancer (FNCLCC) who promoted the PEGASE program. The work of Patricia Marino was supported by the charity organization “Fondation de France.” The authors thank all the participating institutions. We thank Christian de Peretti (GREQAM, University of Aix-Marseille II) for providing us with his computer procedures about bootstrap tests for equality of average costs between two treatments.
