In a clinical trial where a new treatment for cancer is being compared with a standard or alternative treatment, it will generally be the case that reliable data for disease-free or progression-free survival will become available far sooner than reliable data for overall survival. Therefore, despite the fact that gains in overall survival may be the main outcome of interest when deciding on whether or not to approve a new treatment, this decision will often need to be made on the basis of information relating to disease-free or progression-free survival.
Furthermore, for deciding upon whether to make a new drug freely available within a publicly funded health service it is often the case, and is becoming more the case, that gains in actual overall survival times, as measured through say the median overall survival time, are of greater importance than hazard ratios or relative risks for overall survival. This is due to the fact that calculations involving QALYs (quality-adjusted life-years) have become increasingly used in recent years as the basis for deciding upon the cost-effectiveness of new treatments. For example, in the United Kingdom, the size of the gains in QALYs caused by a new treatment is a critical factor in determining whether or not the treatment is approved by NICE (National Institute for Clinical Excellence).
The aim of this study is to attempt to determine how many months it would be expected that a new drug will add to overall survival times for metastatic cancer given that the number of months the drug adds to progression-free survival times relative to a standard drug is roughly already known. This will be achieved by analyzing results from trials for other drugs that have advanced sufficiently into their follow-up periods such that median times for both progression-free survival and overall survival are available.
The two types of metastatic disease that will be focused upon are metastatic breast and metastatic colorectal cancer. Breast cancer was chosen because, apart from nonmelanoma skin cancer, it is the most common type of cancer within the United Kingdom in terms of incidence (according to Cancer Research UK 2005 data), and colorectal cancer was chosen because it has become a standard cancer to examine with regard to the general type of issue being addressed in the present study (Reference Buyse, Burzykowski and Carroll5;Reference Johnson, Ringland and Stokes9;Reference Louvet, Gramont and Tournigand11;Reference Piedbois and Buyse12;Reference Tang, Bentzen, Chen and Siu15).
A natural starting point for the present study is the work of Johnson and colleagues in relation to metastatic colorectal and non–small-cell lung cancer (Reference Johnson, Ringland and Stokes9) and the work of Tang and colleagues in relation to metastatic colorectal cancer (Reference Tang, Bentzen, Chen and Siu15). Using results from past trials, these studies examined the effect on median overall survival of differences in median time to disease progression between treatment and control groups. The results from these studies demonstrated the existence of a positive correlation between changes in overall survival and changes in time to progression for both metastatic colorectal cancer (with correlation coefficients of 0.3 and 0.52 for each of these studies) and non–small-cell lung cancer (with correlation coefficient of 0.6). However, the existence of a positive correlation between changes in these two variables simply suggests that gains in time to progression do not completely disappear during the time between disease progression and death, that is, an x month gain in time to progression is not reduced, on average, to a zero gain in overall survival. In particular, the existence of this positive correlation is consistent with any of the following three theories being true: Theory 1. Increases in median time to progression (TTP) most commonly lead to increases in the time from disease progression to death, that is, the post-progression survival time. Theory 2. Increases in median TTP generally lead to little change in post-progression survival. Theory 3. Increases in median TTP most commonly lead to decreases in post-progression survival.
If it could be resolved which of these three theories is true, then it would be possible to know whether an x month gain in time to progression generally leads to more than an x month gain in overall survival (Theory 1), to roughly an x month gain in overall survival (Theory 2) or to less than an x month gain in overall survival (Theory 3). Following on from a previous preliminary study (Reference Bowater, Bridge and Lilford2), the aim of the present study is assess the degree of support for each of these three theories.
METHODS
A literature survey was conducted over Medline for randomized controlled trials (RCTs) of treatments for metastatic breast and metastatic colorectal cancer published between January 1980 and August 2008. In particular, articles were searched that had been identified as being RCTs through the work of the Cochrane Collaboration (Reference Higgins and Green8). The string combinations searched for over all fields consisted of one item from both lists A and B in Table 1. Articles had to be written in English. For trials found through this search, the inclusion criteria for this study were as follows:
(i) Patients included in the trial either all had to have distant metastatic disease (Stage IV or Dukes D disease) or at least a proportion of the patients had to have distant metastatic disease with the rest of the patients having locally advanced disease (Stage III or Dukes C disease).
(ii) A drug treatment had to be administered to both the treatment and control groups. The drug or combinations of drugs given to the treatment group had to be different from the drug or combination of drugs given to the control group. Any other interventions had to be applied in the same manner to both the treatment and control groups. Note that trials that compare drug treatments with no treatment or with different doses of the same drugs or with treatments involving non-pharmaceutical interventions were excluded. This was done to maintain relevance to the scenario where a regulatory or funding body has to decide between an existing standard drug therapy and a new alternative drug therapy.
(iii) The median time to disease progression and the median overall survival time had to be reported for both the treatment and control groups somewhere in the full article if the publication year was 1995 or later or in the abstract of the article if the publication year was earlier than 1995.
Table 1. String Combinations Used in the Literature Search (At least one item from both lists A and B).
As well as median values for time to progression and overall survival for both control and treatment groups being extracted, the p-value for the change in time to progression based on the log-rank test was also recorded for each trial found by the literature search where it was available. This was done to assess the statistical significance of treatment effects in the various trials. The focus of the analysis was on p-values for treatment differences in terms of time to progression rather than in terms of overall survival due to the fact that a negligible change in median overall survival is consistent with the scenario where a substantial gain in median time to progression is combined with a substantial fall in post-progression survival or vice-versa. This type of scenario would obviously be of interest given the types of hypothesis being tested.
If a trial contained two treatment groups (A and B) and a control group, then the only treatment comparisons included in the study were against the control group, that is, a direct comparison between the groups A and B was excluded. This was done because of the high degree of interdependence between treatment comparisons that would have resulted if this latter comparison had been included. If one of the treatment groups was not clearly defined as being the control group, then the control group was chosen to be the last group for which the median time to progression was reported.
As already stated, the aim of this study was to assess the degree of support for the three theories outlined in the Introduction. These three theories are illustrated in Figure 1. Postprogression survival was defined as the difference between median overall survival and median time to progression. To allow standard statistical tests to be performed, these three theories were assessed within the context of several specific hypotheses.
Figure 1. Guide to theories and hypotheses being tested.
With regard to Theory 1 (that increases in median TTP most commonly lead to increases in post-progression survival), we tested whether percentage gains in post-progression survival (PPS) are greater than or equal to (as opposed to less than) percentage gains in median TTP (e.g., median TTP increases from 8 to 12 months and PPS increases from 12 to at least 18 months) and whether percentage gains in PPS are greater than or equal to half of the percentage gains in median TTP (e.g., median TTP increases from 8 to 12 months and PPS increases from 12 to at least 15 months). The first of these two hypotheses corresponds to line (a) in Figure 1 and the second hypothesis corresponds to line (b). With regard to Theory 2 (that increases in median TTP generally lead to little change in PPS), we tested whether there are no gains or losses in PPS (which corresponds to line [c] in Figure 1). With regard to Theory 3 (that increases in median TTP most commonly lead to decreases in PPS), we tested whether percentage losses in PPS are greater than or equal to (as opposed to less than) percentage gains in median TTP (which corresponds to line [e] in Figure 1) and whether percentages losses in PPS are greater than or equal to half of the percentage gains in median TTP (which corresponds to line [d] in Figure 1).
Note that these hypotheses are based on percentage changes in survival times rather than absolute changes in survival times. This was done to be consistent with the accepted principle of meta-analysis that the best measures of effect size for forming combined estimates are those that are independent of trial baseline measures, especially when such measures would be expected to vary substantially between trials, for example, baseline or control survival rates.
Results for the treatment comparisons found by the literature search were plotted in the type of diagram given in Figure 1. For ease of graphical presentation, this type of diagram is set up so that the decision on whether to look at changes in going from control to treatment or treatment to control is made to ensure that the change in median TTP is always positive. Therefore, in terms of Figure 1, the median TTP for the treatment group can never be less than the median TTP for the control group. If there was exactly no difference in median TTP between the control and treatment groups then the treatment comparison concerned would carry no information regarding the relationships between changes in median TTP and changes in post-progression survival that are under investigation. Therefore, these treatment comparisons were not included in the main analysis.
In order for gains in post-progression survival to be illustrated in a way that is symmetrical to losses in post-progression survival, the vertical axis in Figure 1 is defined differently either side of the level representing no change in post-progression survival. In particular, if for any treatment comparison, there is a percentage increase in PPS in going from control to treatment, then this increase is plotted above the “no change” level, whereas if there is a decrease in PPS in going from control to treatment, then the percentage increase in PPS in going in the opposite direction, that is, from treatment to control, is plotted below the “no change” level.
Theories 1 to 3 outlined in the Introduction were statistically tested using Spearman's rank correlation test and the sign test. In particular, Spearman's rank correlation test was used to test for the presence of positive or negative correlations between percentage gains in median TTP and percentage changes in PPS. The sign test was used to measure the support for the statistical hypotheses outlined above, that is, the hypotheses that correspond to lines (a) to (e) in Figure 1. The test statistic for the sign test was the number of treatment comparisons that lie above the line associated with the given hypothesis, as illustrated in Figure 1, divided by the number of comparisons that lie below this line. This test was performed under the null hypothesis that the population value for this test statistic equals one.
RESULTS
The number of RCTs found by the literature search that satisfied the given inclusion criteria was 95 and 74 for metastatic breast and colorectal cancer, respectively, of which 77 and 69, respectively, were published in the period 1995 to 2008 (and, therefore, for which survival times not reported in the abstract were searched for in the full article). Moreover, from all the RCTs found, the number of individual treatment comparisons that satisfied the inclusion criteria and that qualified as being sufficiently independent to be included (according to the given exclusion criteria) was 102 and 85 for breast and colorectal cancer, respectively. The number of these treatment comparisons for which there was exactly no difference in median time to progression between the control and treatment groups was four and seven for breast and colorectal cancer, respectively. This meant that 98 comparisons for breast cancer and 78 comparisons for colorectal cancer entered into the main analysis. The references for these studies are given in Supplementary Tables 1 and 2, which can be viewed online at www.journals.cambridge.org/thc2011013.
Figures 2 and 3 are plots of the percentage change in post-progression survival against percentage gain in median time to progression between the control and treatment groups for metastatic breast and colorectal cancer, respectively. The type of diagram used in these figures is the same as the type of diagram discussed earlier that is shown in Figure 1. Each point in Figures 2 and 3 represents a treatment comparison included in the study, with the style and shade of the points indicating the statistical significance level of the difference in time to progression between the control and treatment groups.
Figure 2. Trial results for metastatic breast cancer.
Figure 3. Trial results for metastatic colorectal cancer.
It can be seen that, for both Figures 2 and 3, the points cluster more around line (c) than lines (a), (b), (d) and (e). This implies greater support for Theory 2 (that increases in median TTP generally lead to little change in post-progression survival) than for the theory that increases in median TTP commonly lead to substantial gains or losses in post-progression survival (which is consistent with Theories 1 and 3).
This graphical analysis is backed up by the results of a statistical analysis. In particular, for both metastatic breast and colorectal cancer there is no statistical evidence to suggest that percentage changes in PPS are correlated with percentage changes in median TTP (as indicated by the Spearman rank correlation test producing p-values of .37 and .11 for breast and colorectal cancer, respectively). This conclusion is not affected by whether or not the analysis is restricted only to treatment comparisons where the change in time to progression is statistically significant at the 5 percent level of significance (as indicated by the Spearman rank correlation test for this restricted analysis producing p-values of .78 and .54 for breast and colorectal cancer, respectively).
If the hypothesis that increases in median TTP lead to no gains or losses in PPS (which corresponds to line [c] in Figure 1) is directly assessed by using the sign test then it is not rejected at the 5 percent level of significance for either breast or colorectal cancer (with p-values of .08 and .57, respectively). On the other hand, if the hypothesis that percentage gains in PPS are greater than or equal to half of the percentage gains in median TTP (which corresponds to line [b] in Figure 1) and the hypothesis that percentage losses in PPS are greater than or equal to half of the percentage gains in median TTP (which corresponds to line [d] in Figure 1) are separately assessed by the sign test then these two hypotheses are rejected at the 0.1 percent level of significance for both breast and colorectal cancer. The results of these sign tests, therefore, imply greater support for Theory 2 than for the theory that increases in median TTP commonly lead to substantial gains or losses in post-progression survival (which is consistent with Theories 1 and 3). Again the conclusions of this statistical analysis are not sensitive to whether or not the analysis is restricted only to treatment comparisons where the change in time to progression is statistically significant at the 5 percent level of significance (i.e., for both cancers, the former hypothesis is still not rejected at the 5 percent level of significance while the latter two hypotheses are still rejected at the 0.1 percent level of significance).
Over all treatment comparisons, the average percentage change in median TTP from control to treatment (which is always a gain under the definitions being used) is 36 percent and 32 percent for breast and colorectal cancer, respectively. By inverting the sign test and assuming a stable linear relationship between percentage changes in PPS and percentage changes in median TTP, it can be stated with a 95 percent confidence level that a 36 percent increase in median TTP for metastatic breast cancer would result, on average, in percentage changes in PPS lying somewhere between gains limited to 10.7 percent or losses limited to just 0.3 percent. By applying the same assumptions to the data for colorectal cancer, it can be stated with a 95 percent confidence level that a 32 percent increase in median TTP for this metastatic cancer would result, on average, in percentage changes in PPS lying somewhere between gains limited to 9.8 percent or losses limited to 4.4 percent. There is good evidence, therefore, that if changes in post-progression survival are not, on average, very small compared relative to changes in median TTP, they will at least be in the same direction as changes to median TTP. As a result, it can be concluded that, if gains in overall survival are not, on average, similar to gains in median TTP, they will be at least as large as gains in median TTP.
Over all trials and treatment groups, the average time to progression for metastatic breast cancer is 6.9 months and the average time to death is 20.6 months. For metastatic colorectal cancer, average time to progression is 6.0 months and average overall survival is 14.4 months. On the basis of these average survival times, the dotted curves in Figures 2 and 3, indicate the losses in post-progression survival that are required so that gains in median TTP are, on average, canceled out; that is, overall survival does not increase despite gains in median TTP. The percentage of treatment comparisons that lie above these curves, and are, therefore, consistent with overall survival increasing as a result of an increase in median TTP, is 85 percent and 83 percent for breast and colorectal cancer, respectively, and is 95 percent and 97 percent, respectively, with regard to only treatment comparisons where the change in TTP is statistically significant at the 5 percent level. According to the sign test, the percentage of treatment comparisons lying above these curves is statistically greater than 50 percent at the 0.1 percent level of significance in all these instances.
Figures 2 and 3 both contain a small number of outlying points. For the treatment comparison labeled as point 1 in Figure 2 (which corresponds to study 59 in Supplementary Table 1), the median TTP is 5.5 and 6.4 months for the control and treatment groups, respectively, whereas PPS is 11.8 and 4.9 months, respectively. For point 3 in Figure 2 (study 81 in Supplementary Table 1), median TTP is 3.0 and 6.3 months for the control and treatment groups, respectively, whereas PPS is 8.1 and 4.1 months, respectively. Finally, for point 2 in Figure 3 (study 61 in Supplementary Table 2), the median TTP is 4.2 and 7.2 months for the control and treatment groups, respectively, whereas PPS is 3.0 and 8.7 months, respectively.
However, the interpretation of these three outlying points perhaps should be made in the light of additional relevant information. In particular, the vertical positions of point 1 in Figure 2 and point 2 in Figure 3 have been determined using overall survival data that are based on a very small number of patients with the result that the confidence intervals for median overall survival in these two cases are very wide. Furthermore, point 3 in Figure 2 is one of three treatment comparisons for metastatic breast cancer based on a cross-over trial design where patients in the treatment group switch to the control treatment at the point of disease progression and vice-versa. The outcome for point 3 is consistent with patients who did not receive the better treatment in the progression-free phase benefitting from this treatment in the post-progression phase. Note that none of the treatment comparisons for metastatic colorectal cancer are based on cross-over trials.
DISCUSSION
On the basis of an analysis of past trial results, it would appear that drugs for metastatic breast or colorectal cancer that extend, by a given amount, the time period between the start of treatment and disease progression (i.e., time to progression) have a strong tendency to extend, by roughly the same amount, the period between the start of treatment and death (i.e., overall survival).
One of the main strengths of the approach that has been used is that relationships have been analyzed between actual survival times and not between the hazard ratios or relative risks for progression-free and overall survival as has been done in previous studies (Reference Ballman, Buckner and Brown1;Reference Burzykowski, Buyse and Piccart-Gebhart3;Reference Burzykowski, Buyse, Yothers, Sakamoto and Sargent4;Reference Buyse, Burzykowski and Carroll5;Reference Hackshaw, Knight, Barrett-Lee and Leonard7;Reference Lamborn, Yung and Chang10;Reference Piedbois and Buyse12;Reference Sargent, Wieand and Haller13). This has allowed direct conclusions to be drawn with regard to the kind of outcome variables, for example, time to progression in months and overall survival in months, that are of great importance to both clinicians and policy makers. In addition, this study has not examined relationships in outcome variables within only the control or treatment groups as has been done by other researchers (Reference Edwards, Bonodonna, Valagussa and Gamel6;Reference Louvet, Gramont and Tournigand11;Reference Sherrill, Hirst, Wu, Amonkar and Stein14) but relationships in the differences in outcome variables between treatment and control groups. Therefore, this study is relevant to the type of scenario where a decision needs to be made about the efficacy of a new treatment in relation to an established treatment.
The main weakness of this study is that it is of course based on a retrospective analysis. Therefore, the conclusions that have been drawn do not take account of the possible arrival in the future of a new class of treatments for metastatic cancer for which the relationship between time to progression and overall survival does not fit into the pattern reported within this study. The conclusions of this study also do not take account of future changes in practices and policies with regard, for example, to how rapidly and frequently patients are switched between treatments due to drug resistance. In response to this though, it should be noted that the results presented in this study are not tied to a particular philosophy, policy, or drug treatment but instead are based on the various practices, policies, and drug treatments that have been employed in treating metastatic breast or colorectal cancer since the 1980s.
SUPPLEMENTARY MATERIAL
Supplementary Table 1
Supplementary Table 2
CONTACT INFORMATION
Russell J. Bowater, BSc, PhD (rjb@xanum.uam.mx), Assistant Professor, Department of Mathematics, Universidad Autónoma Metropolitana, Unidad Iztapalapa, AT-242, Av. San Rafael Atlixco No. 186, Col. Vicentina, 09340, Mexico City, Mexico.
Philippa E. Lilford, Medical student, School of Medicine, University of Southampton, Boldrewood Campus, Bassett Crescent East, Southampton SO16 7PX, UK
Richard J. Lilford, PhD, FRCP, FRCOG, FFPHM, Professor of Clinical Epidemiology, Department of Public Health, Epidemiology and Biostatistics, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
CONFLICT OF INTEREST
All authors report that they have no potential conflicts of interest.
