The increase in health-care expenditures and in the need for cost containment in many countries during the past decades have led to a growing interest in economic evaluations as a basis for decision making in health care (5;7;14;15;21). Scarce resources make it necessary to prioritize, and economic evaluations can, at least in theory, achieve an efficient use of these resources. Economic evaluations can provide valuable information in many types of decision making, for example, related to health-care financing, pricing, reimbursement, treatment guidelines, and investment in new technologies or research.
There is a rapid increase in the number of persons trained in health economics, and methodological standards are developing. Guidelines of performing cost-effectiveness studies, for example, have been published by several authorities, journals, and organizations (2;9;12;16). There also has been an exponential growth in publication of economic evaluations during the past 2 decades (19). However, despite this improving environment for use of economic evaluation in health-care decision making, the actual influence is still unclear (6;13;14). There are large variations in use across different medical technologies, diseases, countries, and so on. The pharmaceutical area is probably where economic evaluations have been most extensively used. Information about cost-effectiveness of new drugs is important for choices between different therapies and is also widely used in reimbursement and pricing decisions. Several countries are introducing mandatory requirements to submit economic evaluations in connection to reimbursement applications (1;2;9;12).
Although the impact of cost-effectiveness information in decision making is unclear, it is interesting to analyze how the evidence of cost-effectiveness was distributed among the new drugs introduced during the past years and also when the evidence was available. The objective of this study, therefore, was to analyze the amount of cost-effectiveness evidence for new drugs introduced on the Swedish market between 1987 and 2000, by systematically reviewing the published evidence of cost-effectiveness of these new drugs.
ROLE OF ECONOMIC EVALUATIONS OF NEW DRUG THERAPIES
Every new drug therapy involves a choice of how to use it. This choice is primarily based on criteria concerning efficacy, safety, and quality. However, because resources are scarce, it is not always possible to use the most effective therapy having the best safety and quality. Additional criteria, for example, cost, ethical, and social, must be considered in the choice between therapies. The increased influence of cost-effectiveness on the pricing and reimbursement decision has also made the evidence of cost-effectiveness a fourth area in which the manufacturers now need to provide evidence before they can gain market access (7;8).
The cost-effectiveness hurdle has the primary purpose to achieve a cost-effective use of drugs, that is, maximizing the gained health for the resources spent. Although a cost-effective use of drugs is not, and should not, be the only criteria for choosing therapies, achieving a cost-effective use of drugs would mean that we should reduce the use of drugs that are not cost-effective and increase use of those that are cost-effective. From an institutional perspective, shifting the use of drugs to those that are cost-effective is most important for drugs with large sale, because they often have the largest impact on the drug budget and are used by most patients. It is most rational, therefore, for the authorities to primarily get information about cost-effectiveness for drugs with a large sale. Analysis of cost-effectiveness, however, is not always necessary. Drugs with a proven added clinical benefit in combination with a lower price than the comparator will always be cost-effective, and no further analysis, therefore, is needed. The opposite situation is, of course, found for drugs with less effect and a higher price. Cost-effectiveness analysis can provide valuable information in those cases where a clinical benefit must be valued against a higher price. For pricing and reimbursement decisions, it is of primary interest, therefore, to have information about cost-effectiveness for drugs with improved effect or safety that are priced higher than previous treatment. This has been particularly stressed by some reimbursement authorities, who have required cost-effectiveness data only for drugs with a suggested price premium compared with previous alternatives (12;17). From a manufacturer's perspective it is most rational to produce cost-effectiveness information for drugs for which this information can increase the sale, that is, for drugs for which cost-effectiveness is considered a reason for using or not using them. This, in other words, means that they will produce economic evaluations that meets the decision-makers demand. It is also most important for the manufacturers to produce economic evaluations for drugs with large sale potential, because this strategy will lead to the largest revenue.
Study Hypotheses and Statistical Analyses
This study will assess the published evidence of cost-effectiveness for new chemical entities (NCEs) introduced in Sweden between 1987 and 2000. NCE drugs contain chemical entities not previously used in any drug formulation. Two hypotheses will be tested using linear regression models. The first will analyze factors related to the amount of cost-effectiveness evidence. The hypothesis is that drugs with improved effectiveness or safety (high therapeutic benefit) will have more published evidence of cost-effectiveness than drugs with low therapeutic benefit. The approval year is included as an explanatory variable in the model, because the included drugs have been on the market for different time periods and, thus, have had different lengths of time to be evaluated. An ordered logit regression model is used, because the amount of cost-effectiveness evidence will be measured using an ordinal scale.
The second hypothesis will analyze the relation between the sale of the NCE drugs and the evidence of cost-effectiveness. The hypothesis is that drugs with large sale will have more evidence of cost-effectiveness than drugs with low sale. The approval year is again included as an explanatory variable, because drugs have been on the market for different time periods but also because the pharmaceutical sales vary over time. Because sales data are expected to have a skewed distribution, a linear regression model with untransformed variables may not follow the underlying assumptions of the model. Therefore, the regression model is fitted using a Box-Cox transformed dependent sales variable. A large part of the drugs are expected to have zero sales, and only drugs with a sale above zero are included in the analysis. The reason for this approach is that the factors influencing the probability that a drug has a sale may be different from the factors influencing the size of the sale conditional on having a sale at all.
Material
In total, 442 new NCE drugs, approved in Sweden between 1987 and 2000, were analyzed. Information on approval year and therapeutic class was obtained from the Medical Products Agency in Sweden, and sale statistics were obtained from the National Corporation of Swedish Pharmacies (3). Information about the innovative therapeutic benefit was obtained from a previous published study (20) and from unpublished data from the authors of that study. The therapeutic benefit was divided into three groups, A–C, with the following definitions (20):
- Group A: Markedly to modestly effective therapy or diagnosis for a disease not adequately prevented, treated, or diagnosed before.
- Group B: Moderately to markedly improved effectiveness, safety, route of administration, or dosage compared with other marketed drugs.
- Group C: Little or no therapeutic gain.
Information on published studies for the NCE drugs was obtained from the Health Economic Evaluation Database (HEED; 19). HEED was used as a source of cost-effectiveness information, because it only contains health economic studies and because it also includes summaries of the studies. Relevant studies were identified by searching for brand and generic names of the NCE drugs.
The studies found in the database were reviewed and questions related to different characteristics (study type, outcome measure, comparator, results, and quality) of the studies were analyzed. The questions were only registered for those studies that gave some information about the cost-effectiveness of any of the NCE drugs. One requirement was that the study must be a full economic evaluation, that is, give information about both costs and effects of the therapy. Reviews of cost-effectiveness studies were not included, because the information in these reviews already could be found in the original studies. The analysis was based on the summaries included in the HEED.
After identification of all relevant studies for a specific drug, the evidence of cost-effectiveness was assessed based on the answers to the questions. All NCE drugs were classified according to the degree of cost-effectiveness evidence on a three-graded scale. The criteria used in the classification were similar to criteria used in previous health technology assessments (4):
- Group 1, much evidence of cost-effectiveness Criteria: At least two relevant cost-effectiveness analyses, cost-utility analyses, or cost-consequence studies, where the drug either was cost saving and not less effective or had a cost-effectiveness ratio less than 50,000 USD per life years/quality-adjusted life years (QALY) gained (11;18;22). One of the questions was about the relevance of the study, and a few studies were considered irrelevant. The major reason was that the study was clearly defective in some aspect.
- Group 2, some evidence of cost-effectiveness Criteria: Several possible alternatives, for example, only one study according to the criteria above, cost-minimization analyses or studies with questionable outcome measures.
- Group 3, little evidence of cost-effectiveness Criteria: No studies fulfilling the criteria for the two classes above.
Descriptive statistics of the amount of cost-effectiveness evidence in different therapeutic classes and in different years will be reported, and the two study hypotheses will be tested with regression analysis.
RESULTS
The search of HEED identified 3,138 articles. A large part of the articles was not further analyzed, because they either did not give complete information about the cost-effectiveness or because they were review articles. The articles selected for further analyses were approximately one fourth of the original number of articles. Approximately 10 percent of these selected articles were cost-utility analyses with QALY as outcome measure, and another 15 percent had life years gained as outcome measure.
Table 1 shows that 265 of the 442 drugs did not have any article studying its cost-effectiveness. The table also shows the distribution of the NCE drugs by different therapeutic benefit groups and the number of articles evaluating the drugs in the different groups. The two groups with the lowest therapeutic benefit also had the largest share of drugs without any published studies.
The distribution of the drugs by different approval year shows that the number of cost-effectiveness articles per drug decreases with the introduction year (Table 2). This finding may be explained by the fact that drugs introduced early have had more time to be evaluated, but there are also large variations in the number of published studies during different years. The data for drugs approved during the last years are uncertain, because there is a time lag before studies are included in the HEED.
Time of Publications
The publication years of the articles reviewed in HEED were compared with the approval years of the drugs, and the lag between approval and publication was calculated (Figure 1). Most articles were published approximately 1 to 5 years after the approval year and very few articles were published before or during the approval year. However, it is difficult to draw any further conclusions, because the drugs were approved during many different years. Only drugs approved during the first years, for example, could have articles published a long time after the approval.
The drugs and the published articles were also divided by the anatomic therapeutic chemical class (ATC) of the drugs. ATC groups J (drugs against infectious diseases) and N (drugs against diseases in the central nervous system) had most new drugs approved and ATC groups J, N, and A (drugs against diseases in the gastrointestinal tract) had the highest average number of published articles per approved drug. It is again difficult, however, to make any conclusions from this finding, because individual drugs with a large number of studies influenced the mean number of studies in some therapeutic classes.
Classification of Evidence of Cost-Effectiveness
The classification of the drugs by the amount of evidence of cost-effectiveness placed 51 drugs into group 1 (much evidence), 84 into group 2 (some evidence), and 307 into group 3 (little evidence). Table 3 presents the number of drugs in the different cost-effectiveness and therapeutic benefit groups and shows that therapeutic benefit group B had a slightly larger share of drugs in the highest evidence group.
Sale Statistics
A histogram of the average yearly sales during the first 3 years after approval shows a skewed distribution for which most drugs had a very low sale and a small group of drugs had a very high sale (Figure 2). Only drugs approved before 1998 were included in the analysis because sale statistics only were available to year 2000.
Average yearly sale during the first 3 years after approval. Share of the total number of drugs yearly sale.
Regression Models
The two study hypotheses were tested with the regression models specified above. The first regression model (regression 1 in Table 4) shows that the approval year and therapeutic benefit group B had significant (at a 0.05 level) relation with the cost-effectiveness evidence group. Drugs with early approval year and drugs in therapeutic benefit group B were related to a better cost-effectiveness evidence group. This finding indicates that drugs with improved effectiveness, safety, route of administration, or dosage compared with other drugs had more published evidence of cost-effectiveness than other drugs, which thus supports the first study hypothesis.
The second regression model (regression 2 in Table 4) shows that the variables cost-effectiveness evidence group 3 and approval year were significant (at a 0.05 level) related to a lower sale. This finding, thus, supports the second study hypothesis, saying that there is cost-effectiveness information for drugs with larger sale.
DISCUSSION
During the 14 years analyzed in the study, there were a large number of new drugs approved and also a very large number of published economic evaluations. We reviewed the published articles and evaluated whether they gave any information about cost-effectiveness of the NCE drugs. Analyzing the amount of published cost-effectiveness evidence, however, is different from analyzing the cost-effectiveness of the drugs, because many drugs may be cost-effective even though there has not been published any study to show this finding. No economic evaluations are required, for example, for drugs that are obviously cost-effective, that is, are more effective and have lower prices. The amount of cost-effectiveness evidence in our study depended on two factors: the cost-effectiveness of the drug and the number of published cost-effectiveness studies for the drug. The cost-effectiveness also varies over time, as prices, comparators, and so on changes. Also, we did not make any distinction between comparison alternatives or in which patients the drugs were used, which means that a study was found to show evidence of cost-effectiveness if the drug was found cost-effective against any comparison alternative in any patient population. This approach means that all drugs could theoretically be found cost-effective against some alternative. This way to measure the cost-effectiveness evidence was used because it had not been possible to value the cost-effectiveness compared with chosen alternatives, because studies comparing all relevant alternatives were not available. The purpose of the study was not to compare the cost-effectiveness of individual drugs, but to assess the total amount of cost-effectiveness evidence.
Most articles studying the cost-effectiveness of a drug found the drug to be cost-effective against some alternative, which meant that the number of articles giving some cost-effectiveness information was highly correlated with the classification of cost-effectiveness evidence. The limited number of studies reporting drugs to not be cost-effective is interesting information, considering the issue of publication bias.
The timing of the publications showed that most articles were published approximately 1 to 5 years after approval of the drugs. It is difficult, however, to make any further conclusions about this timing, because the number of articles published in different year varies very much. Drugs approved recently were more likely to have articles published early after approval, probably because the total number of publications has increased during the years.
The regression analyses indicated that drugs with improved effectiveness or safety over other marketed drugs were more likely to have more evidence of cost-effectiveness than drugs in the other two therapeutic benefit groups. This finding supports the hypothesis that new and more effective or safe drugs in competitive therapeutic submarkets would be more often studied as it is interesting for both producers and the health-care financiers to analyze if the added clinical benefit is worth the extra price premium these drugs often obtain in relation to its competitors (10).
The analyses also found that drugs with low sale were less likely to have evidence of cost-effectiveness than drugs with high sale. This, in other words, indicates that drugs with high sale more often were evaluated in published studies, which thus supports the hypothesis that it is most interesting to have cost-effectiveness information for drugs with large sale. It is, however, difficult to make interpretations of the results because the causality between the variables is unknown. The relation between cost-effectiveness evidence and sales could, for example, indicate that drugs with high sale more often are studied in cost-effectiveness evaluations, but it could also be an indication that cost-effectiveness evidence can lead to larger sale.
The results, however, must be interpreted with care, because there were weaknesses in the analyses. The number of published articles was obtained from all years, whereas sale was measured during the first 3 years after approval. Also, the valuation of cost-effectiveness evidence was based only on the summaries in HEED. The full publications were not reviewed, which might have influenced the classification. More studies of the influence of cost-effectiveness evidence are needed to be able to make any further conclusions.
The results of this study indicated that the publication of cost-effectiveness information for the new drugs introduced in Sweden between 1987 and 2000 may be considered rather rational, that is, the economic evaluations were performed for drugs for which this information may be considered most important. The results also showed that most of the studies were published between 1 and 5 years after introduction. This timing may not be considered rational, because information about cost-effectiveness is very important at the introduction of new drugs, for the pricing, reimbursement, and formulary decisions. On the other hand, it may be considered rational to wait for reliable data before making the evaluations. The lack of published evaluation before or early after approval may indicate that early modeling studies based on clinical trials was unusual for the drugs assessed in this study. This finding may reflect a desire to have more-reliable real-world data before making the studies or may also indicate that conducting economic evaluations was not a common part of the drug development process at the pharmaceutical companies during these years. It is also possible, however, that the cost-effectiveness information was available for decision-makers before the studies were published. There may also be economic evaluation published in sources not included in HEED.
Of all new drugs analyzed here, published economic evaluations were found for only approximately half of them. However, as more authorities are demanding economic evaluations, it will probably be easier to assess the cost-effectiveness of new drugs in the future, because more studies then will be available. Early information about cost-effectiveness is important for decision making about pricing and reimbursement. The early information, however, is often limited by the lack of data on the consequences of a treatment in real clinical practice, in particularly the long-term consequences. It is becoming increasingly important to reassess the costs and consequences of treatments, as more information becomes available. It is plausible, therefore, that the amount of cost-effectiveness information published a fairly long time after introduction of a drug also will increase in the future.
Policy Implications
As cost-effectiveness is becoming a more important criterion for adopting new medical technologies, it is important that cost-effectiveness information of appropriate quality is available when decisions are being made. The increasing demand for economic evaluations for pricing and reimbursement decisions will probably lead to much more focus on early cost-effectiveness studies based on clinical trial data. It is, however, also important to reassess decisions based on early information, as more reliable data becomes available.
CONTACT INFORMATION
Jonas Lundkvist, MSc (jonas.l@healtheconomics.se), Medical Management Centre, Karolinska Institute, 171 77 Stockholm, Sweden
Bengt Jönsson, PhD (Bengt.Jonsson@hhs.se), Professor, Center for Health Economics, Stockholm School of Economics, P.O. Box 6501, 113 83 Stockholm, Sweden
Clas Rehnberg, PhD (clas.rehnberg@mmc.ki.se), Associate Professor, Medical Management Centre, Karolinska Institute 171 77, Stockholm, Sweden
