Increasing costs for drugs and other healthcare interventions in combination with limited available resources and a more evident cost responsibility has led to an increased interest in the cost-effectiveness of health care. The introduction and sale of drugs is a highly regulated process. The first step is the market authorization, which is granted by a medical agency, for example, the Food and Drug Administration or the European Medicines Agency (EMEA) in collaboration with national agencies like the Medical Products Agency (MPA) in Sweden. These agencies evaluate the quality, efficacy, and safety of the drugs, based on information supplied by the manufacturers. The price of the drugs, and the cost-effectiveness of them, is currently not a criterion for market authorization, and information about these aspects are, therefore, not required by the pharmaceutical agencies. This information is, however, often requested in the next step of market introduction, the pricing and reimbursement process. Mandatory requirements for economic evaluation in reimbursement decisions have during the past decades been introduced by reimbursement authorities worldwide. In Sweden, for instance, the establishment of the Pharmaceutical Benefits Board in 2002 has led to a requirement of manufacturers to submit health economic evaluations as part of their applications for reimbursement. Pharmaceutical companies have, therefore, economic incentives to provide cost-effectiveness information, and a general interest in health economics has, as a result, grown among both buyers and sellers of pharmaceuticals (2).
Although there are many different techniques for conducting economic analyses of medical interventions, a common approach is to collect data on outcomes and costs along clinical trials (piggyback studies). This way of conduct has certain advantages but also limitations regarding fundamental study design (6). In addition to design problems, economic evaluation in the context of clinical studies has rendered further criticism due to claims that marketing objectives exceed scientific purposes when it comes to industry sponsored studies (3). In an attempt to meet the critique and improve quality, gain credibility, and extend the usefulness of health economic evaluations, experts worldwide have tried to bring about a research consensus about the methods used in this type of studies. Development of a standard approach or guidance document for conduct and report of economic evaluations has been aimed at, as well as a better understanding of, in which settings different methods are appropriate (4). Despite various attempts to bring about uniformity, the literature reveals a great variation of methodology and reporting of studies. The International Society for Pharmacoeconomics and Outcomes Research task force group presented in 2005 a report of areas of consensus regarding the methodology of joint clinical/economic analyses (8).
Performing economic evaluations alongside clinical trials is common, and various standard approaches and guidelines on appropriate conduct have been developed. However, because there is no requirement on cost-effectiveness for market authorization of a drug, the health economic parts of trial protocols are generally not thoroughly reviewed before conduct. The information available about how the quality of these protocols compares to the suggested standards is, therefore, also limited. To shed some light on this issue, this study aims at learning about the occurrence and design of health economic evaluations conducted alongside clinical trials in Sweden between 1995 and 2005.
MATERIAL AND METHODS
An examination of clinical trial protocols that had arrived as applications to the MPA between 1995 and 2005 was performed. Protocols arriving during the first half of odd years within the time period (i.e., 1995, 1997, 1999, 2001, 2003, and 2005) were included. This limitation was used to reduce the number of protocols to review, without reducing the total evaluated time span. A preliminary extended analysis of the material showed no exceptional seasonal variation in the occurrence of health economic protocols. A period of 10 years was chosen to get an indication of the change in occurrence during a longer time period. Only protocols concerning phase III (late premarketing studies) and phase IV trials (postmarketing studies) were considered, because economic evaluation alongside phase I and II trials are very rare. Information on date of arrival was obtained from an internal database of the authority (for protocols arriving during 1995–2003) as well as a central database of the EMEA (for protocols arriving during 2005). For each year of interest, a list of all phase III and phase IV study protocols that had arrived to the MPA during the period of January 1st to June 30th was obtained. The protocols were identified by protocol specific numbers (registration number, protocol number, or EduraCT-number) and collected from the archive or from the MPA employee assessing the protocol at the time.
To ensure that all the protocols were examined consistently, each protocol was examined by following a checklist of variables. Information in protocol amendments was not considered. Protocols (2, 9 percent) that could not be found in the archive or in the office where it was registered to be were not further searched for and, hence, excluded from the analysis.
The protocols were classified according to indication for use of the investigated substance/drug as well as the Anatomical Therapeutic Chemical (ATC) classification. The ATC system is used to classify drugs according to the organ system they target. Only the first letter in the ATC code, that is, the main group of the ATC system, was considered.
RESULTS
Occurrence
A total number of 680 protocols from 1995 to 2005 were examined and among them 98 (14.4 percent) included a health economic part. The presence of an economic evaluation as part of the protocol varied from 10.7 percent in 1995 to 16.3 percent in 2003 (Table 1). Among all the examined phase III protocols, 18 percent contained a health economic part as compared with 7 percent among the phase IV protocols.
Ninety-two percent of the protocols containing health economic evaluations were commercial, that is, industry-sponsored studies. Seventeen percent of all commercial trials contained a health economic study plan compared with 5 percent of all noncommercial trials.
Setting
Among all multicenter trial protocols, 19 percent included an economic evaluation, whereas 11 percent of the national multiple-site trials contained a health economic evaluation and less than 1 percent of the national single site studies.
Time Frame
The majority of the protocols did not clearly define the time frame for the economic evaluation, and this aspect was, therefore, not further analyzed.
Indication
The occurrence of health economic parts in the protocols was highest among trials where the indication for the investigated drug was infection (25 percent). This group was followed by psychiatric disorders (24 percent), sexual transmitted diseases (23 percent), transplantation (20 percent), and dermatology (20 percent) (Table 2).
The prevalence of economic evaluations among protocols with substances classified according to the ATC system was highest in the groups L (antineoplastic and immunosuppressive drugs) and C (cardiovascular system), followed by B (blood and blood-forming organs) and J (general anti-infectives). However, many of the substances under investigation were not classified according to the ATC system or this information was missing.
Study Design
In general, very limited information on the study design of the economic evaluations was available. Most of the time, a strategy for analysis was missing or incomplete. Only 34 percent of the protocols with an economic evaluation had a comprehensive and specified strategy for analysis.
In 13 percent of the health economic evaluation protocols, it was stated that modeling was going to be used. A brief description of the modeling technique was then usually included.
Comparator
The information on comparators used for the economic evaluations was limited. The comparator used for the clinical evaluations was always clearly defined, but it was often not explicitly stated that this drug/therapy also served as a comparator in the economic evaluation. It is, however, likely that the clinical comparator also will be used as a comparator in the economic evaluation: 32 percent of the economic evaluations were to use an active treatment as a comparator, 21 percent used placebo, 8 percent used an active treatment as well as placebo, and 40 percent did not specify the comparator used in the economic evaluation.
Costs
Among the protocols with a health economic evaluation, 46 percent considered only direct medical costs and 5 percent considered only indirect costs, 23 percent considered both direct medical costs and indirect costs, and 11 percent of the protocols considered direct medical and direct nonmedical costs as well as indirect costs. There are, however, different ways of defining direct nonmedical costs, and they were, in a few cases, classified as indirect costs. There were 15 percent of the health economic protocols that did not specify the costs that were going to be considered but instead used unspecific expressions such as “all costs,” “cost,” or “information on resource utilization.”
In 55 percent of the economic evaluations with a strategy for analysis, information on what cost perspective that was going to be used was missing. Among protocols that did specify perspective, 19 percent adapted a societal perspective and 19 percent a healthcare perspective. Perspectives other than these represented 7 percent of these protocols. Information on cost adjustments for different geographical areas was present in 39 percent of the protocols that had a strategy plan for analysis of health economic data.
Effects
If there was no specified strategy for analysis, the health economic effect measure was usually not clearly specified. The outcomes of a clinical trial usually consist of several different clinical parameters and sometimes also a patient reported outcome or quality of life. It was generally not stated if one or several of these outcomes were to be considered in the economic evaluation. Because it was difficult to get clear information on the effect measure used, only the protocols with a strategy for analysis were evaluated on this matter. Among those with a specified study plan, 32 percent used quality-adjusted life-year (QALY) as the outcome measure, 23 percent used a surrogate measure, and another 23 percent used another outcome measure (e.g., life-years-saved, willingness-to-pay, or event avoided). The outcome measure was not clearly specified in 23 percent of the protocols with a strategy for analysis.
Quality of life (QoL) was measured in 20 percent of the protocols that did not contain a health economic part as compared with 68 percent among the protocols that did include health economics. Disease-specific quality of life measures were most common in both groups. In 30 percent of the studies with health economics, a generic QoL measure was used as compared with 18 percent among the studies with no health economic part. The EQ-5D measure was the most common generic measure of QoL among studies with a health economic evaluation.
DISCUSSION
The most important findings from this study are the observed trend toward an increased conduct of health economic studies, that the prevalence of health economic evaluations was highest among phase III trials, and that most of the studies were commercial, but also that there in general was very limited information on strategies for analysis. The presence of health economic evaluations in phase III trials was considerably higher than among phase IV studies. This finding is not surprising because there at the moment are less regulatory incentives for performing economic evaluations after marketing approval and reimbursement decisions have been made. Phase IV studies, however, give valuable information to decision makers, and the demand for this type of studies will likely increase in the future. There was also a much higher presence of economic evaluations among commercial trials than noncommercial trials, which probably may be explained by the economic incentives of the industry, and the requirements to perform economic evaluations for pricing and reimbursement decisions.
Most of the economic evaluations were to be conducted alongside a multinational trial or a national multiple-site trial, but few mentioned if (and how) the costs were going to be adjusted for different geographical areas. Because data collected in a multinational setting might be hard to transfer to different settings, this is an important aspect of these analyses (7). There is an active discussion and method development in this area at the moment and methodological standards for dealing with this type of data is emerging.
Most of the protocols did not specify a strategy for health economic analysis. In general, very limited information on the study design of the economic evaluations was available and complete study plans were often missing. Sometimes a referral to the case record form or a separate protocol, which was going to be submitted at a later stage, was made. The absence of requirements to include complete study plans for economic evaluations in the trial is a likely explanation to why this information is not presented in details. The lack of information makes it difficult to assess the quality of the studies.
Approximately one third of the studies specified that they were to use QALYs as an outcome measure. This strategy is in many cases desirable, because it enables comparisons of results between different disease areas (1;5). Almost a fourth of the protocols were going to use surrogate/intermediate measures. Because surrogate measures are expected to be translated into final outcome measures, the overall validity depends upon the way in which the relationship between intermediate and final outcomes has been quantified. The large use of surrogate end points is likely explained by the fact that they are also often used as end points in the clinical trial.
Approximately 20 percent of the protocols stated that placebo was going to be used as a comparator. Placebo is a common comparator in clinical trials aimed at proving a clinical effect of a drug, but is rarely a good comparator in an economic evaluation where the comparator preferably should be the most commonly used, or current “golden standard” treatment.
Very few protocols (13 percent) stated that they were going to use modeling. This modeling may, however, likely be used to a greater extent in the final analyses than what could be understood from the protocols.
The collection of resource use and data should reflect the adopted perspective of the analysis, as well as the appropriateness of measuring specific elements in a clinical trial environment. Nearly half of the protocols considered only direct medical expenditures. Indirect costs are very important for some diseases but are more complicated to measure in a clinical trial setting. It is, therefore, possible that these costs are included in the final economic evaluation, although the data are obtained from other sources. The Swedish Pharmaceuticals Benefits Board recommends the inclusion of indirect costs in economic evaluations (9), but agencies in many other countries do not.
Detailed descriptions of analysis methods and assessment of uncertainty were rarely presented in the protocol. On the whole, most of the examined economic evaluations had an incomplete presentation of the study design, probably due lack of incentives, because there currently are no requirements on cost-effectiveness studies for market authorization of new drugs. Although general requirements for randomized clinical trials in Europe since 2004 is strictly regulated by a European Union directive (2001/20/EC) and supported by European guidelines for various therapeutic areas, there are no specified demands on the health economic parts of the protocols. Subsequently, the economic parts of clinical trial protocols—when present—are given much less attention during assessments of clinical trial protocols. This finding means that there is no motivation for a sponsor to report all details of the economic evaluations they are planning to conduct. Also, the best method and data to use for an economic evaluation are not always apparent. Resource use, for example, from a clinical trial setting is often not representative of the real resource use in clinical practice. It is, therefore, common to include data from other sources, and an “incomplete” data collection from a trial does, therefore, not mean that the final analysis will be of low quality.
The potential risk of bias on economic evaluation is a widely discussed topic, and there is a concern that marketing objectives in industry-sponsored studies might introduce bias and give unreliable results. Some argue that it is possible to adjust the study design to fit the purpose of the trial, that is, to show that a drug is cost-effective. It would, therefore, be interesting to find out how many of the economic evaluations were actually performed and published and if they followed the given study plan. Stricter requirements for specifying study plans in advance (e.g., in the clinical trial protocol) could be one way to ensures that the study design will not be adjusted along the way. Assessment of the study protocols by experts could in this case ensure that standard methods are used, although these quality assessments must be made on a case to case basis because the appropriate parameters for each trial depends on what is being studied.
POLICY IMPLICATIONS
In conclusion, economic evaluation is a common part of clinical trials, but the description of these analyses in the trial protocols is usually limited. The low quality and infrequent use of health economic protocols in clinical trials found in our study most likely reflects the present situation also outside Sweden. Regulatory agencies, sponsors, and academic researchers are urged to jointly elaborate and implement relevant guidelines to provide quality-assured pharmacoeconomic data to all parties concerned.
CONTACT INFORMATION
Anne Lindfors, MSc (Anne.Lindfors@ki.se), Department of Public Health Sciences, Karolinska Institutet, 171 77 Stockholm, Sweden
Nils Feltelius, MD, PhD (nils.feltelius@mpa.se), Associate Professor, Department of Medicine, Karolinska Institutet, 171 77 Stockholm, Sweden; Senior Expert, Department of Clinical Trials, Medical Products Agency, Box 26, 751 03 Uppsala, Sweden
Jonas Lundkvist, PhD (jonas.l@healtheconomics.se), Researcher, Medical Management Centre, Karolinska Institutet, 171 77 Stockholm, Sweden
