Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-06T11:48:48.376Z Has data issue: false hasContentIssue false
  • English
  • Français

ECONOMIC EVALUATIONS IN ITALY: A REVIEW OF THE LITERATURE

Published online by Cambridge University Press:  21 April 2004

Livio Garattini ,
Paola De Compadri ,
Rosamaria Clemente  and
Dante Cornago
Livio Garattini
Affiliation:
Centre for Health Economics
Paola De Compadri
Affiliation:
Centre for Health Economics
Rosamaria Clemente
Affiliation:
Centre for Health Economics
Dante Cornago
Affiliation:
Centre for Health Economics
Rights & Permissions [Opens in a new window]

Abstract

Objectives: To review the economic evaluations (EEs) done in Italy by Italian authors, following a common scheme to allow some comparisons of the studies selected and with the international reviews.

Methods: We selected all the original studies published by Italian authors (in Italian or English) in national and international journals. The period considered was January 1994 to December 2001. Both full and partial economic evaluations were included. Three international databases were interrogated: MEDLINE, Embase, and HealthStar; further articles were added from the internal database of our center (CESAV), which also classifies Italian local publications and journals specialized in health economics.

Results: A total of ninety-nine studies were reviewed. More than half of the fifty-seven full EEs focused on drugs as type of intervention (n=38), followed by diagnostic screening (n=7). The NHS viewpoint was the most used (n=55 studies), followed by that of society (n=27) and hospitals (n=12). Sixty-eight studies only analyzed direct costs and twenty-nine included both direct and indirect costs. Twenty-five of the thirty-eight pharmacoeconomic full EEs were sponsored by companies. In sixteen of the twenty-five sponsored studies, the sponsor's products were the dominant alternative.

Conclusions: The review showed that, in Italy, like elsewhere, there is a gap between theory and practice in EEs, and sponsors can considerably affect the results of EEs.

Keywords

Economic evaluationItalyReviewMethodology
Type
RESEARCH NOTES
Information
International Journal of Technology Assessment in Health Care , Volume 19 , Issue 4 , December 2003 , pp. 685 - 691
Copyright
© 2004 Cambridge University Press

One of the most important changes in health care systems in the past twenty years involves the spread of large databases, covering many aspects of health. This finding stems from the need for large sets of information, collected with rigorous methods, to permit research in outcome measures of health care (3).

Economic evaluation (EE) in health care is a young discipline that has grown significantly all over the world in recent years (17). EE comprises the various techniques proposed by health economists to rationalize the allocation and the employment of scarce resources. In some countries (e.g., Australia, The Netherlands, Canada), public authorities formally require EEs as support for pricing and reimbursement procedures for new drugs and this requirement has boosted their use (4).

To our knowledge, two reviews of EEs based on large international databases have been done so far. The first was based on the United Kingdom National Health Service (NHS) Economic Evaluation Database (EED) at the University of York. Established in 1995 and supported by the British NHS, its aim is to identify published EEs relevant to the NHS, making them available to all potential users (16). The second study was based on the Health Economic Evaluation Database (HEED) of the Office of Health Economics, a research center supported by the Association of the British pharmaceutical industry. This database is a subscription service that provides standardized abstracts of published EEs (18).

These international reviews stimulated us to conduct a similar exercise on the EEs published by Italian authors. Our center (CESAV) is the Italian collaborator in a project aimed at creating a European Network of Health Economic Evaluation Databases. The EURONHEED project was started by the CES (Collège des Economistes de la Santé, the French health economists' association), which recently developed a database similar to HEED, called CODECS (24). CODECS collects published works in the field of health EE and offers abstracts of published EEs. This strategy should make local EEs published in national journals available in the European context.

This review analyzes the “state of the art” of EEs in Italy, following a common scheme to allow some comparisons of the studies selected and, as far as possible, with the international reviews mentioned above. We also comment on the methodological choices and financial support of EEs.

ITALIAN BACKGROUND

Italy has an NHS that provides universal coverage and comprehensive health care free at the point of delivery (9). According to the Italian Committee for Economic Planning (Comitato Interministeriale Pianificazione Economica, CIPE), the body in charge of pricing, the cost-effectiveness ratio is one of the criteria for price negotiations on innovative drugs (5;8). The level of reimbursement is related to the clinical importance assigned to the drug by the National Drug Committee (Commissione Unica del Farmaco, CUF). The CUF lists pharmacoeconomics as one of the subjects to be considered in submitting dossiers for innovative drugs. Therefore, EEs are recognized as an important support in the regulation of the pharmaceutical sector (8). However, to date, Italian public health authorities have not issued official guidelines, meaning they have not yet made any specific effort to standardize methods.

Against this background, the number of EEs has risen considerably in the past decade in Italy. Italian research groups are increasingly involved in the EEs of health care programs, and almost all the major pharmaceutical companies have introduced health economics as a management task (2). Three pharmacoeconomic societies have been founded, and various methodological recommendations have been published (1;10;22).

METHODS

EEs can be classified in two broad categories according to the technique adopted. Cost-minimization analyses (CMA), cost-effectiveness analyses (CEA), cost-utility analyses (CUA), and cost-benefit analyses (CBA) are considered full EEs, because they compare different diagnostic or therapeutic alternatives. Alternatively, EEs such as cost of illnesses (CoI) and cost analyses (CA), which quantify the cost of a pathology, a clinical event, or therapy, are considered partial EEs (6;13). Both categories were considered in this review. The CEA studies were considered separately from CUA and CMA studies, because most Italian researchers still consider they belong to different categories.

Three international databases were interrogated: MEDLINE, Embase, and HealthStar. The period considered was January 1994 to December 2001. The key words used for searching were “cost-benefit analysis,” “cost-effectiveness analysis,” “cost-utility analysis,” “cost of illness,” “cost analysis,” combined with the key word “Italy.” This search led to 587 studies. We selected all the original studies published in Italian and international journals (in Italian or English) co-authored by at least one Italian author and concerning the Italian setting. Many articles were found in more than one database, which led to double or triple counting (280). In case of duplicate publication, we chose the most recent version or the version published in the journal with the highest reputation (or impact factor if available). We excluded 229 studies for different reasons: they were editorials or reviews, the setting was not Italy, none of the authors was Italian. A further twenty-nine studies were added from the internal database of CESAV, which also classifies Italian local publications and journals specialized in health economics; this is our original contribution to the EURONHEED project.

Eventually, we selected 107 articles published by at least one Italian author and conducted in the Italian setting for reviewing. We investigated the following variables: type of pathology, type of study, type of intervention, source of efficacy, decision tree, modeling, viewpoint of the analysis, sensitivity analysis, category of costs, source of health care resources, and source of hospital unit costs. Four of these items (type of intervention, source of efficacy, decision tree, modeling) were investigated only for the full EEs, because they do not pertain to the methodology of the other EEs. Every article was independently reviewed by two researchers, to validate the classification and the final results. A statistical analysis (chi square, Pearson's test, p<.05) was performed to assess whether the presence of at least one economist as co-author affected the methodological choices of the studies.

RESULTS

Among the 107 articles, 8 were excluded because of a lack of clear and justified results; therefore, 99 articles were reviewed. Figure 1 shows the rising number of EEs during the period 1994-2001 in Italy. Table 1 summarizes the main characteristics of the EEs selected in this review.

Economic evaluations in Italy.

The articles covered a wide range of pathologic conditions, cardiovascular diseases being the most studied. Forty studies were CEAs, more than one-fourth were CoIs (n=28), fourteen were CAs, eleven were broadly CMAs, CUAs, and CBAs, whereas the remaining six adopted a combination of different forms.

Of the fifty-seven full EEs, thirty-eight focused on drugs, seven on diagnostic screening, five on vaccines, four on preventive programs, two on specific surgical interventions, and the last one on a clinical treatment (non-invasive mechanical ventilation). More than two-thirds of the CEAs considered treatments, whereas most CBAs were applied to preventive interventions; twenty-nine full EEs based their demonstration of efficacy on clinical trials, seventeen on clinical databases, and five on meta-analyses. The decision tree was used in only nine studies, modeling in fourteen (six Markov, two Gompertz, one DEAL, five combinations of different models).

The viewpoint was clearly stated in two-thirds of all the ninety-nine EEs selected, (n=66) and was implicit in another thirty-three. The NHS viewpoint was the most used (n=55 studies), followed by that of society (n=27) and hospitals (n=12). The viewpoint was a combination in four of the five remaining studies, a private company in the last one – this was a CBA that assessed the indirect costs of employees to see whether an influenza vaccine would improve their productivity.

Sensitivity analysis was done in less than half the studies (n=45); it was done in thirty-seven of the fifty-seven full EEs (65%); sixty-eight studies only analyzed direct costs, twenty-nine included both direct and indirect costs, one also included intangible costs, and one focused exclusively on indirect costs (see above).

In 37% of the studies, experimental data were used to estimate the consumption of health care resources; 28% collected them in clinical trials, and 18% from clinical records. Of the seventy-six EEs that assessed hospital costs, forty-six used DRG tariffs as unit costs, twenty-six based unit costs on microcosting studies, and the remaining four studies extracted unit costs from the literature.

To assess whether the presence of at least one economist as co-author significantly affected the methodological choices, studies were classified in two groups: (i) “with economists” (at least one of the authors graduated in economics), and (ii) “without economists.” Table 2 shows the results of the statistical analysis conducted on a selected list of items. A significant difference was found only for two items: the source of resource consumption and the type of EE conducted. Accordingly, economists seem to prefer studies based on experimental data and more frequently do partial EEs (i.e., CoIs and CAs).

DISCUSSION

We reviewed the EEs done in Italy by Italian authors, following a common scheme to allow some comparisons and, thus, assess the “state of the art” of these studies. Like in other reviews, we have probably missed some significant studies. However, this probability was minimized by using systematic bibliographic search techniques. The extensive criteria used to avoid missing studies was the main reason for the large number of articles rejected after applying the inclusion criteria.

To assess the results of the review according to the attitude of the Italian pharmacoeconomic community, we used the most recent informal guidelines published in Italy (1), which stem from a workgroup of experts and, therefore, should be the result of a broad consensus. The results show a good agreement with regard to the type of costs to be investigated: nearly a third of the studies included indirect costs in addition to direct ones, whereas intangible costs (which the guidelines strongly recommend excluding) are almost ignored. However, despite the recommendations to use both the NHS and societal perspectives, always to do sensitivity analysis, and to rule out partial EEs, only four studies used the two viewpoints together, more than one-third of the full EEs did not assess the robustness of results with sensitivity analysis, and many partial EEs have been undertaken in Italy – according to the results of the statistical analysis economists even seem to prefer them.

Moving to information sources, clinical trials – as proposed by the guidelines – were used for efficacy in more than half of the full EEs, whereas only one-third estimated hospital unit costs through microcosting studies – the source recommended by the Italian expert panel because it should reflect real costs better than tariffs and prices.

Comparing the present review to international ones (16;18), many findings are not directly comparable because these studies do not analyze the EE methods in detail. However, the comparable results are broadly similar. The technique most often used for EEs is CEA (85% in Nixon et al.; 70% in our review), the source of efficacy was most frequently the clinical trial (66% in Nixon et al.; 51% in our review), and pharmacoeconomic studies accounted for more than one-third of all EEs analyzed (36% in Pritchard; 38% in ours). These similarities are further confirmed by comparing our results to those of a recent Spanish methodological review (11). In Spain too, most EEs only considered direct costs (82% as in our study) and the technique most used was CEA (62%).

The methodological problems of EEs in Italy are probably similar to those in other countries (7). This problems may be explained by the difficulty of applying standard methods to all cases and gives further backing to the idea that experts should not set down rigid recommendations, if they cannot be followed by the health economists themselves. Many alternatives are available, but each has its own strengths and weaknesses. We share the opinion of scientists who maintain that standardization or regulation of methods may be counterproductive if guidelines attempt to set up too rigid an approach to evaluation (21). The adoption of a method as a standard should not mean that other methods can be ignored, nor should it constrict the scope of the analysis (20). On the other hand, methodological choices considerably affect the final results of EEs and their heterogeneity makes any kind of comparison difficult (12;19).

The most widely debated concern about potential biases relates to the influence of sponsors on the EE results (14;15;19). More than half of the full EEs focused on pharmacological therapy. Accordingly, we investigated the source of funding for pharmacoeconomic studies only. We found that twenty-five of the thirty-eight pharmacoeconomic full EEs were sponsored by companies (66%). The sponsor was clearly acknowledged in fourteen studies, two studies were co-authored by company employees, and we checked the sponsorship of the remaining nine studies directly with the authors. In sixteen of the twenty-five sponsored studies (64%) the sponsor's products were the dominant alternative (i.e., more effective and less costly); in five studies the sponsor's drug was recommended in specific conditions (e.g., selective use according to age or severity of patients), whereas in the four remaining studies, the authors outlined that the sponsor's drug was more effective but more expensive. It is worth mentioning that one sponsored study even demonstrated the dominance of a statin recently withdrawn for safety reasons all over the world.

These results in Italy confirm the legitimacy of the debate in the literature on possible bias in the results of published EEs sponsored by the pharmaceutical industry (15). To cope with it, Drummond suggested clarifying the contractual relationships between sponsors and researchers (7). More emphatically, a group of scientists claimed that researchers should sign contracts guaranteeing their right to control the design of the study, their access to relevant data sets, and the freedom to publish negative results (23). However, this seems difficult to achieve in practice and even more difficult to detect. Because the risk of bias cannot be easily ruled out, encouraging scientific journals to push for transparency in methodology seems more realistic (14).

In conclusion, the review showed that, in Italy, like elsewhere, there is a gap between theory and practice in EEs and sponsors can considerably affect the results of EEs. An awareness of these limits should lead the Italian health economics community to a more lively and constructive dialogue.

The authors acknowledge the useful critical comments from Mike Drummond and Gérard De Pouvourville. The opinion expressed and any errors made are, however, the responsibility of the authors. Special thanks go to Judy Baggott and Clara Chiantaretto for editing the manuscript.

References

Attanasio E, Bruzzi P, Capri S, et al. 1999; Raccomandazioni per la conduzione degli studi di farmacoeconomia: La guida GISF. Mecosan. 29: 65 72.Google Scholar
Colombo GL, Serra G, Terranova L. 2001; Indagine sulla diffusione della farmaeconomia nell'industria farmaceutica che opera in Italia. Pharmacoeconomics. 3: 91 103.Google Scholar
De Pouvourville G. 2000; Changes in health economic data bases. Hepac. 1: 93.Google Scholar
Drummond MF, Jönsson B, Rutten FFH. 1997; The role of economic evaluation in the pricing and reimbursement of medicines. Health Policy. 40: 199 215.Google Scholar
Drummond MF, Dubois D, Garattini L, et al. 1999; Current trends in the use of pharmacoeconomics and outcomes research in Europe. Value Health. 2: 323 332.Google Scholar
Drummond MF. 1992; Cost of Illness studies. A major headache? Pharmacoeconomics. 2: 1 4.Google Scholar
Drummond MF. 1998; A reappraisal of economic evaluations of pharmaceuticals: Science or Marketing? Pharmacoeconomics. 14: 1 9.Google Scholar
Fattore G, Jommi C. 1998; The new pharmaceutical policy in Italy. Health Policy. 46: 21 41.Google Scholar
Garattini L, Tediosi F, Ghislandi S, Orzella L, Rossi C. 2000; How do Italian pharmacoeconomists evaluate indirect costs. Value Health. 3: 270 276.Google Scholar
Garattini L, Grilli R, Scopelliti D, et al. 1995; A proposal for Italian guidelines in pharmacoeconomics. Pharmacoeconomics. 7: 1 6.Google Scholar
Garcìa-Altés A. 2001; Twenty years of health care economic analysis in Spain: Are we doing well? Health Econ. 10: 715 729.Google Scholar
Hjelmgren J, Berggren F, Andersson F. 2001; Health economic guidelines: Similarities, differences and some implications. Value Health. 4: 225 250.Google Scholar
Koopmanschap MA. 1998; Cost of illness studies: Useful for health policy? Pharmacoeconomics. 14: 143 148.Google Scholar
Krimsky S. 1999; Conflict of interest and cost-effectiveness analysis. JAMA. 282: 1474 1475.Google Scholar
Morgan S, Barer M, Evans R. 2000; Health economists meet the fourth tempter: Drug dependency and scientific discourse. Health Econ. 9: 659 667.Google Scholar
Nixon J, Stoykova B, Glanville J, et al. 2000; The U.K. NHS economic evaluation database. Int J Technol Assess in Health Care. 16: 731 742.Google Scholar
OHE-IFPMA. 1999. Health Economic Evaluation Database (HEED). London: Office for Health Economics;
Pritchard C. 2001; What do we know about the economic evaluation literature? A review of HEED. J Drug Assess. 4: 9 20.Google Scholar
Rennie D, Luft H. 2000; Pharmacoeconomic analysis: Making them transparent, making them credible. JAMA. 283: 2158 2160.Google Scholar
Rittenhouse BE. 1996; Is there a need for standardization of methods in economic evaluations of medicines? Med Care. 34: DS13 DS22.Google Scholar
Sloan F. 1995. Valuing health care. Cambridge: Cambridge University Press;
Società Italiana di FarmacoEconomia (SIFE). 1999; Linee guida per la conduzione di studi sull'efficacia e sul costo dei trattamenti farmacologici. Giornale Farmacoeconomia. 3: 147 153.
Task Force on Principles for Economic Analysis of Health Care Technology. 1995; Economic analysis of health care technology: A report on principles. Ann Intern Med. 122: 61 70.
Ulmann P. 2000; The CODECS Project. Hepac. 1: 129 130.Google Scholar
Figure 0

Economic evaluations in Italy.

Figure 1

Main Characteristics of the 99 Economic Evaluations (EEs) Reviewed

Figure 2

Statistical Analysisa