During the past years, the number of economic evaluations (EEs) has dramatically increased worldwide. Many of these studies were carried out as pharmacological and clinical trial byproducts sponsored by the pharmaceutical or medical device industry, and others were customized to the request of different health technology evaluation agencies. In this sense, the recent trend of including findings on the cost-effectiveness of clinical interventions in the reviews led by national regulatory agencies in Australia, Canada, and the United Kingdom, among others, does not only consider the EE as a guideline for decision making on licensing and reimbursement of drugs and medical technologies for public coverage of services, but also addresses the question of how these EEs must be included in the decision-making process (Reference Harris, Buxton, O'Brien, Rutten and Drummond7). Furthermore, EEs are increasingly requested as a criterion to include a particular technology into public coverage as a “fourth hurdle” after efficacy, safety, and quality have been taken into account (Reference Taylor, Drummond, Salkeld and Sullivan12). Surprisingly, despite that growth in this field has expanded considerably, there is still scarce evidence on the degree of influence that these studies have on decision makers with respect to health resource allocation.
A postal questionnaire survey implemented in the United Kingdom in 1997 (Reference Drummond, Cooke and Walley3) concluded that, despite the well-established evaluating culture of the National Health Service and the potential impact that EEs might have to modify the resource allocation criteria, its budgetary rigidity caused their results to lose most of their potential. Other studies carried out in the United States to explore how managed care organizations formulated their coverage policies showed that, despite the general knowledge on EE and their greater use over the past years, they were still considered second-line outcomes. In addition, American decision makers preferred to use information most specifically directed to contingent decisions they had to make on a daily basis (Reference Sloan, Whetten-Goldstein and Wilson11).
The Euromet Project, a survey conducted to assess the impact of EEs on health decision making, pointed out that the most important drivers to include technologies for reimbursement were the evidence of efficacy and effectiveness rather than economic data (Reference Hoffmann, Graf and on8). Unfortunately, except for some recent studies focusing on priority-setting methods (Reference Evans, Tan-Torres, Adam and Lim5;Reference Ezzati, Lopez, Rodgers, Vander and Murray6), scarce evidence is available on the use of EEs to assist decision making in developing countries, including Latin America.
Argentina is a middle-income country with major healthcare problems related to both equity and efficiency. Its plural and fragmented healthcare system consists of three large sectors: public, private, and social security. Social Security covers more than 50 percent of the population, distributed among close to 300 entities of varying scope and size. The Superintendent of Health Services of the Ministry of Health (SSS) is responsible for oversight of the social insurance plan's compliance with the Compulsory Medical Plan (PMO), guaranteeing system quality and coverage and cost recovery of public hospitals. The public hospitals provide coverage to the population on demand and, in fact, act as reinsurance for the health insurance plans, because they maintain a flow of free care for the insured population. The private insurance sector covers 7 percent of the population.
Since the beginning of the recession in Argentina in 1999, but mainly since the economic plummeting of late 2001, Argentina has been suffering an unparalleled economic and financial crisis. This new scenario made possible that some regulatory agencies such as the SSS started to promote the formal evaluation of technologies to be included in its PMO. Nevertheless, except for very few academic centers, there are not yet public agencies explicitly dealing with health technology assessment (HTA) in our country.
Our study was intended to determine (i) whether EEs, which are key components of HTAs, were considered and used by decision makers in Argentina and (ii) the impact that Argentina's economic crisis had on health decision-makers' sensitivity toward the use of EEs. In addition, this study also reports the criteria decision makers used for resource allocation and the barriers and facilitators to promote and extend the utilization of EEs to guide policies about coverage of health services.
METHODS
A qualitative research was carried out based on semistructured interviews and focus groups with key informants from the health sector according to Euromet Project approach (Reference Costa, Rovira and Rovira1). We included the perspectives of decision makers, both in the public and private sector, belonging to different levels of decision as follows: (i) the macro level, corresponding to decision makers responsible for the financing and service coverage (health secretariats, and social insurance and private insurance managers); (ii) the meso or intermediate level, corresponding to the purchase and delivery of services (hospital managers); (iii) and the micro level, corresponding to clinical decision making (clinicians).
Data Collection
Focus group and interviews took place between May and August 2003. The research team had discretion to select the participants, balancing key informants from different levels and settings across the health sector. All participants gave their written informed consent. A questionnaire guide was designed to frame the interviews and focus groups and was adapted to each respondent or group of respondents according to their hierarchy and level of decision. Interviews and focus groups started by asking what decisions they had to make on a daily basis with regard to resource allocation, not explicitly mentioning the aim of the study. Taking as examples the situations exposed by the respondents; specific questions were asked about which criteria were used for making decisions. Only after the economy issue came up did the coordinators lead the conversation deep into this subject, asking the respondents specifically about their knowledge and attitudes about the utilization of EEs to allocate resources in health care. The individual interviews and the focus groups were conducted by members of the study team (M.B. and V.D.). All of the interviews and focus groups were audiotaped for subsequent transcription.
Data Analysis
Data codes established by using topics from the interview guide were supplemented by additional codes identified by using a grounded theory-based approach to capture emergent themes. A “constant comparison” strategy ensured internal consistency in the coding process. Text segments were coded independently, based on the existing code definitions, identifying new codes when comments did not fit the existing ones. After the codification of each focus group or interview, the coders met to discuss and resolve problems and discrepancies related to the use of established coding categories and to determine the relevance of emergent categories and themes. Atlas-TI Version 4.0, a software program, was used to support the analysis.
RESULTS
Population
A total of three focus groups were conducted: one with insurance auditors representing the macro-level decision makers (four participants); and two with clinicians, representing the private and public micro-level decision makers (four and six participants each). A total of six interviews were carried out: (i) with a public health officer (macro-level decision maker); (ii) with social insurance and private insurance managers (private macro-level decision makers); and (iii) with hospitals managers (private and public meso-level decision makers). A total of twenty individuals participated in the study. Results are shown below answering the specific objectives of this study.
Criteria Used in the Decision-Making Process with Regard to Resource Allocation
There was evidence of a “cross transfer” of responsibilities from one to the other as to the role each decision level plays in resource allocation decisions. The clinicians showed some difficulty in recognizing their specific involvement in decisions on resource allocation. They suggested that these issues were the responsibility of health secretariats, hospitals, or health insurance managers. Some private insurers mentioned that “clinicians are those who always make the decisions and once they indicate a practice, it cannot be denied.” In addition, hospital managers suggested that they do only administer resources and that, in general, the chief of each service is the one who makes decisions or asks for resources and, finally, clinicians are those who allocate them. Public health decision makers mentioned that, in many cases, the resource allocation was driven by budgets from Ministries or Secretaries of Health.
The specific criteria guiding the different decision makers are included in Table 1. This table includes verbatim fragments mentioned by the participants.
Table 1. Resource Allocation Decision-Making Criteria by Target Group and Illustrative Verbatims
Allocation based on best scientific evidence was mentioned as an excluding criterion for clinicians and as very important for the rest of the informants. These decisions were considered to be easier when there were evidence-based clinical guidelines available and relevant information ready to be retrieved at the point of service though the use of guidelines was very poor.
A clinical/epidemiological criterion was mentioned both by physicians who have to make decisions regarding patient care (individual impact and benefit) and by public officials who have to define budgetary priorities (social impact and benefit). The conflict between “individual versus social perspective” in resource allocation was present in several interviews and focus groups.
We included as formal economic criteria those that explicitly mention costs and consequences of alternative interventions. Paradoxically, this criterion was more explicitly mentioned by managers than by financers/purchasers, while clinicians disregarded in their individual clinical practice. Although social insurance or private insurance considers costs in relation to “benefits” as a concept, this strategy becomes more relevant when authorizing a service not included in the PMO. However, respondents explained that this criterion was secondary to the scientific evidence of efficacy. Another criterion mentioned by a social security manager was related to availability of resources, that is, what kind of diagnostic or therapeutic resource was available at the point of care, emphasizing the relevance of local resource availability over any other criteria. Financial incentives received by clinicians, mainly from the pharmaceutical and medical technology industry, to provide specific interventions were also considered an important driver for decision making.
Knowledge and Use of EEs: Impact of the Crisis
EE to assist managers in resource allocation decisions was not a well-known tool, therefore, was not frequently used. However, after the crisis, decision makers became more sensitive as to the “need for a more efficient resource allocation in order to optimize their utilization,” but the explicit need for EE/HTAs was not introduced, probably because they did not know their concept or because they were not familiar with their methods. Additionally, especially among clinicians, the influence that economic criteria might have on clinical decisions was rejected.
Argentina's new economic and financial scenario was mentioned in most of the interviews and focus groups. Respondents spontaneously introduced the crisis as a resource allocation decision-making triggering factor. EE/HTAs were shown to have limited influence, despite few experiences with its use at higher decision-making levels (intermediate and macro). More importantly, most cases showed that what respondents interpret about EE were partial evaluations of costs based on utilization measures.
HTA reports containing systematic reviews of efficacy, costs, and potential budgetary impact of clinical and healthcare interventions were occasionally used and there were some examples of commissioning these assessments to expert groups. However, they questioned the fact that most of the reports did not include local data that might compromise their validity.
Potential Use of EEs and Approach
In general, a decision-maker's proactive attitude regarding a more important role of EE in the future was shown at different levels. Despite that the influence of costs was widely accepted, the most important criterion to adopting or discontinuing a technology was its evidence of efficacy and safety.
Some focus groups also tangentially mentioned some underlying ethical aspects of EE, such as the fact that, even if there were good evidence on the efficacy of a certain treatment, and the cost were higher, there would be justifications not to cover it, especially in developing countries. Informants raised the need that state regulatory agencies and public and private purchasers increasingly use EEs to help make decisions on resource allocation in the future.
Barriers and Facilitators for the Use of EE Study Results
Barriers for the utilization of EEs have been classified into seven categories (Table 2). Some of them were expressly mentioned by the focus group and interview participants; others were detected by the group of investigators when data were analyzed.
Table 2. Barriers for the Use of Economic Evaluation for the Decision-Making Process Regarding Resource Allocation and Illustrative Verbatims
Lack of knowledge about EE/HTAs and/or the difficulty in interpreting the scientific literature have been considered as the most important obstacles against making use of the EE results. We found that many respondents said they knew what an EE was, but they did not know its taxonomy with regard to different types of analyses. Some participants admitted that they found it difficult or impossible to appraise scientific studies. They also expressed that a greater dissemination of EEs basic concepts among decision makers through seminars, courses, workshops, and different specific educational activities, as well as physicians becoming more sensitive to these concepts during their undergraduate and graduate formal training, would make consideration and use of these tools more feasible.
Reluctance toward the use of the economic or “economicist” factor was frequently related to lack of knowledge or to some beliefs that, behind economic evaluations, there were interests trying to limit the quality of patient care. Additionally, for clinicians, the main concern was related to efficacy/effectiveness making unlikely the discontinuation of a treatment just because it is not cost-effective. There were certain prejudices related to the usual source of the economic data in our country (pharmaceutical industry, private insurance, and so on) that conditioned its credibility and acceptance.
A relevant obstacle found was the existing difficulty of access in general to EE/HTA studies. There was no agreement among respondents as to how easy access to scientific information is. Two hospital directors said that physicians at their hospitals have easy access to information through the Internet; the third one said that there were huge difficulties in accessibility.
Another barrier mentioned by some respondents was the lack of transferability of study results. More specifically, it was said that lack of knowledge together with lack of external validity, due to the fact that studies are usually carried out in developed countries, with costs, effects, and values very difficult to extrapolate to our local framework, usually make these studies poorly applicable and out of context for decision making in developing countries. Efforts to adapt studies carried out in developed countries to our context, or even better, to generate local EEs considering relevant topics, costs, and utilities obtained at a local level, could facilitate their introduction and application in Argentina.
The difficulty to move resources from one item to another in a budget was mentioned as a potential barrier, as this rigidity prevented recognition of the impact of greater or lower efficiency on expense allocation. In this sense, a flexible budget was considered a facilitator that would increase the use of EE in the future.
Increasingly, decision making among clinicians is related to the fear of lawsuits or malpractice lawsuits. This issue was raised by the auditor's group, mentioning it as a barrier against the application of EE as it is presumed that judges do not manage this type of scientific information and consequently act against the institution or the physician who did not implement or authorize a particular practice even though insurance eventually covers the interventions considered unnecessary according to evidence.
The existence of individual, corporate, and social pressures to influence in the decision-making process was pointed out to patients, physicians, patient or professional groups, scientific societies, social mass media, and so on. Respondents were highly consistent when saying that, although it is not a regular phenomenon, many times the patient's demand is a critical factor at the point of care. This criterion was mentioned among clinicians who frequently feel part of the pressure from patients asking for more diagnostic testing/treatment than necessary. On the other hand, public and private purchasers both commented that they usually have to meet the patients' request, no matter if the service is considered inappropriate or unnecessary by auditors, or often, they also have to accept the inclusion of a new practice/service when an institution-recognized physician just requests it, regardless of the real need of including that practice or service. The interest private insurance managers have in preserving its institutional image may play a role at the time of making a decision, especially when it is decided to give a benefit that, according to evidence, is not deemed necessary or adequate.
Last but not least, the health system institutional fragmentation comes up as one of the biggest obstacles against adopting EEs. Reformulation of the incentives to promote policies toward integration of services and alignment of decision-makers' interests at different levels would make easier the introduction of EEs. Interestingly, there was an ample consensus among decision makers that the creation of a national regulatory agency to help define priorities with regard to health technology coverage policies would be of great value as a reference framework for all health sectors.
DISCUSSION
Argentina's recession, which started in 1998 and exploded in late 2001 with the foreign debt default, the end of peso–dollar convertibility, and huge devaluation, apart from increasing the population's impoverishment, caused unprecedented financial crisis in the healthcare systems. It could be thought that, once the first months of the crisis were overcome and with a persisting and tight scenario, decision makers both in the public and in the private sectors would try to migrate from an implicit to an explicit rationing scenario by defining, for example, a list of priority services to be covered. In this new scenario, EEs could have their place as tools for assisting decision makers in the priority setting process. Williams and Cookson (Reference Williams and Cookson13) described a four-stage process in the historical development of resource allocation decision making: stage 1 where decisions are made based on intuitions, stage 2 based on precedents from prior decisions, stage 3 involving codification, and stage 4 where weights and quantitative trade-offs are incorporated into a formula. It seems that our respondents were shifting from stage 1 to stage 2 according to this framework. Despite an ample consensus about the need of reallocating resources more efficiently, EE/HTAs were not clearly identified as tools to improve this process.
Coyle proposed a four-step model for the EE result impact analysis: (i) dissemination of findings, (ii) acknowledgment of the importance by decision makers, (iii) result understanding by decision makers, and (iv) use of findings as tools in decision making. The author recommends that decision makers should get involved in conducting more studies and that health economists should convince them of the importance of giving emphasis to the efficiency factor and the relevance on the use of information. In addition, studies should be “temporarily” linked to decision making and their results should be widely dissiminated (Reference Coyle2).
In our study, lack of knowledge about potential EE applications posed tangible barriers. Even though this finding was also mentioned in other published surveys (Reference Drummond, Cooke and Walley3;Reference Hoffmann, Graf and on8), many of the barriers found were related to methodological or technical aspects. In this sense, the current status of ignorance of health decision makers in Argentina makes irrelevant any other type of technical consideration. Therefore, it is necessary to initially promote formal educational instances to make decision makers become sensitive toward their use, as suggested by many respondents.
According to a recent study, there is very little experience of the use of EE as a tool to inform the decision-making process in the healthcare systems of most Latin American countries, including Argentina. Nevertheless, a considerable awareness of the need to understand, conduct, and apply EEs to improve allocation of resources was perceived (Reference Iglesias, Drummond, Rovira and NEVALAT10). As a matter of fact, among the few experiences of local use of economic evidence, the HTA reports were mentioned. These reports are being increasingly commissioned to groups of experts from the academic sector by the State, Social Security, and Private Sector agencies, to evaluate coverage policies of medical technologies.
In our study, an important barrier mentioned by most participants as that impairing the external validity of EE was the difficulty to generalize the results of studies usually conducted in developed countries to settings in developing countries. Usually, the clinical and cost-effectiveness of a health technology is heavily influenced by local factors, such as demographic characteristics of the population, local epidemiology, lifestyle, availability of services, clinical practice patterns, incentive structure, without mentioning differences in outcome measures, such as many clinical outcomes or the valuation of utilities and social value judgments (Reference Drummond, Sculpher and Torrance4).
Another reason why it is difficult to apply EE results may be related to the narrow focus that EEs study as they usually compare particular interventions that are not necessarily related with the decisions made on daily basis. Most EE/HTAs report on aspects related to technical efficiency. This concept refers to optimal use of resources when providing or producing a given health intervention. Allocative efficiency, on the other hand, is typically used in health economics to refer to distribution of resources among different programs or interventions, thus obtaining the most socially desirable resource, with the resource available (Reference Hutubessy, Chisholm and Torres9). These studies have been promoted by the World Bank or the World Health Organization to be applied to developing countries.
Study Limitations
Two of our focus groups had only four participants. Nevertheless, focus group dynamics could be established through participant interaction. In fact, the interviews as well as focus groups reached adequate saturation criteria.
CONCLUSIONS
Even when economic considerations to prioritize resource allocation are increasingly being accepted in Argentina, and although this phenomenon has become faster after the economic crisis, the use and application of EE/HTAs are still very limited. Lack of conceptual and technical knowledge, difficulties in accessing to the studies, lack of credibility in data sources and in their external validity, together with the sector institutional fragmentation and certain skepticism and rejection from clinicians are barriers against disseminating their use. Moreover, individual or social pressures plus fears of litigations are factors that influence in a society that ignores the value of explicit policies. A decision-maker's specific training and involvement in the EE result on the one hand and conducting local studies on the other hand were identified as decisive factors to facilitate the growing use of EEs in the resource allocation decision-making process.
POLICY IMPLICATIONS
The crisis our country has suffered over the past years, which, fortunately, is being overcome, should serve as a driver to a more explicit resource allocation priority setting process to improve both technical and allocative efficiency. Better implementation of the use of EEs in general and of HTAs specifically in Argentina is dependent upon health decision makers being sensitive to the value of EEs, along with the conduct of local studies with local implications. Finally, the creation of a national agency to guide health sector coverage policies with respect to medical and health technologies will become a critical tool to improve efficiency, quality, and equity.
CONTACT INFORMATION
Adolfo Rubinstein, MD, MSc, PhD (arubinstein@iecs.org.ar), President, Institute for Clinical Effectiveness and Health Policy (IECS), Viamonte 2146, 3er piso, C1056ABH Ciudad de Buenos Aires, Argentina; Chief, Division of Family and Preventive Medicine, Hospital Italiano, Tte Gral. Juan D Perón 4272, 1199 Ciudad Buenos Aires, Argentina
María Belizán, MSc (mbelizan@iecs.org.ar), Executive coordinator and Researcher, Institute for Clinical Effectiveness and Health Policy (IECS), Viamonte 2146, 3er piso, C1056ABH Ciudad de Buenos Aires, Argentina
Vilda Discacciati, MD (vilda.discacciati@hospitalitaliano.org.ar), Researcher, Division of Family and Preventive Medicine, Hospital Italiano, Tte Gral. Juan D Perón 4272, 1199 Ciudad Buenos Aires, Argentina
