Development and implementation of health technologies has become a challenge to numerous health systems around the world. With growing healthcare costs, priority setting is becoming an important part of the functioning and effectiveness of healthcare systems (Reference Sabik and Lie1).
Even though Lithuania started taking its first steps in health technology assessment in approximately 1993, it still remains among the countries that continue implementing a health technology assessment (HTA) system. Due to political, economic, and other factors, institutionalization of the HTA system remains inconsistent (Reference Jankauskiene2). Article 54 of the Law on the Health System of the Republic of Lithuania of 1994 prohibits the use of health technologies that have not been assessed or approved, except for in cases specified by the Law (3). The health technology assessment procedure is to be established and approved by the Ministry of Health of the Republic of Lithuania; however, the procedure is yet pending (Reference Jankauskiene2). Some health technologies are assessed by private or governmental institutions; however, there is no unified system of assessment that would contribute to the design and implementation of evidence-based health care (Reference Jankauskiene2).
Decision makers that take part in processes of health technology implementation and funding not only face the priority setting issue but also suffer from insufficiency of impartial and evidence-based information. As a result, fragmented decisions are made in line with the interests of policy makers or stakeholders rather than those of the public. Such issues could be addressed by priority setting, which is one of the first steps in the development of health technology assessment. Additionally, this effort would contribute to a more efficient health system. The objective of this research is to design a model for HTA priority setting, which would address national needs and assess its feasibility for the health system by using the opinion of national experts.
METHODS
The following methods were used to design and test the model for HTA priority setting: literature search and review and, Delphi technique. The Delphi technique is well suited as a method for consensus building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects (Reference Hsu and Sandford4). The Delphi technique was used to achieve a consensus of experts on HTA priorities in Lithuania as well as to test the feasibility of the developed model.
Literature Search
The literature was searched for the priority setting systems used internationally. The literature was searched in databases (Pubmed, Medline), Web sites of Europe-based health technology assessment agencies and international health technology assessment organizations (INAHTA, HTAi, EUnetHTA), international libraries and e-journal databases. The following search criteria were used: (i) only priority setting systems used in Europe were considered; (ii) the period was 1996–2011 (1996 was chosen as the starting date due to recommendations on priority setting by the Europe-wide EUR-ASSESS project) (Reference Henshall, Oortwijn and Stevens5); and (iii) the search was undertaken in English using the keyword combinations: priority setting, prioritization, health technology assessment. The literature search was carried out in 2011.
The literature review showed that variability exists in the priority setting systems across HTA agencies; however, many agencies usually use a criteria-based system for prioritizing health technologies. Based on the findings of the literature review, a model for HTA for priority setting was designed.
The Design of the Model for HTA Priority Setting
According to the experience of other countries and taking into consideration the very early stage of HTA development in Lithuania the design of the model for HTA priority setting was created consisting of four stages: (i) selection of a panel of experts; (ii) indication and selection of health policy topics; (iii) identification of health technologies; and (iv) priority setting. The model for HTA priority setting is provided in Figure 1.
Figure 1. A model for HTA priority setting.
Selection of the Expert Panel
In model designing and testing, selection of experts is an essential stage that seeks a reduction in possible clashes of expert interests and inconsistency in the object of expertise. Three key criteria were used for selection of experts: knowledge in HTA fundamentals; representation of the stakeholder groups such as health policy makers and implementers, healthcare sector administrators and the academic community; and involvement in different sectors of the health system. Experts were selected from among well-known national specialists, bearing in mind that no less than half of them had to work in different sectors undertaking several professional roles. A panel of eleven well-known health system experts was selected, comprised of health policy makers (3), health professionals working in health care (Reference Hsu and Sandford4), and members of the academic community (university teaching and research staff) (Reference Hsu and Sandford4).
The general public was not involved at this stage in the process of HTA priority setting, because awareness of the HTA among the general public is very low in the country. Consequently, it was hypothesized that the academic community would be able to represent public interest as well.
Delphi Study
The Delphi study comprised three rounds: (i) ranking of priority health policy topics; (ii) identifying the relevant health problems and health technologies to solve them; and (iii) assessing health technologies on the basis of criteria. The Delphi study was conducted by e-mail in 2011, ensuring the anonymity of respondents. The response rate for each stage of the research amounted to 100 percent.
Delphi-1 Round
The researchers defined health policy topics considering the current issues and priorities pertaining to national health policy. During the Delphi-1 round, the experts were asked to rank health policy topics defined by the researches as well as named by the experts themselves, considering health concerns of each field as well as their scope and evaluating their relevance in the long-term perspective. The researchers identified the key health policy topics considering the compatibility of expert opinions, which was expressed as the coefficient of concordance (Reference Schmidt6).
Delphi-2 Round
During the Delphi-2 round, experts were asked to name no more than three health problems for selected health topic and suggest health technologies to solve them. The researchers selected health problems and health technologies using the following selection criteria: (i) consensus of expert opinion; (ii) experience of experts in health technology assessment. To ensure reliability of health technology selection, the test–retest method was used. Two weeks after the first selection, the researchers independently repeated the health technology selection on the basis of assessment criteria.
Delphi-3 Round
During the Delphi-3 round, experts were asked to use the five-point Likert scale (where numerical values from 1 to 5 respectively range from “totally in disagreement” to “totally in agreement”) to assess health technologies on the basis of the following criteria: budget impact; health benefit; alternatives; expected level of interest from policy makers; timeliness; evidence; and ethical, legal, and social implications. These criteria were adapted to the national context by researchers based on a recent systematic review that identified twelve priority setting systems with different priority setting criteria used among HTA agencies (Reference Noorani, Husereau, Boudreau and Skidmore7).
Moreover, experts were requested to assign weights to each priority setting criterion from 1 (the lowest weight value) to 5 (the highest weight value), attaching the lowest weight value to the least important criteria.
Final priority score
The final priority score of each health technology was defined using the methodology used for priority setting in the field of health technologies, using the formula (8): priority score = W1lnS1+W2lnS2+ . . . +W7lnS7; where W is the criterion weight; S is the criterion score; and ln is the natural logarithm.
RESULTS
The model for HTA priority setting was designed on the basis of the experience of other countries (Reference Noorani, Husereau, Boudreau and Skidmore7–Reference Stevens and Milne11), tailoring the findings to the national context. Considering the importance of stakeholder participation in decision making, the model accentuated the expert selection stage. During this stage, it is important to facilitate involvement and participation of all stakeholder groups in the process of HTA priority setting to ensure representation and externalization of group interests (health policy makers and implementers, healthcare sector administrators, and members of the academic community).
Selection of Health Topics
Sixteen health policy topics were suggested during the Delphi-1 round. However, considering the consensus of expert opinions measured by concordance coefficient, the following health policy topics were selected: healthy lifestyle, cardiovascular diseases, oncologic diseases, mental health, accidents and traumas, diabetes, environmental health, and communicable diseases. Following the scale of the coefficient of concordance (Reference Schmidt6), opinions of the expert panel had a moderate degree of agreement, while agreement in each stakeholder group ranged from high to very high.
Identification and Prioritization of Health Technologies
On the basis of the Delphi-2 round results, the researchers selected fifteen relevant health problems and health technologies to solve them from those offered by experts. The list was created from twenty-two health technologies, which could be grouped as follows: pharmaceuticals (Reference Sabik and Lie1), vaccines (Reference Sabik and Lie1), medical equipment (Reference Sabik and Lie1), diagnostic methods (Reference Schmidt6), public health interventions (8), organizational and managerial form systems (Reference Sabik and Lie1), and other health technologies (Reference Hsu and Sandford4).
On the basis of expert opinion during the Delphi-3 round, national HTA priorities were set. Table 1 provides the final ranking for each health technology, tabulating the estimated final priority scores. Priority was given to public health interventions and diagnostic methods (five first positions) by experts.
Table 1. Expert Opinion on Health Technologies for Their Assessment in Order of Importance
Priority Setting Criteria and Their Weights
The study also aimed to assess the criteria that not only could be used in HTA priority setting, but also their relative importance and to test the criteria in the developed model. Experts were asked to assign a weight to each criterion during the third round or the Delphi-3 round. Table 2 lists the criteria and their weights for HTA priority setting. Seven criteria were included in HTA priority setting and assessed by experts. From among the criteria, the greatest weight was attributed to the criteria “health benefit” (mean = 4.75; SD = 0.43), while the least was attributed to the criteria “ethical, legal and social implications” (mean = 2.75; SD = 1.18). According to experts, the latter criterion was the least important in terms of importance for HTA priority setting. The criteria “alternatives,” “expected level of interest from policy makers,” and “evidence” received the least estimate values.
Table 2. Expert Opinion on Importance of Criteria for Priority Setting: The Descriptive Analysis
DISCUSSION
Based on the experiences of other countries, the theoretical model was designed for the context of Lithuania as a country that has little experience in health technology assessment as well as limited participation of stakeholders in policy making. The model's adaptability to the national health system was confirmed by expert consensus using the Delphi technique.
The priority setting process required naming health technologies that would contribute to resolution of significant current and future health concerns. The experts were inclined to prioritize health technologies for prevention. This reflects specific features and the level of Lithuanian health policy, which still pays insufficient attention to evidence-based solutions for disease prevention strategies. Use of such technologies significantly contributes to delivery of public health improvement goals, which correspond to national policy targets. However, in an international context, public health and health promotion interventions are less frequently assessed by HTA agencies (Reference Banta, Hatziandreu and Dauben12;Reference Holland13). With growing necessity to ensure efficient use of health technologies in the healthcare sector, timely assessment could become especially relevant to decision makers. However, effective measures should be taken to increase the accessibility and usability of health technology assessments among healthcare stakeholders. Most countries still find it challenging to integrate health technology assessments into decision-making processes (Reference Barbieri and Drummond14). Numerous reasons are to blame for the still limited impact of health technology assessments on decision making (Reference Oliver, Mossiales and Robinson15). The strength of the developed model for HTA priority setting lies in the involvement of policy makers and other stakeholders who take part in processes of health technology implementation and funding. Furthermore, the increased interaction between researchers and policy makers in this model has been shown to enhance the use of research results in practice (Reference Innvaer and Vist16).
In Lithuania, the first study of its kind not only highlighted the particulars of HTA priority setting but also revealed the advantages and limitations of the methodology that was used. The aforementioned research has primarily focused on expert selection. It should be noted that subjective expert opinions are impossible to avoid, thus this factor had an impact on the results of the research. However, the investigation on the degree of agreement between expert opinions allows for the assertion that the method used was reliable. Participation of appropriately selected well-known experts representing key stakeholders promoted the diverse opinions required to ensure a transparent and unbiased HTA priority setting process. The research revealed high degrees of agreement among expert opinions, which was reflected by a respective coefficient of concordance, which demonstrates the reliability and content validity of the methodology used. In the field of HTA priority setting, public interest is becoming more and more significant (Reference Gagnon, Desmartis and Lepage-Savary17). The following reasons determined the noninvolvement of the stakeholder to represent the public interest: the public is insufficiently informed about the implemented HTA system in the healthcare sector; moreover, the country still lacks a theoretical model that would explore possible options for participation of the public in decision making regarding health technology assessment; furthermore, this model should be tested. In the future, participation of the general public is crucial.
The tested model for HTA priority setting can be used and adjusted depending on the HTA development in the country. If required, in the future the list of criteria could be supplemented with additional clearly defined criteria. Additionally, the reliability of the process for attribution of criteria weights should be separately tested to ensure efficiency. Final decisions in defining priority health technologies should be taken, considering the results depend on the calculation methodology used. Therefore, it is advisable to design and test various calculation methodologies and assess the possibilities afforded by their usage. These issues reveal some of the prospects for important future research.
This model is primarily intended for the national healthcare sector; however, it can also be used to evaluate peculiarities pertaining to HTA priority setting in the international context, especially in countries such as Lithuania that only started taking steps toward HTA implementation. Moreover, it can also be used to share information and experience in designing and improving priority-setting models.
CONCLUSIONS
The model for HTA priority setting is an instrument comprising four clearly defined stages, using an expert panel consensus process. This model is exceptional for inclusion of stakeholders into the process of HTA priority setting; additionally, it can be adjusted and used for the entire health system, that is, not only reflect personal health priorities but also public health priorities. Although certain methodological limitations are particular to the model and more in-depth or additional evaluations are required, it could be adapted for the national health sector as well as internationally.
POLICY IMPLICATIONS
The designed model could be adapted to other countries implementing HTA systems and experiencing a limited impact of health technology assessment on decision making.
Lithuania has made the required political decisions and drafted necessary documents regulating the implementation of the HTA system (3;18), thus the research is a relevant and timely contribution to further development of the system. To ensure targeted implementation or development of health policy in this area, a joint political agreement and will is required as well as constructive action and use of evidence in policy-making processes.
CONTACT INFORMATION
Danguole Jankauskiene, MD, PhD, (djank@mruni.eu), Professor at the Institute of Political Sciences, Vice-dean of the Faculty of Politics and Management, Mykolas Romeris University, Vilnius, Lithuania
Gintare Petronyte, PhD (ginpetr@mruni.eu), Lecturer, Institute of Political Sciences, Mykolas Romeris University, Vilnius, Lithuania
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.
