The increasing pace of development of innovative healthcare technologies forces both regulators and providers to be responsible for the portfolio of technologies that has to be promoted and adopted for the delivery of health services. Many economists and policy analysts consider innovation in technology—along with weak cost-containment measures—to be a major driver in rising healthcare costs (15). The Health Technology Assessment (HTA) discipline has received increasing attention over the past few years as an instrument that can support decision making at each level in healthcare.
The literature pertaining to HTA has recently started to investigate the applicability of HTA guidelines at a hospital level. The problem of selecting new technology is not a novelty for hospitals—in fact, the budget process copes with this issue every year—but what is new is the increasing commitment to the improvement of the rationality and fairness of this priority setting exercise (Reference Nyland and Pettersen13). In fact, the resources devoted to adopting or maintaining technologies are more and more limited at a hospital level and the demand is increasing more and more. From this point of view, hospitals are required to improve both accountability and responsibility in the budget process (Reference Daniels and Sabin4), in particular concerning the adoption of new (expensive) technology.
The rising interest of hospitals in HTA can be explained as the commitment to renew the present practices of assessing and selecting new technology, taking into account some of the guidelines developed within the HTA discipline. A recent study has claimed that approximately 66 percent of Danish hospitals apply an HTA-based framework on a regular basis to support decision making (Reference Ehlers, Vestergaard and Kidholm7). A recent article, published by one of the most well-known Italian financial newspaper, has claimed that approximately 83 percent of Italian hospitals apply the HTA discipline for priority setting (Reference Cicchetti and Marchetti3). In particular, the 2006–2008 Italian National Healthcare Plan remarks that HTA is the natural basis for improving the rationality and accountability of technology adoption and management in the national healthcare system at every level.
This study aims at providing preliminary evidence about the integration of the budget process and HTA guidelines at a hospital level. The extent of integration is reviewed in terms of accountability of data collecting. The proposal forms used to give information to the budget process about the adoption of new technology have been reviewed to identify which dimensions are accountable at a hospital level.
The review of the proposal forms to suggest the adoption of a technology to the budget committee provides a useful picture of the extent to which the concepts of rationality, multidimensionality and evidence, which are typical of HTA, have penetrated the budget process. Although HTA at a hospital level and HTA for governmental resource allocation decisions have different purposes, they should share the same principles. In an ideal world, HTA at a hospital level should cover all the domains of HTA, but this is not realistic in the real world. The review of the proposal forms shed light on which data are required to inform decision making regarding the adoption of new technology at a hospital level.
Methods
The authors used a multiple case study methodology (Reference Yin17) for the empirical research. The in-depth analysis of a single case allows the peculiarities and the determinants to be understood at an early stage of knowledge, while a cross-case comparison is appropriate when attempting to externally validate the findings from a single case study. The research design is based on two sequential steps. First, a literature review about priority setting at a hospital level, aimed at supporting the interviews was carried out. Second, an analysis of the selected cases was conducted. In the following, both steps are described briefly.
Electronic Literature Search
The authors carried out an electronic literature search from January 1990 onward covering PubMed, Medline, and Cilea to collect the relevant contributions about priority setting at a hospital level. The references of the selected contributions were also reviewed. The priority setting exercise was limited to the choice of which portfolio of equipment or medical devices would be adopted by a hospital. The authors excluded the other healthcare technologies (such as medicine) because they were particularly interested in understanding the degree of integration between the budget process and the HTA guidelines. To be included, a contribution had to describe, in whole or part, a method or criteria to select an equipment or a medical device. The identified contributions were reviewed for relevancy by the three authors separately, on the basis of the title and abstract. If at least one reviewer identified a contribution as being potentially relevant, the full paper was obtained. The collected papers were then reviewed and selected if all the authors considered them to meet the selection criteria. A brief textual description was written for each priority setting method, in particular concerning the various analysis dimensions and the required proof of evidence.
Multiple Case Study Analysis
The empirical exercise involved four Italian hospitals (see Table 1 for details; the real names have been made anonymous for confidentiality reasons). These hospitals were selected because of the interest they have paid in HTA over the past few years and because of the possibility of having access to confidential information. Moreover, over the past 2 years, the demand for new technology in the hospitals in the sample has more than doubled compared with the availability of financial resources. The analysis was carried out as follows:
• At the beginning of each case, a relationship was established with the Health Director of the hospital. He was briefed about the research project and the authors asked to be introduced to two Clinical Department Heads, to the Head of Clinical Engineering and to the Head of the Financial Department;
• The authors personally interviewed the selected figures; they undertook a semistructured interview for each respondent (each interview on average lasted 1 hour) to gather the information required;
• The proposal forms used to collect the data required to inform the selection of new technology were collected and reviewed in comparison with the dimensions that emerged during the literature review;
• A preliminary within-case analysis was performed; the purpose was to consider each case study as a separate one to understand to what extent each hospital is accountable regarding the selection of new technology; the findings from each case were checked with the interviewees during two meetings to identify and correct potential misunderstandings.
• Finally, a cross-case analysis was undertaken to compare the patterns that emerged in each case study to obtain a wider understanding and explanation.
Table 1. Details of Hospitals Included in the Case Study Sample
FRAMEWORK
The research about priority setting at a hospital level was mainly focused on the healthcare services portfolio that health agencies or hospitals should deliver to maximize the health gains for a given population against scarce resources. Numerous studies have investigated hospital performances in terms of efficiency, assuming the hospitals as a black box. Other studies have carried out health technology assessment (HTA) exercises about specific healthcare technologies, while neglecting to investigate the relationship between the incremental cost-effectiveness ratio and the peculiarities of each hospital. Other studies have investigated how a specific set of variables may or may not influence the adoption of a specific healthcare technology by hospitals. Despite that, the number of studies that argue a reference framework for technology adoption at a hospital level is limited. In particular, the authors decided to select seven contributions according to the inclusion protocol described in the Methods section.
Uphoff and Krane (Reference Uphoff and Krane16) conducted a pioneering study in this field. They proposed a reference flowchart for capital planning and technology assessment at a hospital level. They also proposed a checklist of twelve questions that cover different analysis perspectives, such as: linkage to strategy, health and social impacts on patients, safety, risks, and costs. Ehlers et al. (Reference Ehlers, Vestergaard and Kidholm7) proposed the Danish mini-HTA as a tool to conciliate the HTA philosophy and the practice of technology assessment. The assessment exercise is based on twenty-six questions, many of which are similar to the ones proposed by Uphoff and Krane (Reference Uphoff and Krane16). The questions are grouped into four main assessment perspectives: technology, patient, organization and the financial aspects. Briones et al. (Reference Briones, Loscertales and Pérez Lozano2) described the results of the Spanish GANT project (Guide for Acquisition of New Technologies) which identifies five assessment perspectives: technology, health problem, evidence on efficacy, effectiveness and safety, organization, and economics. Azzone et al. (Reference Azzone, Lettieri and Masella1) applied capital budgeting techniques to give information on the adoption of an innovative Linear Accelerator for oncology treatment in a Italian hospital. The authors identify four main perspectives against which the impacts of the adoption of a healthcare technology should be reviewed: increase of revenues, containment of costs, improvement of flexibility and creation of strategic options. Sloane et al. (Reference Sloane, Liberatore and Nydick14) investigate the possibility of using the Analytical Hierarchical Process (AHP) to aid the selection of devices for cardiovascular diseases at a hospital level. After three interactions with the decision makers, they identified four main assessment perspectives: technological features, clinical factors, safety, and costs. Farrar et al. (Reference Farrar, Ryan and Ross9) applied discrete choice modeling to priority setting to select the optimal combination of clinical service developments within a given budget. The model is based on cost-utility comparisons. Preference are measured by asking the consultants to make choices between different scenarios involving different levels of a limited set of benefits. The authors identified five main benefits: (i) health gain for patients;(ii) contribution to education, training and research; (iii) contribution to professional development; (iv) strategy area (national versus local); (v) level of evidence of clinical evidence. A specific value for each level of benefit is proposed. The preferences give information on the estimation of the regression equation to give weights to the benefits. Finally, information on the costs is combined with the benefit scores to estimate a cost per unit of benefit ratio for each of the proposed developments. Lettieri and Masella (Reference Lettieri and Masella11) argued that hospital-based HTA should consider two dimensions: (i) potential value, and (ii) degree of applicability. The potential value of a healthcare technology is related to social gains, economic gains, and knowledge development. The degree of applicability is measured considering five areas: (i) economics, (ii) organization, (iii) technology, (iv) resources, and (v) demand.
The identified contributions propose different reference frameworks to give information on technology assessment at a hospital level. The criteria have been reviewed, compared and systematized in a reference framework based on five main perspectives (i.e., technology, patient, organization, economics, evidence). The proposal forms collected from the hospitals were reviewed considering the reference framework. The set of criteria for each perspective is shown in detail in Table 2.
Table 2. Explanation of the Criteria for the Five Perspectives (When Not Self-explanatory)
Results
The results are summarized in Table 3–7 and discussed in the following for each perspective.
Table 3. Criteria Used for the Assessment of the “Technology” Dimension
aFor example, the Food & Drug Administration, the European Commission for Medical Devices, scientific associations, lobbies of patients, etc.
bNHS, National Healthcare System.
Table 4. Criteria Used for the Assessment of the “Patient” Dimension
Table 5. Criteria Used for the Assessment of the “Organization” Dimension
aFTE: Full Time Equivalent.
bThis criterion aims at measuring the possible inertia to change
cFor example, the possibility of accommodating the technology in the existing setting
dFor example, changes in current management practices because of a technology overlapping.
Table 6. Criteria Used for the Assessment of the “Economics” Dimension
ae.g. costs of equipment, rebuilding, training, etc.
bNHS, National Healthcare System.
cFor example, purchase, leasing, service, etc.
Table 7. Dimensions of “Evidence” Required for the Proposal
The “Technology” Perspective
All the hospitals require a description of the technology (equipment or medical device) and the purposes of the adoption. Some of them (A,C) support the filling in of the proposal through a checklist, while others (B,D) have blank forms. While two hospitals (A,B) require the proposer to specify the priority for the clinical department or for the hospital, the other two hospitals (C,D) do not require this information. Hospitals C and D collect the level of priority during the first budget meeting when the budget committee clarifies the regional priorities and the consequent hospital priorities. The budget committees in hospitals C and D believe that this way of acting allows them to understand what the proponents think is useful, regardless the regional and hospital priorities. All the hospitals prefer technology to be proposed independently by the vendor. For this reason, a statement of vendor's reputation is not required. The information is collected before the technology is adopted to benchmark the potential vendors. Hospital D requires the statement of the expected evolution of the technology in the next years. None of the hospitals collects information about the existence of alternative technologies, despite the current practice to do so.
The “Patient” Perspective
The “patient” perspective covers two areas: health-related benefits (e.g., increase of health gains, decrease in mortality ratio, etc.) and satisfaction-related benefits (e.g., increase in patient satisfaction, decrease in length of stay, etc.). All the hospitals collect information about the expected clinical benefits for the patients. All the proposal forms have an open question, because of the variety of outputs or outcomes related to the adoption of a new technology. Regarding the health-related benefits, none of the hospitals explicitly requires the benefits to be stated in terms of quality of life or concerning the social and employment situation. A clear statement of the ethical and psychological implications is not required either. The same is true for satisfaction-related benefits. No data are compulsory about the expected increase or decrease of a patient's (or his/her family's) acceptance or satisfaction of the new technology or the health services that will be delivered through the adoption of the technology. Moreover, no data are required about the impact in terms of length of stay. These data would also be relevant for the “Economics” perspective because of the ability to improve efficiency. The absence of specifications regarding possible adverse events and the required strategies to contain them is more controversial. Only hospital D requires the benefits to be stated in terms of safety.
The “Organization” Perspective
Three hospitals require some statements in their proposal forms, about change management issues. Hospital A is the only one which completely neglects this perspective. However, the other hospitals are accountable for different criteria. Hospital C only tracks the potential impacts because of the overlapping of the new technology with the technologies already in place. Hospitals B and D are accountable for the expected changes in the present workflow and the implications in terms of training. It is interesting that hospital B is accountable for changes in roles and competence and also for implications in terms of training, whereas hospital D is only accountable for training needs, and does not need to track the expected changes in roles and competences. Moreover, hospital B is the only one that requires the transient period to reach the steady-state condition and the level of complexity expected to put the technology into routine to be stated. Finally, none of the four hospitals collects information about changes in cooperation between the clinic department proposer and the other departments or the changes in cooperation with other hospitals, the primary care and/or the long term care.
The “Economics” Perspective
The proposal forms have collected numerous data about the “Economics” perspective. All the hospitals require the start-up costs to be stated in terms of initial investment, space and people involved. Regarding the start-up costs for people involved, hospital D requires this need to be stated in terms of hours a week for the different types of staff (e.g., physicians, nurses, technicians, managers, etc.), while the others require the number of heads of each typology. Expected revenues and costs are analyzed, even although hospitals A and B require the clinical departments to specify this information, while hospitals C and D require the Financial Department to produce the expected profit and loss accounts. The revenues and costs analysis is limited to the hospital's boundaries. In fact, the statement of the expected costs for the patients or for the National Healthcare Systems is not required. None of the four hospitals considers the gains related to image or reputation. It is relevant to note that none of them requires a sensitivity analysis to understand the variance of results.
The “Level of Evidence”
The Evidence Based Medicine (EBM) approach is a pillar on which the HTA discipline is based. The clear understanding of the level of evidence associated to the sources of information and to each contribution from literature is an essential requirement for the decision-making process at each level in healthcare. The behavior of the four hospitals is different concerning the EBM approach. Two hospitals (A,B) do not require any proof of evidence about the data provided to promote the adoption of a healthcare technology. Hospital D requires at least five citations about the effectiveness of the technology under review, without specifying the level of evidence required (i.e., randomized clinical trial vs. the opinion of a panel of experts). Finally, hospital C requires all the necessary evidence about the cost-effectiveness of the technology under review to be provided and specifies the existence of guidelines to use the technology in the daily clinical practice.
DISCUSSION AND CONCLUSIONS
The review of the proposal forms for the adoption of new technology shows that hospitals have different levels of accountability concerning the data that HTA frameworks at a hospital level claim to be relevant. Data that are not collected in proposal forms can be collected during further meetings by the budget committee: in view of this, further research is required to understand the level of integration between HTA and budgeting. In particular, the authors believe that at least four questions should be investigated: (i) Should HTA be multidimensional at a hospital level? (ii) Should HTA be evidence-based at a hospital level? (iii) Should HTA be accountable at a hospital level? (iv) Should HTA be at a hospital level? First hand considerations about each question can be made from the previously discussed results.
The four hospitals collect data to focus their technology assessments on the aspects that really matter to them as organizations. In particular, it is possible to affirm that hospitals are interested in (i) whether the technology works; (ii) what it costs; (iii) what the revenues are; (iv) what the advantages are in implementing the technology. These aspects are broadly covered in the proposal forms in the perspectives about “technology,” “patient,” and “economics.” However, two considerations can be made. First, it is interesting to note that the proposal forms have an open question about the expected benefits for the patients, without requiring specific criteria such as benefits on quality of life, length of stay, potential adverse events or patient satisfaction. A first explanation is that there is a variety of benefits that can be achieved by adopting new technology. From this point of view, an open question is the best solution. A second explanation could be that hospitals are more interested in outputs than in other outcomes. This is not surprising. In fact, the Italian reimbursement system is based on DRGs and takes into account the outputs. Hospitals are encouraged to focus on clinical benefits (and also on the length of stay as a driver to decrease costs) and pay less attention to aspects such as patient satisfaction. A second consideration deals with the heterogeneity of the level of accountability about change management. For example, only hospital B requires data about the transient period to reach a routine or data about groups affected by change. The explanation is related to the lack of skills on project management in the budget committee. In fact, physicians assess clinical issues, managers assess financial issues, technicians assess technological issues. Generally, none of them approaches the adoption of technology as a project that is characterized by organizational and change management issues. Despite these findings, the authors believe that HTA, at a hospital level, should be multidimensional and cover the four perspectives. Only in this way can physicians, managers and technicians select the portfolio of new technologies that is the best balance between different competing targets such as cost containment and quality improvement (Reference Uphoff and Krane16).
HTA should be evidence-based at each level (Reference Drummond, O'Brien and Stoddart6). The review of the proposal forms shows that two hospitals are not accountable concerning the level of evidence. This finding is coherent with many contributions on physicians' behavior toward the adoption of new technology. A large number of physicians adopt new equipment or new medical devices when they are confident that the level of effectiveness of their practice will increase (Reference Escarce8). Their confidence is rooted roots in either the guidelines provided by their scientific association or in the opinion of the first users, especially when they are the key opinion leaders. Escarce (Reference Escarce8) claimed that the acquisition of information about the consequences of adoption derived from the personal experience of early adopters may influence other physicians to try the technology in their own practice.
The accountability of budget processes is necessary to guarantee the fairness of the decision-making process and prevent decisions from being inappropriately influenced by a single (influential) individual (Reference Lumsdon12). Any proposal should be assessed against similar criteria to guarantee the fairness of the selection process. This means that each proposer is accountable and responsible for the data he/she provides. In fact, the data provided are also the basis to deliver feedback—when the technology is adopted—to the proposer, showing the variations against targets and discussing the causes of these variations. Feedback is necessary, on one hand, to create a reputation for the proposer in terms of fairness and competence and, on the other, to support the organization learning about the ability of a hospital to adopt and manage new technology. When the variability of results is not adequately managed and budgets for investments are not affected by reputation measures, physicians believe they are entitled to ask more and more without being responsible for the results (Reference Nyland and Pettersen13). For example, none of the four hospitals in the sample performs a systematic analysis of the results after the adoption of a technology.
The last consideration concerns the role that HTA should play at a hospital level. As mentioned in the introduction, in an ideal world, HTA at a hospital level, should cover all the domains of HTA, but this is not realistic in the real world. The authors firmly believe that the principles of rationality, multidimensionality and evidence that are typical of HTA at a governmental level should also apply at a hospital level and be considered in the budget process. However, the real world requires that the potential barriers are understood. During the interviews, two barriers were identified. First, there is a shortage of resources. HTA is time consuming and a full HTA exercise could be incoherent with the need of assessing numerous proposals within a budgeting window (Reference Drummond and Weatherly5). In this view, a full HTA exercise is only possible for those technologies which are high-cost or high-risk. Second, it is sometimes difficult to work in a multidisciplinary committee, while decision makers at a hospital level prefer to assess a technology according to procedures based on sequential functional areas (e.g., clinic, technical, economic). In fact, each group of decision makers has developed its own language and this prevents communication and collaboration (Reference Haines and Jones10).
This study has shed first light on the integration, at a hospital level, between budgeting and HTA. Based on this review of the proposal forms, Further research is recommended to have a full picture. Further research questions could concern: How do the proponents of a new technology fill in the proposal forms? How do they judge the perspectives and criteria? How does the budget committee act to increase the accountability and fairness of decision making?
CONTACT INFORMATION
Emanuele Lettieri, PhD (emanuele.lettieri@polimi.it), Researcher, Cristina Masella, Management Engineer (cristina.masella@polimi.it), Full Professor, Department of Management, Economics and Industrial Engineering, Politecnico di Milano, 32, P.zza Leonardo da Vinci, Milano, Italy, 20122
Umberto Nocco, Clinical Engineer, (umberto.nocco@ospedale.varese.it), Director, Clinical Engineering Department, AO Ospedale di Circolo e Fondazione Macchi, 57, Viale Borri, Varese, Italy, 21100
