Healthcare decision makers are facing the challenge of rising healthcare costs in opposition to limited financial resources. One major source of expenditure is the growing number of health technologies, including drugs, diagnostic tests, medical devices, and procedural interventions (Reference Haas, Hall, Viney and Gallego1;Reference Joshi, Stahnish and Noseworth2).
To inform reimbursement decisions, the systematic assessment of health technologies regarding effectiveness and safety is well established in several countries. In addition, cost-effectiveness and the clinical value of technologies are often evaluated before their introduction into clinical practice (3). Health technology assessment (HTA) is therefore still focused primarily on new technologies entering the market (Reference Haas, Hall, Viney and Gallego1;Reference Mc Kean, Noseworthy, Elshaug and Clement4). However, many health technologies currently in use have never been assessed systematically and/or have not been re-evaluated since their entry into the healthcare system. Thus, the number of technologies that are ineffective or have become obsolete is unknown (see Table 1). Unfortunately, technologies that do not provide “value for money” (Reference Haas, Hall, Viney and Gallego1) are not automatically replaced by more effective, safe and efficient alternatives, but rather exist in parallel. Active identification and systematic (re)assessment of potentially ineffective technologies throughout their entire life cycle is thus gaining importance to provide the best possible care for patients and to facilitate the optimal allocation of resources. The goal is not to eliminate technologies and withdraw resources on a grand scale but to spend money on the most effective, safe and cost-effective technologies, to save or to re-allocate freed resources. However, the active step toward disinvestment by mean of implementation of a standardized reassessment process remains a sensitive issue for decision makers (Reference Mc Kean, Noseworthy, Elshaug and Clement4) and stakeholders and is often taken reluctantly.
Table 1. Definitions
1. Ruano Ravina A, Velasco Gonzalez M, Varela Lema L, Cerda Mota T, Ibargoyen Roteta N, Gutierrez Ibarluzea I. Identification, prioritisation and assessment of obsolete health technologies. A methodological guideline. Santiago de ComPostela: Galician Agency for Health Technology Assessment (AVALIA-T). 2009.
2. CHERE. Reducing the use of ineffective health care interventions. Centre for Health Economics Research and Evaluation, 2010. Working Paper 2010/5.
3. Elshaug AG. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197:556–560.
4. Joshi NP, Stahnish FW, Noseworth TW. Reassessment of health technologies: obsolescence and waste. Ottawa: Canadian Agency for Drugs and Technologies in Health (CADTH); 2009.
5. Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23.
6. Pearson S, Littlejohns P. Reallocating resources: How should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12:160–165.
7. Leggett L, Noseworthy TW, Zarrabi M, et al. Health technology reassessment of non-drug technologies: Current practices. Int J Technol Assess Health Care. 2012;28:220–227.
8. Mc Kean G, Noseworthy T, Elshaug A, Clement F. Health technology reassessment: The art of the possible. Int J Technol Assess Health Care. 2013;29:418–423.
The aim of this study is to provide an overview of existing programs for the identification, prioritization, and assessment of ineffective/obsolete health technologies, focusing on their differences and similarities regarding their processes. In addition, the lessons learned from these programs regarding barriers and challenges associated with their implementation are discussed to elicit relevant factors that should be considered before establishing a disinvestment program. Finally, general recommendations to facilitate further developments in the field are derived from good practice examples and analysis of the programs (see Table 2).
Table 2. Key Messages on Lessons Learned from Current Programs
METHODS
Based on a review article by the Centre for Health Economics Research and Evaluation from 2010 (5) and other overview articles (Reference Joshi, Stahnish and Noseworth2;Reference Leggett, Noseworthy and Zarrabi6;Reference Gerdvilaite and Nachtnebel7), a literature review was conducted in five databases (The Cochrane Library, CRD Database, EMBASE, Ovid MEDLINE, and Web of Science) in May 2013 to identify further programs for the identification of ineffective health technologies. The main search terms included disinvestment, ineffective or obsolete interventions and technologies, reallocation, and reassessment. Only documents published in English or German that contained information either on specific programs used to identify ineffective technologies or on general issues concerning methods for disinvestment/health technology reassessment and/or resource reallocation, were included. No limitation regarding the design of articles was applied. In addition, relevant journals and HTA agency Web sites were hand-searched to ensure a comprehensive overview. References from articles were also screened. To identify gray literature, an unsystematic Web search was performed using Google. To capture experiences concerning implementation facilitators and barriers associated, the role of different target groups, dissemination methods, and the uptake and impact of recommendations, international experts were consulted by means of a half-standardized questionnaire conducted by means of e-mail and/or telephone interviews. Experts had either been involved in the development of a certain program or had been scientifically engaged in a program's implementation.
Programs were excluded if they (i) lacked a detailed description of the objectives, methods, applied criteria and outputs; (ii) did not focus on the identification of potentially ineffective, unsafe, or inefficient health technologies for optimizing resource use in health care; (iii) were still in the development phase and had not yet been implemented in practice; and/or (iv) were not designed as a permanent program.
To identify relevant differences and similarities among the programs, a standard set of criteria where applied including objective(s), target group(s), identification and prioritization methods and criteria, stakeholder involvement, outputs, dissemination strategies, and implementation methods. The information was summarized narratively.
RESULTS
Of 593 references retrieved from the systematic search and hand-search overall, 120 were included. Seven programs for the identification of ineffective health technologies at national, regional and local level met the inclusion criteria (see Table 3: I–VII); five identified from the basic review article and two with the literature search. The information was complemented with questionnaires collected from five international experts (response rate of 62.5 percent). Of the seven identified programs, five are related to HTA (I–V) while two were developed and are used irrespective of HTA activities. Three of seven programs had been implemented in Europe (I–III), one in the United States (VI), two in Australia (IV, V), and one was established internationally (VII).
Table 3. Overview of Identified Programs: Differences and Similarities
GuNFT, Guideline for Not Funding existing health Technologies in health care systems; NICE, National Institute for Health and Care Excellence; SBU, Swedish Council on Technology Assessment in Healthcare; PBAC, Pharmaceutical Benefits Advisory Committee; MBS, Medical Benefits Schedule; PBMA, Program Budgeting and Marginal Analysis; Y, Yes; N, No; MoH, Ministry of Health; n.s., not specified; HTA, Health Technology Assessment
Goals, Initiators and Target Groups
Although government-initiated (top–down) programs, the majority (I–V), can be distinguished from those developed by individual institutions/organizations (bottom–up; VI, VII), the nature of the initiator does not impact on the goals of the programs because all the programs describe quality of care and patient safety as top priorities. The majority (I–V, VII) also mention cost reductions and reallocation of resources as major goals. The aims are often intertwined and cannot be strictly delineated. However, goals and initiators have an impact on the choice of target groups. Top–down programs aimed at cost reduction and/or improvement of structures and processes (I–V) consistently address decision makers to ensure the direct implementation of recommendations. Patients and consumers, however, are an important target group for three programs (I, II, VI), especially for the physician-initiated US-American Choosing Wisely® initiative aimed at patient empowerment. Health professionals (mainly physicians) are addressed by both top–down and bottom–up programs.
Most programs (I, II, VI, VII) do not focus on specific types of health technologies; in contrast, one program is limited to pharmaceuticals (IV), one explicitly excludes them (V), and one program currently excludes diagnostics (III).
Identification of Ineffective Technologies
For all programs, the first step is the identification of potentially ineffective health technologies. Even though several information sources, that is, literature-based (e.g., systematic reviews, HTA reports, guidelines, and routine data) and expert-related (e.g., communication with health professionals and patients), are used by most programs, methods for identification vary (see Table 3). Identification criteria, in contrast, overlap between the programs with effectiveness (I–VI), efficiency/cost/cost-effectiveness (I–VI), available alternatives (I, II, IV, V), and benefit (I, II, IV, VI) being the most frequently used ones, reflecting the aims of the programs. Stakeholders (physicians, health professionals, researchers, and patients) are already involved at this early stage either as members of an advisory panel (II, IV, V, VII), a working group (VI) or as individual consultants (I, III). Involvement of physicians in the identification process is considered particularly crucial as they are included by the majority of the programs.
Prioritization
In the second step, the majority of the programs prioritize the identified technologies. Only one program uses a standardized tool, the PriTec Prioritization Tool (I), whereas the others use a variety of methods (see Table 3). The groups mainly involved in the prioritization process are researchers and/or physicians; other stakeholders are rarely included. Despite different methods, prioritization criteria are identical to a large extent, with costs/efficiency (I, II, V–VII) most frequently mentioned.
Outputs and Dissemination
Outputs derived from assessing prioritized technologies are also quite similar (see Table 3): either HTA reports (I–III, V) or concise lists summarizing the recommendations (IV, VI, VII) are the most common ones. Dissemination strategies are either active or passive. In the latter case, recommendations are published on the internet (II–VI) or by means of print media (VI) with no “active” notification of target groups. Although this process may facilitate access by a broad audience, it also relies on interested and technically skilled target groups. An active dissemination strategy includes communicating face-to-face with target group(s) and using consumer organizations (VI) to facilitate access to patients and consumers. The National Institute for Health and Care Excellence (NICE) has developed a dissemination strategy including a broad range of both active and passive tools: monthly “Recommendation reminders” (active), “Commissioning guides,” “Cochrane Quality and Productivity topics,” (Reference Garner and Littlejohns8) and a database (passive).
Implementation of Recommendations
Despite the fact that the implementation of recommendations is the most crucial step, only two programs (I, II) describe an explicit strategy for implementation: The “Guideline for Not Funding existing health Technologies in healthcare systems” (GuNFT) emphasizes the importance of informing stakeholders and creating a positive image of disinvestment. At NICE, the implementation of recommendations is supported by the “NICE implementation program,” which targets mainly politicians and health professionals to facilitate the application of guidance in practice. For the other programs, no description of a certain implementation process was found in the literature.
The Choosing Wisely® initiative relies on physicians to implement recommendations in daily practice and encourages patients and consumers to have comprehensive discussions on treatment options. The Pharmaceutical Benefits Advisory Committee's (PBAC) recommendations are transferred into the Pharmaceutical Benefits Scheme by delisting pharmaceuticals and vaccines that are proven to be ineffective/ obsolete. The same procedure, related to the health technologies listed in the Medical Benefits Schedule (MBS), is described in the “Framework for reviewing existing MBS items.” In the Program Budgeting and Marginal Analysis (PBMA) program, the recommendations of the advisory panel are usually directed at specific organizations (e.g., an individual hospital). These recommendations are agreed upon and implemented directly by the organization's decision makers.
Thus, across programs it becomes apparent that consequences of disinvestment recommendations may take two forms: binding decisions that are followed directly by action, or nonbinding information for target groups. In any case, the implementation of such recommendations may result in: (i) a change in application area/scope of use, (ii) withdrawal of resources (disinvestment), (iii) complete removal from practice, or (iv) no change to the current practice.
Facilitating Factors and Barriers to Implementation
The overall barriers and challenges of disinvestment activities have been described in detail elsewhere (Reference Leggett, Noseworthy and Zarrabi6;Reference Elshaug, Hiller, Tunis and Moss9;Reference Henshall, Schuller and Mardhani-Bayne10). In this study, we explicitly focus on barriers, facilitators and key success factors for the implementation of the analyzed programs. Interviews with international experts were the main source of information, complemented with information from the literature search, for four programs (I, II, VI, VII). For the other three programs, no information could be obtained.
The identified facilitators include the following: (i) The broad involvement of stakeholders and, foremost, of physicians throughout the process to enhance acceptance of decisions. (ii) A structured and evidence-based process that includes transparent methods for the identification, prioritization and assessment of ineffective health technologies. (iii) A dissemination strategy that is tailored to the target group(s). (iv) Consideration of local contexts when defining identification and prioritization criteria and formulating recommendations. (v) Encouragement of political discussion and raising awareness before and during implementation.
Based on the international experiences, the following barriers must be overcome: (i) Additional human and financial resources are needed for the sustainable implementation of a program, especially if it cannot be embedded in previously established programs. (ii) A good evidence base is needed for both the identification of technologies and the formulation of evidence-based recommendations. The first especially holds true for programs using literature-based information sources for identification. (iii) The lack of a well-planned implementation strategy that involves all potential stakeholders and is aligned with the initial goal(s) of the program. If recommendations are not translated directly into binding guidelines and coverage changes, there is need for a broad dissemination strategy that targets physicians in particular. In the absence of such a strategy, a lack of acceptance may hinder implementation considerably. (iv) Finally, a lack of support from decision makers and an absence of strong leadership can hamper practice changes. In particular, politicians could facilitate implementation by raising awareness of the importance of optimal resource allocation in health care as well as by promoting a positive image of the reallocation process with stakeholders.
DISCUSSION
Despite differences in the methods, outputs, and consequences of disinvestment recommendations, all programs have the same goals: improvement of health care and reduction of the waste of resources. They also basically include the same steps, use similar information sources and, more or less, involve the same stakeholder groups at different stages. Even more importantly, all of the programs draw from a pool of core criteria to identify and prioritize ineffective technologies, and physicians are unanimously recognized as key stakeholders throughout the process.
The initiators and driving forces behind different disinvestment/reassessment programs vary. Top–down programs initiated by governmental authorities (or, in the PBMA program, institutional decision makers) can lead to binding guidelines or direct changes in coverage. On the one hand, this approach can enforce implementation of disinvestment or withdrawal decisions by the exertion of legal power. On the other hand, it can provoke resistance of physicians and patients who feel left out of the decision-making process. In contrast, bottom–up approaches may facilitate the uptake of recommendations because physician organizations themselves promote implementation based on their own appraisal. However, because no binding guidelines or statutory withdrawal of resources are brought about, the implementation of recommendations is based solely on the voluntary commitment of physicians and cooperation of patients. Thus, active communication and education to impart knowledge are crucial for the success of these strategies. The popularity of the US-American Choosing Wisely® campaign attests to the lively interest of physicians and patients in that issue. A combination of top–down and bottom–up approaches may yield the best results: a government-supported initiative that provides funding for disinvestment activities resulting in binding guidelines and actual changes in coverage/resource reallocation underpinned by the broad involvement of physicians and other stakeholders developing practice-related, informative recommendations.
Due to differences in healthcare systems, initiators and target groups, both the identification and the prioritization methods differ among the programs. However, the choice of method per se is less important as long as it meets the setting requirements and is “fit for purpose.” An adequate combination of methods consistent with the surrounding conditions (as conducted by the “Framework for reviewing existing MBS items” and PBMA) might be the best solution.
To identify the ineffectiveness of health technologies, all disinvestment programs depend on the availability of evidence for effectiveness, safety and cost-effectiveness. However, there are health technologies with little or no evidence, rendering their evidence-based evaluation impossible. Efforts to overcome this barrier include the “Uncertainties Database” run by the Swedish Council on Technology Assessment in Healthcare (SBU) that aims to encourage research on health technologies with missing evidence. In addition, a new research design described by Haines et al. (Reference Haines, O’Brien and McDermott11) is intended to evaluate technologies with uncertain effectiveness by generating practice-related evidence. For evidence-based reassessment processes in particular, it will be crucial to increasingly assess health technologies throughout their entire life cycle. The controversial nature of some topics may raise the need to consider additional evidence on ethics, values, and social norms.
In addition to the challenge of a poor evidence base, ineffectiveness cannot be defined in absolute terms due to differences between patient groups and indications. Some technologies that are ineffective under certain conditions may be effective in other situations (e.g., different stages of disease) (Reference Garner and Littlejohns8;Reference Elshaug, McWilliams and Landon12), making the process of identification and assessment very complex. Thus, only few technologies will qualify for a complete withdrawal of resources and/or removal from the healthcare system (Reference Mc Kean, Noseworthy, Elshaug and Clement4); in most cases, restrictions to certain application areas will be a more reasonable approach (Reference Garner and Littlejohns8). Decisions regarding the breadth of the considered spectrum of ineffective health technologies must be determined at the beginning of the reallocation process to adjust the identification criteria. Thus, there must be a clear definition of what is considered ineffective within a certain program.
Physicians function as informants, links to patients and “front-line decision makers” (Reference Henshall, Schuller and Mardhani-Bayne10). However, the most successful incentives and “levers” in encouraging practice changes are subjects of current research. New but not necessarily effective health technologies are implemented more willingly in health care than those that are proven ineffective are withdrawn (Reference Howard and Shen13;Reference Scott and Elshaug14). Thus, the continuous involvement of health professionals, especially physicians, is considered essential to ensure support for the sustainable implementation of recommendations (Reference Henshall, Schuller and Mardhani-Bayne10).
Although patients (and consumers) are part of advisory panels or working groups in some programs, there seems to be a gap between the emphasis placed on patient participation and practice. Despite the intention to actively involve them in the identification, prioritization and decision-making process, in reality they are costumers rather than contributors. This may relate to the fact that guidance on suitable methods for including patients and the public throughout the process as well as an allocation of adequate resources for enabling effective involvement are often lacking. The need for more research on guidelines to facilitate a broader involvement was amongst others acknowledged by the HTAi subgroup on patient and citizen involvement (15). The group is providing resources on patient involvement principles and continuously develops tools that may be adapted for disinvestment purposes.
When implementing a disinvestment program, reasonable resource management is long-ranging. A long-term strategy with clear allocation of responsibilities is needed to enable sustainable changes in practice. Financial and human resources must be ensured in the long term to allow continuous coordination and performance (Reference Gerdvilaite and Nachtnebel7;16) of a program. Investments in the implementation of a reassessment program are inevitable: long-term gains in efficiency sometimes require short-term resource inputs (Reference Haas, Hall, Viney and Gallego1;Reference Garner and Littlejohns8). The need for extra resources could be reduced by integrating processes into existing Early Awareness and Alert Systems: identification of new or emerging health technologies could be performed concomitantly with the detection of ineffective and obsolete technologies (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua17). For government-initiated programs, tying a program to existing controlling tools (e.g., maintenance of a catalogue of benefits, conditional coverage, coverage under evidence development) and establishing new tools (e.g., coverage of new technologies provided only that ineffective technologies are removed concurrently) could facilitate the implementation of reassessment processes.
In all healthcare systems, the decision-making process is complex (Reference Atienza Merino, Varela Lema, Velasco, Børlum, Palmhøj and Busse18). Real-world decisions are subjective and emotional to some degree; demands by stakeholders and negative associations between disinvestment and rationing may lead to decision makers’ reluctance to implement recommendations even if the evidence clearly favors disinvestment of certain technologies (Reference Mc Kean, Noseworthy, Elshaug and Clement4). Thus, an increase in political will is needed to balance the influential factors (e.g., regarding values, ethics, and norms) and to allow an evidence-based resource (re-) allocation. Decision makers must be role models for relevant stakeholders, increasing confidence in and raising awareness of evidence-based decision making. The benefits of reassessment activities must be promoted publicly: freed resources can be reallocated to more effective and safe care, improving healthcare quality while containing or even decreasing costs.
The successful implementation of recommendations and decisions is a challenge, but the evaluation of their impact is even more difficult and has not been tackled systematically by any of the programs yet. Communication with experts revealed that results from evaluation efforts are rarely published. In addition, evaluations are mostly qualitative as quantitative impact assessments are described as particularly difficult (Reference Mitton19) given the lack of available data. Even though a thorough impact analysis of disinvestment programs was not within the scope of this article, some evidence for changes in healthcare practice was found: cost savings arising from the PBMA program (Reference Mitton19;Reference Mitton, Dionne, Damji, Campbell and Bryan20) and reallocation of resources following recommendations of PBAC (5) have been reported. For some NICE recommendations, indications for a lack of impact on practice decisions exist (Reference Chamberlain, Martin and Busby21). For the other programs, no evidence regarding their impact was found and thus, no conclusions can be drawn regarding the actual practicality of the programs. A quantitative and qualitative evaluation using relevant indicators (e.g., changes in usage and reimbursement, cost reduction, and/or resource reallocation, acceptance by physicians and patients, unexpected consequences) with data analysis and stakeholder interviews (Reference Mitton, Dionne, Damji, Campbell and Bryan20) should be the final step in the reassessment process. This allows necessary modifications of processes, permanent adjustments to healthcare systems and monitoring of “unintended consequences” (Reference Mc Kean, Noseworthy, Elshaug and Clement4).
LIMITATIONS
We restricted our search to information provided in either English or German language, potentially leading to a loss of literature published in other languages. However, the search rarely detected non-English articles, and most of the HTA Web sites provided information in English. Therefore, it is unlikely that relevant evidence was missed.
In addition, policy development in healthcare systems is affected by multifaceted contextual factors (Reference Lavis, Røttingen and Bosch-Capblanch22), and relevant information regarding implemented programs and activities often is not (or only partly) published in the peer-reviewed scientific literature, making it challenging to systematically catch it. We minimized this limitation by interviewing international experts in the field of disinvestment and reassessment of health technologies and by supplementing the systematic search with a nonsystematic literature search to also identify gray literature.
CONCLUSION
As healthcare budgets remain limited, the need for evidence-based disinvestment decisions based on practicable programs will increase in the future. Currently, a growing number of initiatives have been founded around the world with the goal of raising awareness of this issue. To facilitate progress, these initiatives must be supported and translated into concrete action by decision makers and other stakeholders. Healthcare systems that are taking their first steps toward disinvestment activities need to consider the importance of continuous stakeholder involvement and political support. Transparent methods that are properly tailored toward target groups to evaluate health technologies throughout their entire life cycles must be applied in long-term implementation strategies that include an evaluation of the program's impact.
Apart from differences in conduct and methods, the analyzed programs are quite similar in many aspects. Based on the available information, none of the analyzed programs can be rated as “best practice” in terms of informing real-world decisions on optimal resource management. This article shows that both bottom–up and top–down approaches can improve the quality of health care and enable resource savings or reallocation in supportive conditions. The implementation of a program must be supported by decision makers, physicians, and patients and meet the needs of individual healthcare systems by taking into account the respective context.
Future research could assess the impact of disinvestment recommendations and decisions on healthcare quality and cost reduction. Outcomes such as freed resources, changes in healthcare practices and attitudes of stakeholders could be measured systematically. There is a need for more evidence on the effectiveness, cost-effectiveness and safety of health technologies that have reached the end of their life cycles. Thus, research designed to enlarge the evidence base regarding these outcomes by conducting assessments and collecting data should be promoted. In addition, the question of how to deal with technologies of uncertain effectiveness and suitable ways to involve patients and the public in disinvestment decisions could be further investigated.
CONFLICTS OF INTEREST
The authors report no potential conflicts of interest.
