Internationally, strained economies have resulted in limited healthcare system resources. To maximize investment, health technology assessment (HTA) agencies, governments, policy makers, and academics are exploring the development of decision-making frameworks to value marketed drug technologies. The premise of these frameworks is to proactively conduct an evidence-based HTA of a drug technology to inform use by identifying “low-value” technologies to allow for reallocation of resources to technologies considered “high-value.” The term “disinvestment” is commonly used within the literature to describe decision-making frameworks that value marketed technologies. Although, the concept of disinvestment also includes reallocation of funding to value-added technologies, the literature rarely reviews the process and outcome of reallocation following a disinvestment decision.
For ease of review, this study will adopt the term disinvestment to reflect HTA frameworks that value marketed technologies. It is important to consider the context of these frameworks, as the emphasis should not be placed on cost-savings but instead on optimizing the use of a drug technology, thereby improving the efficiency and quality of care within a public healthcare system (Reference Garner and Littlejohns1;Reference MacKean, Noseworthy and Elshaug2). Disinvestment frameworks value marketed drug technologies that were not initially subject to rigorous HTA or after initial prescription medication coverage approval as a matter of course, due to new clinical/safety/cost data or comparator changes.
Disinvestment frameworks include some or all of the following process steps: identification and prioritization, assessment (including interpretation, organization, and summarization of data and decisions), and decision dissemination strategies. However, the components within these steps and methodology associated with each step differs within the literature.
The objective of this study is to systematically review disinvestment framework process components for drugs and to report on framework components and disinvestment challenges and solutions. In doing so, it is hoped that this information can highlight methodological gaps and better inform the development of applicable frameworks to value marketed drugs.
METHODS
Literature Search Strategy
A literature search strategy was developed and the following bibliographic databases searched: MEDLINE, EMBASE, NLM PubMed, the Cochrane Library, and CINAHL. The search strategy was comprised of both controlled vocabulary and keywords as follows: reassessment, reallocation, reinvestment, disinvestment, delist, decommission, or obsolescence. The search was limited to English language documents only available from January 1, 2000, onward, and was completed on November 14, 2015. In addition, a gray literature search was completed. Web sites of organizations listed as members of the International Network of Agencies for Health Technology Assessment (INAHTA) and Health Technology Assessment International (HTAi) were searched. References from the included papers and gray literature were searched to identify further items for consideration. Details of the literature search strategy may be found in Supplementary Table 1.
Table 1. Definitions of Disinvestment Terms
Selection Criteria
Titles and abstracts retrieved from the literature search were screened for relevance using a screening form. Literature was included if it pertained to health technology disinvestment for drug technologies and contained information relevant to practices or theory of disinvestment of drug technologies. Literature was excluded if it was focused on budgeting or economic analysis without context to disinvestment or reported on case studies without context to a model and/or framework or program for disinvestment. The full text of any relevant items passing title/abstract screening (Level 1) was retrieved. Full-text review was conducted in duplicate with a second reviewer who had no direct involvement in this research. Discrepancies were discussed, consensus was achieved, and a Cohen's kappa statistic was calculated.
Data Extraction
The author extracted relevant data from the selected literature using a standard extraction form (Supplementary Table 2) which was designed a priori. Data extracted included: focus and summary of the key messages; methods; country of focus; disinvestment terms and definitions; identification and prioritization criteria and methods; assessment process; stakeholder engagement and delivery arrangements; and disinvestment challenges and solutions. A second reviewer verified the data on 20 percent of the extraction forms.
Table 2. Key Challenges and Solutions for Drug Disinvestment
Quality Assessment
A modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach was used as the literature selected is descriptive, fact based, and/or theoretical in nature.
Data Synthesis
The characteristics of included articles were summarized using a narrative synthesis. Each of the selected articles was reviewed for terms describing an evidence-based assessment of a drug technology. These terms, their definition, and the number of articles referring to a specific term were recorded. Common identification and prioritization criteria or methods were grouped with reference to their use and the number of articles referencing each term were noted. Disinvestment assessment methods and stakeholder information dissemination strategies were grouped and references captured. Finally, disinvestment challenges and solutions were organized by key theme and specific components captured.
RESULTS
The systematic literature search identified 4,774 articles after duplicates were removed. Of these, 4,711 were excluded after initial screening and 63 progressed to full-text article review. The full-text review excluded twenty-three articles. Nine articles were excluded as they did not refer to disinvestment for drug technologies; twelve were excluded as they contained no information relevant to practices or theory of disinvestment of drug technologies; and two were excluded as they reported on case studies without context to a model and/or framework or program for disinvestment.
Forty articles met the selection criteria and were included in this systematic review. A Cohen's kappa statistic of 0.83 was found for interrater reliability of the inclusion of articles for data extraction. No discrepancies in data extraction were found. A PRISMA flow diagram (Reference Moher, Liberati, Tetzlaff and Group3) (Figure 1) details the number of publications selected through each stage of the systematic review.
Figure 1. PRISMA flow diagram (Reference Moher, Liberati, Tetzlaff and Group3).
Study Characteristics
Supplementary Table 3 provides a summary of the study characteristics. Methods used in the selected literature varied. Four articles contained systematic reviews; three articles were mixed methods, two articles were qualitative studies; a survey was conducted in one article; three articles focused on guideline development; four articles were reviews (but not systematic reviews); fifteen papers were discussion/commentary or position papers, two were PowerPoint presentations; and six were content from Web sites. Supplementary Table 3 contains additional study information including the focus of the citation, a description of the paper, and the country(ies) referenced within the literature.
Disinvestment Terms and Definitions
Disinvestment terms, their definition and the number of articles referring to a specific term may be found in Table 1. The most commonly used term, referenced twenty times, was “disinvestment.” Consensus seems to have settled around the definition of disinvestment originally proposed by Elshaug et al. (Reference Elshaug, Hiller, Tunis and Moss4) with eleven other references also defining disinvestment as “the processes of (partially or completely) withdrawing health resources from existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain for their cost, and thus are not efficient health resource allocations” (Reference Garner and Littlejohns1;Reference Gnjidic and Elshaug5–Reference Elshaug, Hiller and Moss14).
The other term used, also specific to a disinvestment outcome, was “obsolete/obsolescence” (referenced 5 times) (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Gerdvilaite and Nachtnebel10;Reference Morland15–Reference Joshi, Stahlnisch and Noseworthy17). Obsolescence was generally defined as the end of the lifecycle of a technology when it has been superseded by another alternative. There was one reference to “appropriateness.” Appropriateness is “the proper or correct use of health services, products and resources” (18), which is juxtaposed to terms such as disinvestment and obsolescence.
Finally, reassessment (referenced three times) is defined as “a structured, evidence-based assessment of the clinical, social, ethical, and economic effects of a technology currently used in the healthcare system, to inform optimal use of that technology in comparison to its alternatives” (Reference MacKean, Noseworthy and Elshaug2;Reference Joshi, Stahlnisch and Noseworthy17;18).
Frameworks for HTA Disinvestment
Various framework components for the identification, prioritization, assessment, and decision dissemination strategies have been proposed or are in use. Publically available, passive disinvestment lists are meant to spark discussion and action through a reduction in the use of identified low-value tests and treatments. Passive disinvestment lists are generated through national specialty societies experience (e.g. “Choosing Wisely US or “Choosing Wisely Canada”) (Reference Wilson, Ellen and Lavis8;13), during clinical guidance development (e.g., the National Institute for Health and Care Excellence [NICE] searchable database of “do not do” recommendations) (Reference MacKean, Noseworthy and Elshaug2), and through scanning Cochrane reviews (e.g., Cochrane Quality and Productivity Topics available publically through the NHS Evidence Web site) (Reference MacKean, Noseworthy and Elshaug2;Reference Garner, Docherty and Somner19;Reference Hughes and Ferner20).
Conventional health technology assessment frameworks for evaluation are being used infrequently for disinvestment decisions. Examples include NICE's health technology appraisal system (Reference Hughes and Ferner20) and the Pharmaceutical Benefits Advisory Committee's (PBAC) explicit criteria for removing drugs from the Pharmaceutical Benefits Scheme (Reference Haas, Hall, Viney and Gallego21). Finally, frameworks such as the Ontario Reassessment Framework (22) and two Spanish frameworks: “Guidelines on the Identification, Prioritizing, and Evaluation of Obsolete Technologies” and “Guideline for Not Funding Health Technologies” (GuNFT tool) (Reference Garcia-Armesto, Campillo-Artero and Bernal-Delgado23) have been developed. Little information is available regarding the use and utility of these frameworks.
Identification and Prioritization Criteria and Methods
Drug technology disinvestment criteria for identification and prioritization were numerous and varied by author and framework. However, almost all criteria were developed based on the concept of disinvestment or obsolescence and, therefore, focused on risk to the patient (safety concerns) or lack of improvement to health coupled with a high budget impact or failure to show cost-effectiveness. The most prevalent criteria for drug identification were unacceptable potential risk for patient (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Parkinson, Sermet and Clement7;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10) and evidence that the technology causes overall worsening of health (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Parkinson, Sermet and Clement7;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9).
There lacked a clear differentiation between the actions of identifying or prioritizing a drug for disinvestment as often the same criteria could be found as an indicator for identification, prioritization, or both within the literature. Most frequently cited criteria for identification or prioritization or both included: no scientific clinical evidence proving a technology improves health (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Parkinson, Sermet and Clement7;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Pearson and Littlejohns24), lack of disease burden (Reference Parkinson, Sermet and Clement7;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16;Reference Pearson and Littlejohns24;25), high budget technologies (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16;Reference Pearson and Littlejohns24–Reference Henshall, Schuller and Mardhani-Bayne26), and lack of cost-effectiveness (Reference Parkinson, Sermet and Clement7;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;13;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16). Most prevalent prioritization criteria were safety concerns (Reference Parkinson, Sermet and Clement7;Reference Gerdvilaite and Nachtnebel10;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16;Reference Paprica, Culyer, Elshaug, Peffer and Sandoval27) and impact to public health (Reference Parkinson, Sermet and Clement7;Reference Gerdvilaite and Nachtnebel10;18). Supplementary Table 4 outlines identification and prioritization criteria found through the systematic review.
Methods found for identification of drugs for disinvestment almost always involved a search or monitoring and review of publically available literature and databases (Reference Gerdvilaite and Nachtnebel10;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16;Reference Garner, Docherty and Somner19;Reference Haas, Hall, Viney and Gallego21; Reference Garcia-Armesto, Campillo-Artero and Bernal-Delgado23;25;Reference Henshall, Schuller and Mardhani-Bayne26;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea, Asua, Benguria-Arrate and Galnares-Cordero28). Consultation with clinical specialty groups, clinicians, healthcare administrators, and funders (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Ruano Ravina, Velasco Gonzalez and Varela Lema16;25;Reference Henshall, Schuller and Mardhani-Bayne26), and assessment of variation in technology use (e.g., geographic, provider variation in care) (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Garcia-Armesto, Campillo-Artero and Bernal-Delgado23;25;Reference Henshall, Schuller and Mardhani-Bayne26;Reference Paprica, Culyer, Elshaug, Peffer and Sandoval27) were suggested methods for identification or prioritization or both by many authors. Identification and prioritization methods may be found in Supplemental Table 5.
Disinvestment Assessment Methods
Little information on specific disinvestment assessment methods used to finalize a drug technology disinvestment decision could be found in the selected articles. Authors have provided high-level direction on methods to measure costs, benefit, and value and reference standard HTA evaluation methods. These include evaluating the: disease burden, safety, clinical effectiveness, health gains, cost-effectiveness, opportunity costs (Reference Polisena, Clifford, Elshaug, Mitton, Russell and Skidmore29), and overall value (including ethical, legal, and social issues) (Reference Goodman30). Noseworthy and Clement (Reference Noseworthy and Clement31) indicated that a review must contain an assessment of feasibility and an analysis of consequences, intended and unintended.
Stakeholders and Decision Dissemination Strategies
Politicians, clinicians, specialty societies, health system leaders, industry, and patients are critical components of any disinvestment process. The literature identified that a transparent engagement and consultation process is needed (Reference Garner and Littlejohns1). Decision makers and experts should be actively involved in each step of the disinvestment process to identify candidate technology for disinvestment, to continually improve the methods and infrastructure for disinvestment, and to ensure the feasibility and barriers to disinvestment decisions have been carefully considered (Reference Henshall, Schuller and Mardhani-Bayne26).
Both passive and active methods for information dissemination were discussed by Garner and Littlejohns (Reference Garner and Littlejohns1). Passive dissemination strategies included publication of recommendations on searchable databases or Web sites to encourage stakeholder change (Reference Garner and Littlejohns1). Slightly more active dissemination strategies included incorporating a decision in clinical guidelines or using decision support tools (Reference Henshall, Schuller and Mardhani-Bayne26). Finally, active disinvestment methods, suggested in the literature, included changes to formulary and/or coverage reimbursement listings (Reference Henshall, Schuller and Mardhani-Bayne26).
Disinvestment Challenges and Solutions
Disinvestment challenges were prevalent and detailed within the literature while solutions were broad and hypothetical (Table 2). Stakeholders lack the political, clinical and administrative will to support disinvestment (Reference Elshaug, Hiller, Tunis and Moss4;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;13;Reference Joshi, Stahlnisch and Noseworthy17;Reference Haas, Hall, Viney and Gallego21). Without perceived value or benefit, stakeholders are resistant to losing access to a drug therapy that may still provide some benefit (Reference Elshaug, Hiller, Tunis and Moss4;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;13;Reference Joshi, Stahlnisch and Noseworthy17;Reference Haas, Hall, Viney and Gallego21). As an outcome, stakeholders are hesitant to allocate resources to disinvestment. Without resources, solutions such as methodology and framework development, training of health technology assessors, incentives to clinicians and patients, research to fill data gaps and pilot programs are not possible and a disinvestment strategy will not progress.
Confounding concept uptake even further are the country/region specific complexities of decentralized healthcare structures, variability in insurance services, purchasing processes for drugs, and lack of agreed international disinvestment methodology (Reference Elshaug, Hiller, Tunis and Moss4;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;13;Reference Polisena, Clifford, Elshaug, Mitton, Russell and Skidmore29). Solutions such as multi-stakeholder agreements on health technology processes, international collaboration, and transparent, adaptable disinvestment models require political and administrative will fueled by a perception of value (Reference MacKean, Noseworthy and Elshaug2;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Morland15).
DISCUSSION
Practical solutions are needed to optimize the use of marketed drug technologies due to shrinking or slow-growing budgets. With greater emphasis on the value for money, there has been an increased interest and study of frameworks which can facilitate disinvestment. This systematic literature search aimed to review drug technology frameworks and to report on the framework components, challenges and solutions as found within the literature. In doing so and reporting on the result findings, three broad areas for further study are described below.
Disinvestment and Cost Containment
Country initiatives have focused on “disinvestment” of drug technologies, where costs are reduced by partially or completing withdrawing a drug technology from a health system. The terminology chosen (such as disinvestment) to describe the assessment of a marketed drug can influence stakeholder engagement in the process by inferring a foregone process conclusion (Reference MacKean, Noseworthy and Elshaug2). The use of more decision neutral terms, such as “reassessment,” could improve stakeholder (clinicians, patients, industry) engagement. Reassessment, as defined by MacKean et al., focuses on optimal use and achieving value for money (Reference MacKean, Noseworthy and Elshaug2). Health Technology Reassessment (HTR) may result in numerous economic re- or dis-investment outcomes, which include stopping funding (disinvestment), partial disinvestment (narrowing what is paid for), reinvestment (broadening what is paid for), or no change in use (Reference MacKean, Noseworthy and Elshaug2). By using terms that suggest the possibility of broader outcomes, collaboration and partnerships between stakeholder (clinicians, patients, industry) and governments (policy makers and HTA agencies) could improve. This in turn could lead to multi-stakeholder data generation (e.g., registries, clinical trials) and further political and administrative will.
Identification and Prioritization: Timing and Engagement
The reviewed literature included a variety of identification and prioritization criteria focused mainly on patient risk, high budget impact, and/or lack of cost-effectiveness. Criteria are aligned with the purpose of disinvestment initiatives, that is, one of rationalism where costs of inefficient drug technologies are removed from a finite health system budget to allow for investment in technologies with greater clinical or cost-effective outcomes (Reference Garcia-Armesto, Campillo-Artero and Bernal-Delgado23). Even though disinvestment identification and prioritization criteria are increasingly being adopted internationally, there still exists a question on timing, when and how often technologies are reviewed for disinvestment. In addition, there is a lack of researcher, clinician, consumer, and policy-maker engagement in working together to realize rationalism (Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea and Asua6;Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;13;Reference Morland15;Reference Henshall, Schuller and Mardhani-Bayne26).
Two actions might be considered by HTA agencies to meet these challenges. First, HTA agencies could adopt fixed time periods for HTR (for instance, 5 years after a product is launched on a market, or when new comparators are being assessed) (Reference Elshaug, Watt, Moss and Hiller32). This would eliminate the need for drug technology identification and prioritization reassessment criteria. A standardized process could also incentivize invested stakeholders, such as industry, to generate additional research to fill data gaps or to collect and contextualize data (Reference Garner and Littlejohns1;13;Reference Haas, Hall, Viney and Gallego21;Reference Pearson and Littlejohns24). Alternatively, if this is seen as too resource intensive for government funded budgets (an additional challenge to the concept of disinvestment), an efficient and transparent identification and prioritization process needs to be validated through transparent and collaborative methods by country. After prioritizing a drug for disinvestment, stakeholder engagement could be increased by conducting a therapeutic review reassessment of all drugs within that of the prioritized drugs therapeutic category. In doing so, disinvestment recommendations may result, but at the same time, reinvestment in other drug technologies is also a possibility.
Lack of Tailored Evaluation Frameworks
The valuation of marketed drugs requires a process that is tailored to the assessment, adaptable, transparent, and makes reasoning explicit but also identifies limitations and uncertainty of the evidence, and has a structured approach to decision dissemination and implementation (Reference Elshaug, Moss, Littlejohns, Karnon, Merlin and Hiller9;Reference Gerdvilaite and Nachtnebel10;Reference Elshaug, Hiller and Moss14;Reference Morland15;Reference Henshall, Schuller and Mardhani-Bayne26;Reference Polisena, Clifford, Elshaug, Mitton, Russell and Skidmore29). Qualitative assessment frameworks have been developed which frame decision problems through a structured, consistent approach to decision making by “facilitating the selection, organization, summarization, and interpretation of data and preferences relevant to the decision” and aid decision documentation and communication (Reference MacKean, Noseworthy and Elshaug2). This systematic literature review confirms that, to date, there is not one universally accepted qualitative assessment framework that meets the desired criteria mentioned above or that has been widely adopted for the purpose of drug technology disinvestment.
However, regulatory agencies, such as the European Medicines Agency (EMA) (33) and U.S. Food and Drug Administration (FDA) (34) have now developed and are using qualitative assessment frameworks to aid in benefit-risk decision making and decision dissemination. One area of future research is to consider adapting a qualitative assessment framework used by Regulatory Authorities for marketed drug health technology reassessment to aid in decision-making transparency and information dissemination.
LIMITATIONS
Traditional systematic literature searches for terms related to disinvestment have been documented to have high sensitivity and poor specificity (Reference Parkinson, Sermet and Clement7;Reference Haas, Hall, Viney and Gallego21–Reference Pearson and Littlejohns24). This research encountered the same limitation, where the magnitude of search results were high and required extensive review to target relevant articles. In addition, there is a documented publication bias as government and payer disinvestment initiatives are generally absent from publication (Reference Parkinson, Sermet and Clement7). This bias made it difficult to ensure all current frameworks, their components, challenges, and solutions were documented within this review. Only disinvestment frameworks for drug technologies were targeted for review. At times, the literature was not explicit as to whether a framework was meant to review drug technologies; therefore, some frameworks included in this review may not be proposed or in use with drug technologies.
CONCLUSIONS
Disinvestment components and methods for identifying and prioritizing technologies, undertaking assessments, and disseminating review outcomes vary within the literature and challenges are prevalent. This study suggests that stakeholder engagement could be increased through refocusing the terminology from disinvestment to reassessment or another neutral term. Increased engagement may also be realized through the adoption of fixed time HTRs or therapeutic reviews.
Decision-making frameworks developed to assess the benefit-risk of drug technologies within the regulatory context may be appropriate, if modified, for use in the health technology assessment phase of marketed drugs. These frameworks, address some of the challenges cited in the literature. For instance, they allow for stakeholder involvement, encourage transparent processes, allow for flexibility and uncertainty and can be used to define and communicate the context and drivers of a decision. The ultimate goal of a reassessment framework should be to inform the optimal use of drug technologies to improve the efficiency and quality of care within a public health system.
SUPPLEMENTARY MATERIAL
Supplementary Table 1: https://doi.org/10.1017/S0266462317000277
Supplementary Table 2: https://doi.org/10.1017/S0266462317000277
Supplementary Table 3: https://doi.org/10.1017/S0266462317000277
Supplementary Table 4: https://doi.org/10.1017/S0266462317000277
Supplementary Table 5: https://doi.org/10.1017/S0266462317000277
CONFLICT OF INTEREST
This work has been completed in partial fulfillment of the requirements for Mary Alison Maloney's PhD degree in Health Research Methodology at McMaster University. In addition, Mary Alison Maloney works full-time as the Vice-President of North America Consumer Health Regulatory Affairs for Bayer HealthCare LLC. Dr. Lisa Schwartz's Arnold L. Johnson Chair in Health Care Ethics is funded through a private endowment. Dr. Schwartz is a member of an Expert Review Panel for CADTH. Dr. Mitchel Levine and Dr. Daria O'Reilly have no potential conflicts of interest. This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
