Background
The use of health technology assessment (HTA) to inform decision making is increasing globally (Reference Banta1). Although the overall objective of HTA is to promote an equitable, efficient, and high-quality health system, the extent to which assessments are uniquely deployed in any given geographic setting is variable (Reference O’Rourke, Oortwijn and Schuller2). Many markets have long-established bespoke approaches to HTA; some focus primarily on assessing the relative clinical benefits of a new health technology (e.g., Germany, France), while others integrate formal cost-effectiveness analyses into the appraisal (e.g., England, Australia).
Ideally, HTA appraisals should be customized to the local decision context. Increasingly, however, markets are seeking insights from external HTAs to assist with decisions surrounding the adoption of new technologies. Some markets, particularly in settings where local data are lacking, may import an entire external HTA, inclusive of results, to aid with resource allocation decisions (Reference Banta1). The conduct of “joint assessments,” in which an HTA process is centralized and produced for multiple jurisdictions simultaneously is another emerging trend. The European Network for Health Technology Assessment (EUnetHTA) is one such vehicle; this work has now been formalized as part of a European Union regulatory action on HTA (3).
Although these efforts may allow faster value assessments, they introduce other problems. For example, while HTA bodies often review similar sets of clinical evidence, there is well-documented and country-specific variability in the criteria and priority for use, as well as the key inputs for decision making (Reference Drummond, Barbieri and Cook4;Reference Angelis, Lange and Kanavos5). In addition, although countries may have similar economic profiles, the burden of disease, existing clinical practice and treatment alternatives, design of the health system, overall and out-of-pocket health spending, and data infrastructure can vary substantially. Finally, significant differences have been observed across countries in the values that individuals place on states of health, health dimensions, and the potential benefits or harms of treatment (Reference Heijink, Reitmeir and Leidl6).
These variations naturally lead to questions regarding the appropriateness of transferring value assessments and findings from one setting to another. To gain a better understanding of the differences in the value assessment context across jurisdictions, we reviewed the environment, infrastructure, and practice of value assessment in six countries. We also discussed contextual and practical considerations that limit value assessment transfer between settings.
Methods
We focused attention on Australia, China, Germany, Japan, England, and the United States. These countries were selected because of differences in their approach to and maturity of HTA, variable health system structures, differences in healthcare financing and reimbursement (e.g., fully public vs. public–private hybrid), and geographic diversity. In addition, the conduct of HTA within each of these jurisdictions serves a different function within the health system.
The HTA organizations that we included in our review were Australia’s Pharmaceutical Benefits Advisory Committee (PBAC), China’s National Center for Medicine and HTA, Germany’s Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), Japan’s Center for Outcomes Research and Economic Evaluation for Health (“Core 2 Health,” C2H), and the National Institute for Health and Care Excellence (NICE), serving England and Wales. We also included the Institute for Clinical and Economic Review (ICER), a nonprofit organization with private foundation funding that conducts HTAs in the U.S. without an official government mandate. These organizations do not always represent singular HTA bodies within their respective jurisdictions. For example, several groups in the U.S. have developed frameworks and models for value assessment, including the American Society of Clinical Oncology, Innovation and Value Initiative (IVI), some individual payers, and others. In addition, some groups (e.g., C2H) complement or support broader value assessment activities within the countries of focus.
We evaluated the diverse settings in which each organization operates and explored how differences affect the transferability of value assessments. We also examined the influence of contextual and operational differences on three recent drug appraisals conducted across these organizations.
Findings
Contextual Considerations that Shape HTA
Organizations conducting HTA have different mandates, reflective of the unique context surrounding their respective jurisdictions. These contextual differences not only influence the role of HTA within a system, but also its application to decisions about the uptake of health technologies (Reference Drummond, Barbieri and Cook4). As an example, equitable or extensive uptake of new innovations can be challenging in resource-constrained settings, particularly when such innovations increase health expenditures. China, which is working to achieve universal health coverage as part of a massive healthcare reform, has relatively limited resources with which to optimize health outcomes for its large population. In this context, HTA informs resource allocation as well as pricing negotiations for drugs that are added to the National Reimbursement Drug List (Reference Si, Xu, Chen and Jan7).
Japan, which has already achieved universal healthcare coverage through its statutory health insurance system, established its advisory HTA group (C2H) to inform premium pricing for certain drugs and medical devices. Faced with increasing health expenditures from a growing elderly population and greater demand for innovative technologies, policy makers were seeking to continue to incentivize innovation while addressing rising costs (Reference Kamae, Thwaites, Hamada and Fernandez8). A value-based approach that uses cost-effectiveness analysis (CEA) to adjust the premium portion of high-unit cost or high budget impact interventions was therefore embraced to help promote technological innovation and sustainability in the health system (Reference Kamae, Thwaites, Hamada and Fernandez8).
In other jurisdictions, HTA is intended as an instrument to improve equity in health care. NICE, for example, was founded to address “postcode prescribing,” in which access to drugs varied according to where people lived and whether local health authorities could afford to supply specific technologies (Reference Parsons and Johnstone9). NICE’s mandate, therefore, is to provide national reimbursement guidance to the National Health Service in England and Wales. Australia’s PBAC similarly evaluates pharmaceuticals for inclusion in the national Pharmaceutical Benefits Scheme, with the aim of making affordable medicines available to all recipients covered under the country’s public health insurance program.
Whereas HTA helps inform national reimbursement decisions in England/Wales and Australia, coverage determinations in the U.S. are individually developed across a segmented network of public and private health plans. There is little public transparency into the rationale for formulary decisions, which may often be driven by issues other than value or benefits to the patient (e.g., price negotiations between manufacturers and pharmacy benefit managers). Moreover, there is mixed access to technologies and services across the population, and financial exposure from high out-of-pocket costs. ICER was established with the intention of bringing discussions about value into the public domain, with the goal of promoting sustainable access to high-value care (10). However, ICER does not have an official government mandate, and the extent to which stakeholders consider ICER’s recommendations varies.
Although health insurance in Germany is also composed of autonomous payers known as “sickness funds,” it is not subject to the same segmentation and inequities as the U.S. With few exceptions, these nonprofit statutory funds cover all medications that enter the market. IQWiG evaluates the therapeutic benefit of new technologies to inform how a reimbursement price will be determined. Therapies that are found to have added benefit undergo price negotiation between the umbrella organization of sickness funds and manufacturers, whereas therapies that do not provide such benefit may be included in a reference price cluster, or, if clustering is not possible, negotiated to a level that should not exceed the cost of established therapy (11). The objectives of IQWiG are to maintain quality (through benefit assessment) and efficiency (through a link to pricing mechanisms) in the statutory health insurance funds.
Approaches to HTA
Differences in HTA approach reflect the divergent mandates of the organizations, as well as societal values within each jurisdiction (Table 1). Germany, for example, is critical of the quality-adjusted life-year (QALY) on the grounds of methodological standards, as well as ethical, legal, and cultural norms (Reference Franken, Heintz, Gerber-Grote and Raftery13;14). Societal aversion to rationing, political reluctance to set willingness-to-pay thresholds, and the German Social Code Book V’s assertion that “anyone who needs a treatment is entitled to receive it,” have all been cited as rationale for the limited role health economic evaluations play in decision making (Reference Franken, Heintz, Gerber-Grote and Raftery13;14). Instead, IQWiG focuses on whether a technology provides an added therapeutic benefit for patient-relevant outcomes (most commonly mortality, morbidity, and health-related quality of life) relative to an appropriate comparator.
Table 1. Description of HTA Organizations and Role in Reimbursement Decisions
a Role of C2H is limited to evaluation of premium portion of certain high-cost drugs; evaluation of clinical benefit performed by other agencies in Japan.
b National Healthcare Security Administration convenes experts in health economics to inform negotiation process. Details about how these factor into decisions remains unclear.
c If negotiations fail, interventions may undergo a cost–benefit analysis, but economic evaluation has a limited role in decision-making.
d Products whose expected annual sales are estimated to be JPY 10 billion (USD 90 million) or more are evaluated for CEA.
e No explicit threshold specified, although research has suggested AUD 50,000/QALY (Reference Wang, Gum and Merlin12).
f 1-3× GDP per capita is recommended in national Pharmacoeconomic Guideline but no standard threshold implemented.
g IQWiG uses the “Efficiency Frontier” approach to benchmark the cost effectiveness ratio against the ratios of established treatments within the same disease area.
h $68,800 (JPY 7.5 million) for special products.
i $127,700 (£100,000) for “highly specialized technology.”
j Negative recommendations are not reimbursed; positive recommendations are reviewed by Minister of Health prior to listing on Pharmaceutical Benefits Scheme.
k Under the Act on the Reform of the Market for Medicinal Products (AMNOG), the Federal Joint Committee (G-BA) and IQWiG conduct benefit assessments to inform decisions on the prices statutory health insurance funds pay for new medicinal products.
evLYG, equal value of life years gained; HTA, health technology assessment; QALY, quality-adjusted life-year.
Unlike in Germany, CEA is a central feature of assessments from NICE, PBAC, and ICER. In these settings, the cost-effectiveness threshold is among the key criteria informing the recommendations of an appraisal. Cost-effectiveness thresholds reflect the maximum amount of money a society is willing to invest in a technology to achieve one additional unit of health benefit (e.g., QALY). There is no common method for determining a threshold, and the decision to implement an explicit (vs. implicit) threshold is influenced by the local political, social, and ethical landscape (Reference Cameron, Ubels and Norström15). These differences are evident in our study sample. NICE, for example, applies an explicit threshold range of $25,500–38,300 per QALY (£20,000–30,000/QALY), while ICER presents a range 2–5 times higher ($50,000–200,000/QALY) (10;Reference Wang, Gum and Merlin12). In response to criticisms about the QALY metric, ICER additionally applies thresholds of $50,000–200,000 per equal-value of Life Years Gained (evLYG), which is meant to measure gains in length of life equally irrespective of age, disability or illness. However, ICER developed the evLYG in-house and the metric has not yet been validated. PBAC has not acknowledged an explicit threshold, although evidence suggests that PBAC generally recommends technologies that fall within or below a threshold range of $31,300–44,100 (AU$45,000–$60,000) (Reference Wang, Gum and Merlin12).
While Japan’s C2H also performs economic evaluations, it only recently introduced CEA within the Japanese healthcare system and its role is limited to informing adjustments to the premium portion of certain high-cost drugs and medical devices. Other organizations within Japan are responsible for making decisions about reimbursement and pricing (i.e., the Central Social Insurance Medical Council “Chuikyo” and the Ministry of Health, Labour, and Welfare).
Beyond clinical and cost-effectiveness, other elements of value important to patients and society are considered in the decision-making process; these elements, and the weight placed on them, vary by jurisdiction. Australia, England, and Wales evaluate unmet clinical need and the rarity of the condition, for example, while ICER states that it considers several additional value elements, including caregiver impact, productivity effects, and the effect of an intervention on underserved communities (Reference Angelis, Lange and Kanavos5;10;16).
Case Example: Application of HTA in Different Jurisdictions
As described, the HTA organizations in our sample play different roles and apply diverse criteria to decision-making processes. We evaluated whether such variations led to different conclusions about value for three recently approved therapies: two Calcitonin-Gene-Related Peptide (CGRP) inhibitors for the prevention of migraine (erenumab and fremanezumab), and tisagenlecleucel, a chimeric antigen receptor T-cell therapy for childhood B-cell acute lymphoblastic leukemia and diffuse large B-cell lymphoma. Our analysis focused on the original assessments of each therapy, as conclusions were more likely to be drawn from contemporaneous clinical evidence across the relevant settings. For this analysis, we did not find sufficient data from China and only limited information from Japan was available.
We identified several key differences in the scoping decisions and recommendations for the drugs in our sample (Table 2). Whereas ICER and IQWiG assessed the evidence on erenumab and fremanezumab for a broad population of patients experiencing at least four migraine days per month, NICE and PBAC focused more narrowly on patients with chronic or episodic migraines and an inadequate response to at least three prior prophylactic therapies (Reference Ellis, Walton and Otuonye17–20;29). Some organizations (ICER and NICE) considered the evidence for erenumab and fremanezumab in chronic and episodic migraine separately, while IQWiG noted that the distinction between indications was unclear and did not consider the treatment benefit to be limited to episodic migraines. Similarly, PBAC’s assessment focused on chronic migraine, but the committee noted that the drug was effective in episodic migraine treatment and therefore likely to be used in a broader population.
Table 2. HTA of Erenumab, Fremanezumab, and Tisagenlecleucel
a Benefit assessments conducted by IQWiG and the Federal Joint Committee (G-BA) inform decisions on the prices statutory health insurance funds pay for new medicinal products.
B-ALL, B-cell acute lymphoblastic leukemia; C2H, Center for Outcomes Research and Economic Evaluation for Health; DLBCL, diffuse large B-cell lymphoma; HSCT, hematopoietic stem cell transplant; HTA, health technology assessment; ICER, Institute for Clinical and Economic Review; IQWiG, Institute for Quality and Efficiency in Health Care; MSAC, Medical Services Advisory Committee; N/A, not applicable; NICE, National Institute for Health and Care Excellence; PBAC, Pharmaceutical Benefits Advisory Committee; QALY, quality-adjusted life-year.
Differences in how HTA organizations defined the population of interest influenced their selection of comparators. ICER and IQWiG considered several treatment options to be appropriate comparators for erenumab and fremanezumab, including topiramate, propranolol, amitriptyline, botulinum toxin A, and best supportive care (BSC), although IQWiG further specified that BSC was only appropriate in patients who do not respond to, tolerate, or for whom other prophylactic migraine therapies are unsuitable. Nevertheless, data limitations at the time of original assessment precluded both organizations from making recommendations about the effectiveness of either drug compared to anything but BSC. NICE and PBAC reviewed a narrower set of comparators in the population for whom prior preventive treatment had failed; NICE compared erenumab and fremanzumab to botulinum toxin type A and BSC for chronic migraine, but focused only on BSC as a comparator for episodic migraine (Reference Ellis, Walton and Otuonye17–22;30). Appraisals from PBAC prioritized evidence compared to botulinum toxin type A.
Dissimilar value judgements are also evident in the analyses of these three drugs. For example, IQWiG and ICER issued favorable recommendations about erenumab, while NICE did not recommend reimbursement and PBAC deferred making a recommendation until uncertainties about its clinical benefit, cost-effectiveness, and financial impact could be resolved (Table 2). Discrepancies were also prevalent in judgements about fremanezumab; for example, NICE recommended it for chronic (but not episodic) migraine, while PBAC deferred its decision and IQWiG deemed fremanezumab’s “added benefit not proven” (22).
Appraisals of tisagenlecleucel for B-cell malignancies highlighted differences in how organizations approach rare and/or severe conditions. Although the most plausible incremental cost-effectiveness ratios exceeded conventional thresholds applied in England, NICE recommended reimbursement of the therapy within the Cancer Drugs Fund (24;31). In Germany, an added benefit is automatically assumed for orphan drugs; therefore, IQWiG’s review did not classify the magnitude of additional benefit (26).
Moreover, while the organizations largely reviewed the same clinical evidence for tisangenlecleucel and highlighted similar concerns about its duration of benefit and cost-effectiveness, tolerance for uncertainty varied. Australia’s Medical Services Advisory Committee deferred a recommendation in pediatric leukemia and did not recommend reimbursement for adults with diffuse large B-cell lymphoma until uncertainties in the economic evaluation could be resolved (25). ICER judged there to be insufficient data to model its cost-effectiveness in adults, but found it cost-effective for its pediatric indication (Reference Tice, Whittington and Walsh23;24).
Discussion
Each organization conducting HTA in our sample was established to address unique objectives within the local health system. For example, while HTA plays a supportive role in how new therapies are priced in Japan and Germany, NICE and PBAC explicitly implement HTA to inform adoption decisions for beneficiaries of their national insurance schemes. ICER, which operates in a segmented health system with high patient out of pocket expenses and other access barriers, aims to bring conversations about value into the public domain.
In addition to addressing different decision problems, HTA is tailored to support local standards and preferences. In Germany, economic evaluation does not play a major role in decision making, due in part to opposition to rationing and the principle that everyone is entitled to receive treatment who needs it (Reference Franken, Heintz, Gerber-Grote and Raftery13). In other settings, CEA constitutes a key component of an HTA. Moreover, other, less quantifiable factors that are considered in local prioritization frameworks (e.g., innovativeness, level of unmet need) influence value decisions beyond evaluations of clinical- and cost-effectiveness, and differ considerably by country. As evidenced by our case example, some jurisdictions accept greater evidence uncertainty and/or less favorable cost-effectiveness estimates when an intervention addresses an unmet need for a rare or severe condition. The case example also suggested that the ways in which organizations specify the populations and comparators for their reviews is variable and may not generalize across settings.
Collectively, these differences indicate that the wholesale importation of value assessments from other settings is inappropriate. Nevertheless, markets do seek insights from external or multijurisdiction HTAs. Adjustments to external assessments may be better than indiscriminate importation of methods and findings, although still not ideal.
Some elements of value may be more transferable than others. As noted in our case example, organizations commonly consider similar sources of clinical effectiveness evidence (Reference Angelis, Lange and Kanavos5). Decision-makers and HTA evaluators need to judge whether the available clinical data can appropriately be generalized or adapted to their local context, taking into consideration disease incidence, severity, and the availability of healthcare resources (Reference Drummond, Barbieri and Cook4). Another important consideration is whether differences in clinical practice limit transferability, which may lead to the selection of different comparators.
More comprehensive transferability efforts have taken place through EUnetHTA. The network developed a Core Model for standardizing production of HTA evidence across Europe, which is intended to be flexible to allow for adaptations of assessments within local contexts. However, efforts to apply the model have required substantial adaptation to make reports fit for purpose (Reference Macpherson and Thompson32). The collaborative Joint Clinical Assessments focus on only four domains from the Core Model: description of the technology; description of the health problem; evaluation of relative clinical effectiveness, and relative safety. Domains related to economic, ethical, organizational, patient and social, and legal evidence are excluded from EUnetHTA’s Joint Clinical Assessments. Furthermore, there have been just twelve Joint Clinical Assessments of pharmaceuticals completed since 2016, suggesting how difficult it is to undertake collaborative appraisals when jurisdictions have different priorities and approaches to value assessment.
Much has been written about the potential transferability of economic evaluations specifically, and it remains an area of active research (Reference Drummond, Barbieri and Cook4). In resource-constrained settings, limited technical capacity, insufficient resources, and poor data availability limit the development of de novo models, despite increased demand for economic evidence (Reference Alshreef, MacQuilkan and Dawkins33). These challenges have generated interest in CEA transfer. However, whether CEAs can adequately adjust for the most relevant contextual differences is open to debate. Even in situations where multiple adjustments are applied, major data gaps remain, and the work is extremely labor-intensive. In an evaluation of adjuvant treatment of early breast cancer that was adapted from the United Kingdom to South Africa, for example, locally derived health state utilities and transition probabilities were lacking, and clinical effectiveness data with generalizability to South African ethnic groups were not available (Reference Alshreef, MacQuilkan and Dawkins33). In addition, the model structure did not reflect actual clinical practice in South Africa. Investigators concluded that adaptation is possible but challenging, and depends greatly on how much heterogeneity exists regarding clinical practice and cultural conditions across settings (Reference Alshreef, MacQuilkan and Dawkins33).
Although CEA is just one component of value assessment, analyses that do not adequately represent the health status and preferences of the local population would be inappropriate. Moreover, the ability to replicate or adjust existing economic models from other settings requires a level of transparency that is rarely present in the public domain. In the absence of model sharing, and without local expertise to reconstruct published evaluations from others, the ability to customize existing analyses may be infeasible.
Conclusions
Contextual differences influence both the role of HTA as well as its application to local decisions regarding the uptake of health technologies. These differences limit the transferability of value assessments between settings. De novo appraisals, customized to the local decision context, are undoubtedly the ideal approach to determinations about value. Nevertheless, in resource-constrained settings, limited transferability of relevant aspects of evidence, such as clinical benefit, recognizing and acknowledging the contextual limitations, may be an option. Further investigation is required to produce more insights on whether, when, and how transferability of value assessments may be relevant.
Funding Statement
This work was supported by Janssen, Inc. Pharmaceutical Companies of Johnson & Johnson.
Conflicts of Interest
This work was funded by Janssen, Inc. P.G.S. and D.A.O. were previously employed by the Institute for Clinical and Economic Review (ICER) and continue to receive grant funding from ICER. P.G.S. and D.A.O. have received consulting fees from the Center for Global Development to lead HTA trainings unrelated to the current work. D.A.O. reports additional consulting fees and honoraria from several pharmaceutical and consulting companies, outside the submitted work, and serves as the Chair of the Health Technology Assessment International Global Policy Forum. P.-J.L. has no relevant financial or nonfinancial relationships to disclose. S.S.H. and C.G. are employed by Janssen, Inc.
