Healthcare systems are challenged by the tension between delivering quality care and best treatments to individual patients, serving equitably the entire population covered, and maintaining sustainability (Reference Sabin and Cochran1). Berwick et al. (Reference Berwick, Nolan and Whittington2) proposed that high-value health care will be achieved only if stakeholders pursue a broad system of linked goals related to individual patient health, population health, and healthcare system resource management, referred to as the triple aim. Coverage decision makers frequently face ethical dilemmas as how to represent these apparently conflicting aims in the decisions being made. Reflecting on the extent to which diverse values are represented and traded-off is critical in making accountable and legitimate decisions. Current health technology assessment (HTA) methodologies are constantly challenged to address these issues, as exemplified by the controversy around treatments with acceptable cost-effectiveness ratios that are nevertheless challenging sustainability, such as recent treatments for hepatitis C (Reference Neumann and Cohen3), and issues raised by therapies for rare diseases, which require consideration of aspects usually not formally addressed by current HTA approaches (Reference Wagner, Khoury, Willet, Rindress and Goetghebeur4;Reference Tambuyzer5).
HTA agencies are founded on and make coverage decision making amidst diverse social values (Reference Daniels and van der Wilt6), including substantive values (e.g., fairness), embedded in the evaluation criteria, and procedural values (e.g., transparency, accountability and participation) embedded in the evaluation process (Reference Biron, Rumbold and Faden7;Reference Clark and Weale8). Several have argued for an expanded HTA that goes beyond safety, efficacy, and cost-effectiveness to include elements such as equity, budget impact, and financial protection, embedded in a fair and evidence-based deliberative process that ensures consideration of diverse stakeholder perspectives and meets the conditions for accountability for reasonableness (A4R) (Reference Daniels, Porteny and Urritia9–Reference Baltussen, Jansen and Mikkelsen11).
Hofmann et al. (Reference Hofmann, Cleemput, Bond, Krones, Droste and Sacchini12) stressed the need to acknowledge that value judgments and their diversity are inherent in HTA. Accountability also requires tackling ethical dilemmas in a fair manner, but, although there is a level of agreement on the basic principles for priority setting, as Daniels and van der Wilt (Reference Daniels and van der Wilt6) pointed out, there is no agreement on how to trade-off between competing goals such as population health maximization and fairness. Building on these premises, we propose that resolving the tension faced by decision makers requires further exploration of substantive values on which HTA agencies are founded and of ethical trade-offs inherent to decisions. The objective of this study is to perform this exploration using a narrative approach and reflective multicriteria as a means to foster a culture of pragmatic complexity in an inclusive and deliberative manner to further reasonableness and accountability.
METHODS
Questionnaire
A questionnaire was designed to collect feedback on the mandates and values of HTA agencies, examine ethical underpinnings of the values on which these agencies are founded, and explore trade-offs. EVIDEM was selected as a reflective multicriteria framework to facilitate this process. It contains twenty generic criteria that are explicitly rooted in the key ethical imperatives underlying healthcare decisions: (i) the imperative to prevent or alleviate patient suffering (an aspect of deontology); (ii) the imperative to serve the whole population equitably, which suggests helping first those who are worst off (an aspect of distributive justice, fairness), while providing healthcare to the greatest number (an aspect of utilitarianism); (iii) the imperative to uphold healthcare system sustainability (an aspect of utilitarianism), and (iv) the imperative to make decision informed by relevant evidence and the specific context (an aspect of practical wisdom) (Reference Wagner, Khoury, Willet, Rindress and Goetghebeur4;Reference Goetghebeur, Wagner and Khoury13;Reference Tanios, Wagner and Tony14). (It is acknowledged that in addition to the ones stated, other ethical theories could be summoned to underpin these imperatives.) For the thirteen normative, non–context-specific criteria of the framework, there is universal agreement on the scoring direction (e.g., provided everything else is the same, it is better to have high efficacy, to avoid adverse events, to target the most severe diseases etc.), which makes it possible to design scoring scales that allow assessing an intervention against these criteria in a quantitative manner. (Of note, the term “quantitative assessment” here means eliciting scores that represent interpretations and judgments on the evidence rather than a numerical transformation of the evidence.) For the remaining seven criteria, scoring directions would need to be determined in specific contexts; these are, therefore, assessed qualitatively in the generic framework.
The first part of the questionnaire followed a narrative approach, through which participants provided written statements describing (i) the mandate, (ii) the mission, (iii) the motto, (iv) the social values (substantive and procedural) guiding each agency's process.
The second part followed a reflective multicriteria approach derived from EVIDEM, during which participants were presented with the underlying ethical aspects, methodological justification (nonredundant, independent, operationalizable and complete) and definition of each criterion, and then asked: (a) to explore whether it reflected the values of their institution, (b) whether it was formally considered in their process and (c) provide verbal or written comments through a guided discussion.
In the third part, the quantitative values elicitation exercise, participants explored trade-offs between the quantitatively assessed criteria of EVIDEM (along with their ethical underpinnings) by distributing weighting points across domains and criteria (hierarchical point allocation), as shown in Figure 1. Participants also shared their thoughts on whether such exercise would be helpful to support their individual reflection regarding trade-offs and share their perspectives with others. (Of note, the numbers generated should not be considered as conventional numbers but as a representation of individual reflections.)
Figure 1. Hierarchical point allocation exercise for values elicitation and exploration of trade-offs between criteria along with their key ethical imperatives.
The fourth part of the questionnaire was designed to investigate issues and ethical dilemmas pertaining to appraising interventions for rare diseases. It provided a list of rare disease issues, identified from a systematic review (Reference Wagner, Khoury, Willet, Rindress and Goetghebeur4) and categorized by criteria, and prompted participants to indicate additional issues. To explore trade-offs inherent in appraising interventions for rare diseases, participants repeated the quantitative values elicitation exercise with Population priorities (split into criteria Rare diseases and Other priorities) added as an additional trade-off domain (Figure 1) (Reference Wagner, Khoury, Willet, Rindress and Goetghebeur4).
Workshop Design
We contacted representatives of ten national HTA agencies, based on their interest in values, ethics and multicriteria approaches, to participate in a 1-day face-to-face workshop in Oslo, Norway, in 2015. Participants received the questionnaire before the workshop and completed the first part (narrative approach) individually. During the workshop, participants were introduced to the ethical and methodological principles that underpin the framework and were guided to complete the other parts of the questionnaire. Each part was followed by a structured discussion. The workshop was conducted under the Chatham House Rule to facilitate open exchange.
Data Analyses
All responses from the questionnaires and the discussion were collected, analyzed, and structured in tabular format. Criteria weights collected were analyzed and compared using descriptive statistics. Weights assigned with and without the Population priorities criterion included in the quantitative criteria set were compared to identify where trade-offs related to defined priorities, including for rare diseases, were made. Cumulative weights for clusters of criteria representing key ethical imperatives were also calculated.
RESULTS
Organizations and Participants
Eight participants attended the workshop, each representing one HTA agency: the Belgian Health Care Knowledge Centre (KCE); the Canadian Agency for Drugs and Technologies in Health (CADTH); the Health Technology Assessment Institute (IETS) Colombia; the National Institute for Health and Care Excellence (NICE) UK; the Lombardy Region Health Directorate, Italy; the National Health Care Institute (ZIN) Netherlands; the Norwegian Knowledge Centre for the Health Services (NOKC); and the Instituto de Salud Carlos III, Spain.
Narrative Approach: Mandates, Missions, Mottos and Values of Agencies
Participant statements on the mandates, missions, mottos and values of their agencies expressed a diversity of values (Table 1). Participants represented organizations with the mandate to provide guidance and recommendations to national or regional decision makers. Examination of these statements indicates that, overall, participating agencies aim at promoting the use of impartial evidence in decision making to serve the greater purpose of providing quality care to the population and optimal use of healthcare technology. Many of the various terms provide reflected the key ethical imperatives introduced above, including the imperative to prevent or alleviate patient suffering (reflected in terms such as Improve patient care, Improve patient safety, Clinical benefit, Focus on patients); to serve the whole population equitably (Equitable, Distributive justice, Need), to uphold healthcare system sustainability (Efficiency, Resource stewardship, Sensible care), and to make decisions informed by relevant evidence and the specific context (Evidence-based, Scientific rigor, Scientific excellence and quality). Exploration of procedural values of the organizations reveals a common emphasis on stakeholder involvement/collaboration, independence and transparency.
Table 1. Mandates, Missions, Mottos Values and Ethical Foundations of Agencies
Reflective Multicriteria Approach: Criteria, Ethics, Institutional Values, and Rare Disease Specificities
Participants reported on whether and how each criterion, and its underpinning ethical imperative, reflected the values of their organization and whether it is formally considered in their processes (Table 2). Specific issues reported for rare diseases are also included. Exploring the ethical aspects underpinning the criteria was deemed by participants a useful approach to explore institutional values. Ninety percent of the EVIDEM criteria were thought by at least half of the participants to reflect their institutions’ values, while 70 percent of the criteria were reported by at least half to be considered formally by their institutions.
Table 2. Decision Criteria and Ethical Underpinnings: Participants’ Insights on Their Relation with Institutional Values, Appraisal Processes and Rare Disease Specificities
Criteria Representing the Imperative to Prevent or Alleviate Patient Suffering
The criterion Comparative effectiveness, rooted in the imperative to alleviate/prevent suffering to the greatest extent, was reported by all participants to reflect their institutions’ values and considered in their processes. It was regarded as a dominant criterion, considered extensively in particular with regard to an efficient use of resources, if opportunity cost is sizeable. It was pointed out that high societal expectations related to the benefits of rare disease treatments have to be balanced by careful consideration of meaningful outcomes of effectiveness.
Expressing the call to do no harm at the patient level, Comparative safety/tolerability was reported to reflect all participating institutions’ values and to be formally considered by 75 percent of them. A key criterion alongside effectiveness, considering safety and tolerability was deemed essential to the provision of quality care. Participants pointed out that, unlike regulatory assessment, the HTA process considers the impact of adverse events assuming safety in a regulatory sense to be already established.
Comparative PROs was reported by 75 percent of the respondents to reflect their institutions’ values and by 50 percent to be formally considered. Participants commented that the purpose of this criterion is to take into account patients’ expectations and ethical aspects, in response to the call to respect patients’ autonomy and dignity. However, participants also pointed out that data derived directly from patients, which would be needed to adequately operationalize this criterion, are frequently lacking. Indeed, although noted in the narrative approach as a key substantive value, integration of the patient perspective in the decision-making process remains challenging.
Criteria Type of preventive benefit and Type of therapeutic benefit are rooted in the imperative to eliminate suffering rather than merely alleviate it, either through prevention or cure, respectively. For both of these criteria, 88 percent of participants reported that they reflected their institutions’ values. However, preventive strategies are frequently appraised in separate processes or other agencies, and 38 percent of participants reported that this criterion is not formally considered in their HTA process. The concept of type of benefit, as opposed to extent of benefit assessed under the criterion Comparative effectiveness, was considered to pertain to the notion of maximizing the performance of the healthcare system by aiming toward the most relevant type of benefit. In France, this is made explicit through the concept type of medical service (Service Medical Rendu) used by the Haute Autorité de Santé.
Criteria Representing the Imperative to Serve the Whole Population Equitably
Disease severity, a criterion rooted in the imperative to prioritize those who are suffering or might suffer the most, was reported by 88 percent of participants to reflect their institutional values and to be used formally in their processes. Participants reported disease severity as “a major criterion” and “an essential component.” However, the concept of Disease severity was also questioned on the basis that a condition's natural course may be less important than its course under current treatments. The latter relates to the concept of unmet needs for a given condition, which is a separate criterion rooted in distributive justice with the aim of prioritizing populations who have limited alternatives.
Keeping these two concepts distinct in a value measurement framework allows prioritization of interventions for severe diseases and with unmet needs. All respondents reported that Unmet needs reflected their institutions’ values, which was linked to the goal of improving quality of care, and 75 percent stated it was formally considered. Lack of treatment alternatives for a very severe disease (often the case for rare diseases) was noted as a strong argument in favor of coverage, and one participant reported that this criterion was explicitly formalized for patients with rare diseases. These criteria were not mentioned by participants in the exploration of mottos and mandates, possibly because they reflect an implicit motivating value of healthcare, rooted in compassion (Reference Fotaki15).
Size of affected population, rooted in the imperative of doing the greatest good for the greatest number, was deemed by 63 percent of participants to reflect the values of their institutions. However, more institutions (88 percent) formally considered this criterion, as it was deemed relevant with respect to efficient use of healthcare resources (e.g., size of budget impact). Furthermore, consideration of the size of the population prompted reflections on the role of rarity in resource allocation. It was first noted that rarity is not an indicator of low value of an intervention but also not necessarily of high value, as arguments based on “rights to access” to beneficial therapies are deemed stronger than those based on small population size.
Participants also pointed out that all rare diseases combined represent a significant population. Comments highlighted the debate on trade-offs between the imperatives to alleviate patient suffering, serve the whole population equitably and uphold sustainability, and the challenges to resolve this tension, in particular for rare diseases. This criterion was not mentioned per se in the narrative exploration (Table 1) but triggered a reflection on the social values at stake.
In quest for equity and applying the principle of first helping those who are worst off, societies may prioritize populations that are particularly disadvantaged and face specific challenges in their communities. This type of prioritization, captured by the criterion Population priorities and access, largely depends on local context and values, as opposed to universal prioritization based on disease severity and unmet medical needs. Most participants (88 percent) reported that this criterion reflected their institutions’ values. Indeed one stated that “Consideration of ethical issues, including equitable distribution among vulnerable groups is a core deliberative criterion.” However, only 38 percent reported that this is formally considered. Participants noted that regulatory, and other, policies reflect prioritization of rare diseases (e.g., orphan designation) and that some agencies give special consideration or have developed special approaches to rare diseases. However, approaches to value assessment for rare diseases that profoundly deviate from established general principles were questioned.
Criteria Representing the Imperative to Uphold Healthcare System Sustainability
The majority of participants (88 percent) reported that the criteria Comparative cost consequences – Cost of intervention as well as Other medical costs reflected their institutions’ values. These criteria aim at upholding sustainability and, according to participants, are expressed in the mandate to efficiently allocate scarce healthcare resources. A majority of respondents (63 percent and 75 percent, respectively) indicated that both criteria are formally considered, but, as comments revealed, often only in combined measures such as budget impact or cost-effectiveness.
Comments also highlighted that the best way to consider the different economic aspects involved in the evaluation of an intervention remains a subject of debate fueled by several current controversies (Reference Neumann and Cohen3). Regarding rare diseases, participants noted that, although treatments are often associated with a high cost, small population size limits their budget impact. They also pointed out that “patients with rare diseases require a broad range of medical resources and services,” which highlights the importance of considering the cost of the intervention that is evaluated and other associated medical costs separately.
Fewer than half of participants (38 percent) reported that the criterion Comparative cost consequences – nonmedical costs reflects their institutions’ values or is formally considered. The ethical underpinning of this criterion is the call to preserve societal and individual resources from a broad perspective. Thus it raises the important question of the perspective from which the appraisal is to be conducted. Respondents commented that, although sometimes a societal perspective is needed, there are difficulties in taking this wider perspective “in the context of budgetary silos.”
The criterion Opportunity costs and affordability requires exploring the budget impact of the intervention and potential foregone resources following its implementation with the aim of ensuring healthcare system sustainability. A majority of participants (75 percent) recognized that this criterion was reflected in their institutions’ values, commenting that it is targeted toward efficient use of limited healthcare resources and justifies “implicit rationing.” Half of participants said that “opportunity costs and affordability” was formally considered in their appraisal process, but it was also noted that, although of key importance, it is currently “the most difficult criterion to operationalize.” Others noted that opportunity costs are assessed through budget impact analysis to help clarify financing issues. One participant indicated that “affordability was not deemed to be part of the appraisal but pertained to the decision-making process.” It was noted that “opportunity costs may be foregone disproportionately for rare compared with common diseases because the costs of commercial development of some products may be unavoidably high.”
Mandate and scope of the healthcare system refers to the overall foundation of healthcare and is also a fundamental aspect of healthcare system sustainability. This criterion was viewed by participants as expressed in their respective institution's mission, values, or ethical foundations and stipulated through law or regulation; 25 percent of participants indicated that this is formally considered. This criterion was perceived more as a selection tool for interventions to be appraised rather than part of the appraisal process per se. However, it was acknowledged that some interventions (e.g., in vitro fertilization, growth hormone treatment) may be challenging with regard to a specific healthcare system's mandate and scope and, therefore, would require an explicit consideration of this criterion.
Sustainability in its widest sense includes environmental considerations. Few participants (13 percent) regarded the criterion Environmental impact as reflected in their institutions’ values or as considered in the appraisal, because it was generally deemed to be outside the remit of their organizations.
Criteria Representing the Imperative to Make Decisions Informed by Relevant Evidence and the Specific Context
Quality of evidence considers, based on evidence, the strength of claims about an intervention and includes the important element of relevance to the context of the decision. All respondents confirmed that this criterion reflected their institutions’ values, and 88 percent reported that it was formally considered. According to participants, this criterion reflected scientific excellence and was central to the ability of their institutions to formulate evidence-based recommendations, thus linking the credibility and, hence, legitimacy of their recommendations to the quality of evidence. The lack of knowledge about rare diseases and resulting uncertainty regarding the potential benefits and risks of treatments create a challenge for decision makers.
Expressing another aspect of considering the strength of claims about an intervention in a given context, which is based on expert knowledge and consensus, the criterion Expert consensus/CPGs was reported by 75 percent of participants to reflect their institutions’ values and to be formally considered. Participants noted that this criterion calls for involvement of experts and integration of expert knowledge beyond formal evidence to help ensure that decisions are relevant to the contextual clinical practice, thereby supporting their legitimacy and acceptability.
System capacity and appropriate use in a given context was deemed by the majority of respondents (75 percent) to reflect their institutions’ values and also to be formally considered in their appraisal process. Participants stressed the importance of this criterion and most of its sub-criteria, and drew a link to multiple concepts, including accessibility, quality, and safety of healthcare delivery and “contextual information,” which highlights the variety of system-related considerations that play a role in appraisals.
Awareness of stakeholder pressures and barriers in a given context (criterion Common goal and specific interests) helps ensure that decisions are fair-minded and driven by the common goal rather than special interests. Reflecting on this criterion, half of participants noted that this is aligned with their institutions’ values, but only 25 percent noted that it is explicitly considered in the appraisal process. Participants commented that this criterion is reflected in the pursuit of procedural fairness, which, among others, includes operationalizing the principles of stakeholder involvement and transparency.
Awareness of the “political/historical/cultural context” was deemed by half the participants to be reflected in their institutions’ values and formally considered. However, participants noted that, apart from public health interventions, this criterion is often only implicitly considered. Participants also commented that the sub-criteria “Precedence” (i.e., previous decisions on similar interventions) and “Innovation and research” are both considered, the latter though not through a formal process.
When prompted to suggest additional criteria, participants mentioned the transparency of the reasoning and transparency of evidence used in decision making. Of note, both of these considerations are implicit in multicriteria approaches (Reference Goetghebeur, Wagner and Khoury13;Reference Baltussen and Niessen16;Reference Thokala, Devlin and Marsh17).
Quantitative Values Elicitation Exercise and Exploration of Trade-Offs
The quantitative values elicitation exercise (Figure 2) revealed that, without explicitly established priorities (column A), at the group level, criteria representing the imperative “to alleviate or prevent patient suffering” (i.e., Effectiveness, Safety, PRO, Type of benefit) had the largest cumulative weight (35 percent), followed by criteria representing the imperatives “to serve the whole population equitably” (i.e., Disease severity, Unmet needs, and Size of population) (24 percent), “to uphold healthcare system sustainability” (i.e., Cost of intervention, Other medical costs, Nonmedical costs) (22 percent), and “to make decisions informed by evidence and context” (i.e., Quality of evidence, Expert consensus) (19 percent).
Figure 2. Exploration and visualization of trade-offs between a context when no country-specific priorities are established (A) and a context when country-specific priorities are established (B) (comparison of mean normalized weights elicited through hierarchical point allocation).
Some participants commented that such an approach, while it highlighted the difficulty of balancing the imperatives, may help clarify individual reflection and support the ethical reasoning regarding trade-offs, and the graphical representation might be helpful to share it with others.
Country-specific population priorities are established by healthcare authorities to operationalize the principle of equity (or fairness) within the context of a country's value system. The appropriateness of including an explicit quantitative criterion to reflect trade-offs if priorities are established (including priorities for rare diseases) was discussed. It was noted that “this might lead to a fragmentation of the population and, therefore, it might be more appropriate to keep specific population priorities as a qualitative consideration.” It was also acknowledged that the way population priorities are considered in the decision process needs to be reflected upon.
To explore how the balancing act of decision making might be modified when priorities are established, and to provide a visual illustration of trade-offs, participants repeated the values elicitation exercise with the same criteria set but to which a country-specific “Priorities” domain had been added, which expresses an additional aspect of the imperative to serve the population equitably (column B in Figure 2). Participants who assigned some weight to this domain (half assigned a weight of 0), traded primarily from a reduction in the relative importance of criteria representing the imperative “to make a decision informed by evidence and context” (Quality of evidence, Expert consensus) and of criteria representing the imperative to “alleviate or prevent patient suffering” (Effectiveness, Safety, Type of benefit). At the criteria level, the greatest trade-offs came from Unmet needs and “Quality of evidence.”
DISCUSSION
Reflective multicriteria is useful to explore substantive values of HTAs, to reflect on how these values and their ethical underpinning can be operationalized into decision criteria and to explore the trade-offs particularly critical for rare diseases, and at the heart of the healthcare debate.
The narrative approach revealed a diversity of substantive and procedural values with a common emphasis on scientific excellence, stakeholder involvement, independence, and transparency, with the greater purpose of providing quality care, serving the population, and optimal use of healthcare technology for best resource allocation. Complementary to the narrative approach, the reflective multicriteria approach allowed some elucidation of the substantive values of decision making, that is, the criteria that are considered along with their ethical underpinnings. It also revealed implicit criteria, such as Unmet needs and Disease severity, which are rooted in compassion (Reference Fotaki15) and were not mentioned in the narrative approach. Of note, multicriteria frameworks may be operationalized to also integrate several procedural values, including participative process for deliberation, clarity and transparency, consistency, and clear communication of reasoning and decision rationales (Reference Goetghebeur, Wagner and Khoury13;Reference Baltussen and Niessen16;Reference Thokala, Devlin and Marsh17), which was not explored in this study and calls for further research.
Examining the ethical aspects underpinning the criteria was deemed useful by participants to explore their institutional values. Except for Environmental impact and Nonmedical costs, most criteria were deemed to reflect the values of HTA agencies. By further revealing the founding values of HTA organizations, the analysis allowed exploration and identification of potential discrepancies between the substantive values articulated in their organizations’ mission and the processes in place to operationalize these values; indeed although most criteria were deemed to reflect institutions’ values, 70 percent of the criteria were reported by at least half of participants to be considered formally by their institutions.
Discussion revealed the need to incorporate criteria that pertain to the decision-maker remit (contextual criteria of EVIDEM) into the HTA process to support ultimate decision makers (e.g., Ministry of Health) in optimizing legitimacy of decisions. This is in line with the recent recommendation of Daniels for an “expanded HTA” to support accountable decision making (Reference Daniels, Porteny and Urritia9). Opportunity cost was noted as a key aspect, albeit most difficult to operationalize, that needs to be integrated in the decision process to tackle ethical dilemmas.
The need to clarify how economic aspects are considered was also noted in line with the long-term quest to address the controversy around cost considerations (Reference Sabin and Cochran1); indeed the cost-effectiveness ratio does not support a distinct consideration of the different concepts regarding the types of costs (e.g., cost of intervention, other medical costs, nonmedical costs) (Reference Neumann and Cohen3). The challenges associated with considering the patient perspective was also raised. This is emphasized by the lack of relevant data but must be advanced to truly integrate the patient perspective.
Furthermore, the notion of precedence was noted as an important aspect of the decision process, usually considered informally, but which may have a key impact on decisions. These considerations are fully relevant to assessing treatments for rare diseases, where the basic tension between the aims of alleviating patient suffering, serving the whole population equitably, and upholding healthcare sustainability (Reference Sabin and Cochran1) is particularly intense and can lead to ethical dilemmas. Thus, awareness of the underpinning ethical imperatives plays a key role when making trade-offs.
The quantitative values elicitation exercise highlighted the difficulty of balancing the imperatives to alleviate or prevent patient suffering, serve the whole population equitably, uphold healthcare system sustainability, and make decisions informed by evidence and context. The study suggest that such exercise can support individual reflection and clarify trade-offs by prompting the decision maker to express his/her thinking numerically and visualize it (along with the narratives). The visual representation can then be shared with others to facilitate the conversation to understand each other's reasoning and values. Approached this way, rather than transforming reflection into an algorithm, quantitative values elicitation serves as visualization of the ethical argumentation behind a decision and can assist in clarifying the decision rationale (a hallmark of a fair process) (Reference Daniels and van der Wilt6), and better communicating it. Not replacing the inherently narrative aspect of ethical argumentation, trade-off visualization needs to remain a support to this conversation.
Exploring trade-offs in general and with respect to priorities, such as rare diseases, was challenging, which highlights the need to further reflect on the best way to account for trade-offs and consider population priorities that can be shared and agreed upon with the society at large. Moving beyond the current implicit (“gut feeling”) approach to trade-offs, incorporation of Population priorities as a quantitative criterion is a possible way forward to make the reasoning behind the redistribution of weights across criteria and their ethical imperatives explicit, but such an approach requires further research and consultation.
Awareness of the trade-offs between the key ethical imperative, facilitated by comprehensive and reflective multicriteria, is a step toward tackling ethical dilemmas to foster legitimacy and support health system governance. The generic nature of EVIDEM can provide a structure to capture the diversity of individual values and perspectives (interpretive frames) (Reference Daniels and van der Wilt6) for each coverage decision, thus facilitating, in a pragmatic manner, collective deliberation on how ethical dilemmas might be resolved. Such shared reasoning is necessary to reach a decision that all stakeholders would deem fair and reasonable, because it would be based on a wide reflective equilibrium (Reference Daniels and van der Wilt6).
Combining a fair deliberative process with reflective multicriteria is well suited to operationalize agencies’ procedural and substantive values to foster accountable and reasonable decisions, a prerequisite for their acceptability and successful implementation. It is argued that the ethical framework A4R combined with multicriteria approaches rooted in the fundamental objectives of healthcare systems could provide a pragmatic way for HTA agencies to support policy makers in their quest for legitimacy (Reference Goetghebeur, Castro Jaramillo, Baltusen and Daniels19), and serve as a reminder that compassion is the ultimate legitimacy of all healthcare endeavors.
This exploratory study suggests that further research is warranted on how values on which HTA agencies are founded can be identified and operationalized in processes that support dealing with the basic tension at the heart of the healthcare debate, alleviating patient suffering, serving the whole population equitably, and upholding healthcare sustainability, while applying practical wisdom to make a legitimate decision adapted to the context.
CONFLICTS OF INTEREST
Authors Goetghebeur, Wagner, Samaha, O'Neil, and Badgley report received a research grant from Genzyme. Authors Abrishami, Sarria-Santamera, and Cleemput have nothing to disclose. Authors Castro-Jaramillo and Tringali received a travel grant from the EVIDEM Collaboration
