In a recent commentary piece in this journal (Reference Culyer1), Professor Culyer usefully highlighted many of the issues in economics surrounding costs and context in health-economic evaluation for health-technology assessment (HTA). Although making appropriate reference to both health economics and economics in general, Culyer neglected to mention that economics for HTA can encompass more than the extra-welfarist approach and that other economic value frameworks (EVFs) exist. Although his commentary helps to demystify the topic, still greater clarity and humility with regard to “economic” perspectives on valuation could contribute to improved HTA processes. Indeed, assessing the quality and relevance of EVF outputs as information for priority setting processes may become easier once the fundamental assumptions and value judgments related to EVFs are clarified.
This paper highlights two main alternatives to extra-welfarist economic value frameworks (EWEVFs)—the welfarist (WEVF) and the classical (CEVF)—and it describes how both can inform HTA decision making processes. Each of the three economic approaches here depends on particular sets of premises (in essence, “political” judgments) as to which sorts of value count and the extent to which those dimensions of value are covered. Hence, as Culyer and Jönsson note (Reference Culyer and Jönsson2, p. 2), these can be seen as vital for correctly judging the applicability or relevance of any given EVF.
Theoretical Understanding of Opportunity Cost
This paper supplements earlier studies by clarifying several factors related to economic evaluation for HTA. Conceptual clarity is especially important both when defining opportunity costs and when actually carrying out any corresponding economic evaluation, on account of the implicit or explicit assumptions made, the limitations and uncertainties surrounding the measurement instruments, and the challenges involved in estimating any form of “economic” efficiency. A clear, transparent approach is important also with regard to terminology: as Williams argued several decades ago, the role of economic evaluation in setting priorities for health technologies is easily oversold (Reference Williams3), and the relevance of this has been reaffirmed many times since (Reference Birch and Gafni4;Reference Coast5). Another important reason to strive for clarity lies in a shift witnessed in economic evaluation away from more welfarist views (Reference Drummond6, p. 64) and toward more narrowly focused extra-welfarist EVFs (Reference Drummond, Sculpher and Claxton7). Although the Culyer piece offers a textbook parable related to opportunity cost, it bears remembering that economists have utilized the concept at least as far back as Adam Smith's day (Reference Smith8, Book I, Chapter VI, p. 1). The term “opportunity cost” itself was coined by Green, with the thrust of his definition already involving “the opportunities foregone in accepting a certain line of action” (Reference Green9). Differences between schools of economic thought notwithstanding, Green's definition seems to have been reinforced—by, among others, both Alchian (Reference Coase10) and Buchanan, with the latter stating that “opportunity cost is the evaluation placed on the most highly valued of the rejected alternatives or opportunities” (Reference Buchanan11). Though there is fairly widespread agreement that economic evaluation is intended to inform HTA decision making processes, how this principle gets applied in practical analysis of opportunity costs will reflect both the policy problems facing decision makers and the research questions involved, along with the specific EVF chosen (Reference Culyer1;Reference McIntosh, Donaldson and Ryan12).
At the conceptual level, identifying opportunity costs entails a two-part approach: first, the value of the “new” technology at issue is estimated or defined; then, the estimate obtained is compared with the value placed on the class of all “practicable” alternative technologies, however specified. The first of the two evaluative components assigns a value to the given health technology relative to at least one other way of serving the same group. This valuation addresses not only the estimated additional resource requirements of the new technology, but also takes into account its effectiveness; that is, this first valuation reports or estimates a value for at least one of the outcomes produced by the health technology. The second component places a value on what would have to be forgone for to supply the resources needed for the chosen technology. The objective of any reputable economic evaluation is therefore to provide evidence on whether the technology's economic value (ascertained in the first component) outweighs the economic value of what is foregone (ascertained in the second component). The likely utility of economic evaluation for decision making purposes is markedly lower when either of the two evaluative components lacks plausibility. Accordingly, this paper focuses on clarifying the nature of economic evaluations' information inputs to priority setting processes. From this perspective, it outlines the orientation of three EVFs, which, to varying extents, can address policy problems and identify different forms of opportunity cost (Reference Culyer1). The aim is a critical review of economists' attempts to adopt and operationalize these concepts, bundled as they are with particular aspirations, conditions, and premises.
Concepts of Opportunity Cost in Practice
There are three main “economics”-based approaches to determining whether a given technology's economic value exceeds the value of any action forgone. Each type of EVF—the extra-welfarist, the welfarist, or the classical—imposes its own boundaries on how the valuation is undertaken. For each of the two components described above, the frameworks typically identify (or tacitly accepts) their own sources of “value” and/or metrics thereof. These differences between EVFs stem predominantly from what is deemed to be of value, though EVFs also diverge in how the valuation is conducted.
For a backdrop to examination of differences between EVFs, it is useful to outline the scope of investigations that are possible as part of the economic evaluation of health-care technologies. There are at least five distinct levels at which concepts of opportunity cost can be considered: (i) choices from among particular portfolios of public expenditure (Reference Sloan and Hsieh13); (ii) choices from among the technology portfolios that constitute the basket of publicly provided services (Reference Drummond, Sculpher and Claxton7); (iii) choices between treatments within the limits set for total disease-specific expenditure (Reference Tianviwat, Chongsuvivatwong and Birch14); (iv) choices between mutually exclusive treatments (Reference Edlin, McCabe and Hulme15); and (v) estimates of what may be forgone through using a specific input to the production process, or “resource opportunity cost” (Reference Brent16). The focus here is on level (ii), because the portfolio-of-technology level represents the most prevalent scope adopted by economic evaluations aimed at informing processes of health-care resource allocation (Reference Neumann, Ganiats and Russell17).
Differences between EVFs
The objective for extra-welfarist approaches is often characterized as being to “maximize health” (Reference Culyer18), where the matter of how “health” is defined can be considered very important because of proxying; typically in EWEVFs, rather than “health” per se being maximized, only an indicator of health is maximized. Under EWEVFs, “health” usually refers to the amalgam of (i) an indicator reflecting some dimensions of perceived health status with (ii) “health-state valuations” connected with that indicator (Reference Karimi and Brazier19). Both many of the indicators, and many of the valuations thereof, are typically engineered by health economists themselves. Although extra-welfarist approaches do not dictate a given maximand, most EWEVF applications center on maximizing a combination of precisely this sort of “social valuation” of states of health with estimates of length-of-life impacts, normally operationalized in the form of quality adjusted life-years (QALYs). Under EWEVFs, the first evaluative component's output, typically a cost-per-QALY estimate, is compared with the second “output,” which represents “opportunity cost” (an estimated mean cost per unit of health forgone through diverting resources from other activities). Thus, in principle, EWEVFs address whether total “health” will increase if the new technology is introduced, but do so with an implicit assumption that both the new technology and the activities from which resources are diverted are, as economic theory suggests, perfectly divisible with constant returns to scale. However, as noted by Drummond (Reference Drummond6) and illustrated by Birch and Donaldson (Reference Birch and Donaldson20), ascertaining the new technology's impact on efficiency (net impact on health) in a theoretically well-grounded manner requires avoiding such strict assumptions, which demands a mathematical-programing approach.
The aim with welfarist approaches to economic evaluation is to maximize “welfare,” where analysis is undertaken to identify the improvements in the aggregate welfare of individuals (Reference Sendi, Gafni and Birch21). Valuation using WEVFs is based on the utility individuals gain from how the available resources are used, inclusive of any welfare impacts arising from the way commodities or outcomes are distributed within the population in connection with different uses of resources (Reference Birch, Gafni, Rosen, Israeli and Shortell22). “Social welfare” or “well-being” can be defined in terms of total net willingness to pay (WTP) (Reference Nyborg23), with contingent valuation methods constituting the main source of valuations in WEVFs (Reference Shackley and Donaldson24). In more general terms, WEVF-based analysis compares the additional well-being produced by the new technology with that forgone through diversion of the required resources from elsewhere to support the new technology.
Finally, in classical approaches to economic evaluation, one of the central objectives is to supplement EWEVFs and WEVFs by accounting for preferences or values that are ascertainable only via deliberative methods. The label “classical” refers to the long history of valuation in economics before such developments as the marginal revolution (Reference Quade25). With CEVFs, the goal is to identify and assess, rather than to define and maximize, “health” or “well-being.” That is, in place of a formalized maximand, the targets in a classical approach (Reference Franklin26, p. 136) might involve satisficing (Reference Simon27) or sufficiency (Reference Ilias, Joanna and Ed28), in addition to interpreting, for example, some EWEVF- or WEVF-derived indicator of “economic” efficiency. Often, CEVFs operate with other, non-quantitative information too, and typically encompass deliberation (Reference Daniels and van der Wilt29). Perhaps their most important element is an attempt to avoid being constrained to focus on formal economic efficiency, that is, on the type of neo-classical economic efficiency which is the result of quantitative or mathematical analysis.
EVFs, Opportunity Cost, and the Two Components of Valuation
As the name “economic value framework” suggests, each EVF has its own approach to valuation embedded within it. Under EWEVFs, one frequent approach to judging what is forgone is to assume, both in principle and practice, that it is possible to quantify an opportunity cost and that this quantity is invariant to the size of the program being evaluated, that is, that there can be a fixed “cost per QALY” (Reference Claxton30). However, this is inconsistent with the economic notion of resource scarcity and the general finding that the marginal utility of a good or service decreases as consumption increases. When EVFs employ comparison to some fixed monetary valuation of opportunity cost, they tend to ignore factors such as the potential budgetary impact of the intervention and the “lumpiness” of health technologies (Reference Paulden31;Reference Hurley, Barer, Getzen and Stoddart32).
Although all three EVFs entail estimating cost and effect differences for a new technology relative to a comparator, the discussion above should render it clear that there may be little deeper commonality in how EVFs assign value to alternative health technologies that might be displaced. The onus is generally on the user of the research to identify the possible implications of the chosen value system for the decision making process it is purported to serve (Reference Mooney and Lee33). The discussion below attempts to make the relevant implications clearer for each of the three main EVFs.
Valuation and Opportunity Cost in EWEVFs
Under EWEVFs, the first evaluative component in defining opportunity cost is generally based on cost-effectiveness analysis, which yields an estimate of the mean cost-per-unit health benefit produced by the chosen intervention—that is, an incremental cost-effectiveness ratio (ICER). In EWEVFs, this ratio, an estimate of the inverse of the mean rate of return on the additional investment required to fund the technology, is typically employed in an economic-efficiency metric entailing comparison with some predetermined benchmark ICER, that is, some cost-effectiveness-ratio threshold (CERT) (Reference Culyer34). The latter is usually exogenous to the study at hand. Only rarely under EWEVFs do the activities displaced by the additional investment of resources in the technology get identified, or be valued, on a case-by-case basis. Although some CERTs involve estimates from econometric analysis of possible relationships between current resource use and health-related outputs (Reference Thokala, Ochalek, Leech and Tong35;Reference Claxton, Martin and Soares36), they may also simply represent an arbitrary figure or diktat (Reference Birch and Gafni37). Indeed, CERTs will generally fail to fully reflect the actual displacement resulting from the technology's adoption (Reference Caro38). Many researchers continue to propose CERTs, of various types, despite evidence suggesting that thresholds are merely an economic abstraction and that a single appropriate CERT is likely to remain elusive in most contexts (Reference Cleemput, Neyt and Thiry39).
WEVF-Related Valuation and Opportunity Cost
Under WEVFs, analysis focuses on individuals' preferences and technologies are evaluated for their impacts on “well-being” (Reference Birch and Donaldson20). In some of these frameworks, the two evaluative components are brought together in a single model for analysis of portfolio choice through mathematical optimization. By incorporating resource constraints into the model explicitly, thereby focusing attention on the well-being generated from the entire resource budget as opposed to a single program's share of that budget, the approach addresses opportunity cost considerations directly without requiring the separate valuation of the foregone alternatives that is typical under EWEVFs (Reference Birch, Gafni, Ethgen and Staginnus40). Hence, the emphasis in WEVFs is on comparing across the well-being generated by various combinations (or portfolios) of “health technologies” that the available resources can sustain, and on determining which combinations could improve “welfare.” In addition, the approach can accommodate any other concrete constraints on preferences, in line with policy considerations related to equity, need, and so on. (Reference Birch, Gafni, Ethgen and Staginnus40). It is also important to note here that, in practice, WEVF utilizes WTP estimates which typically rely on methods such as contingent valuation to compare WTP between the new technology in aggregate and whatever must be forgone (Reference Gafni41).
Valuation and Opportunity Cost in Classical Economic Approaches
CEVFs can be viewed as a reaction to various limitations of EWEVFs and WEVFs in practice, especially as the latter are designed to “maximize” via an objective function of one type or another. CEVFs represent an alternative approach, one that need not focus on a single maximand (as EWEVFs typically do) or on a single source of preferences (as is typical under WEVFs, the source being individuals) yet CEVFs can still be in line with conventional interpretations of opportunity cost (Reference Coast5).
How CEVF Approaches can Help in HTA
In light of the above, CEVFs are proposed as an alternative that affords wider scope than either “health maximization” under EWEVFs or “maximization of economic welfare” under WEVFs, as they allow for qualitative use of preferences from groups of individuals, or directly from other stakeholders. Rather than rejecting use of the other EVFs, the CEVF approach supplements them with further information or deliberative analysis, such as incorporating community values (Reference Mooney42) canvassed through various evidence-gathering processes (Reference Vickers43–Reference Garrison, Neumann and Willke45).
A CEVF approach can help inform HTA in three main ways. First, CEVFs can add information to evidence provided by EWEVF and WEVF approaches on the relative efficiency with which “health” and “welfare” are produced, respectively. Although WEVFs may include strong evidence about budget or resource impacts, additional, related information (with either a short or a long time horizon) can still be produced or utilized within a CEVF (Reference Mauskopf46). Second, CEVFs can identify any qualifications or caveats to the EWEVF or WEVF findings, aiming to ensure that the information they provide is interpreted correctly, through an appropriate appraisal of their quality. Although such appraisal is already addressed by many existing HTA processes, it could have greater value due to being integral to a CEVF approach, in line with an iterative, classical vision of valuation (Reference Franklin26). The third main advantage would be that CEVFs can provide fuller awareness of the nature of the research question and its connection with the policy problem, as well as of the types and levels of uncertainty and relevance carried by information from other EVFs (Reference Deaton and Cartwright47;Reference Manski48). One major contribution that CEVFs can make to HTA processes is to force more clarity into the terminology surrounding EVFs. This point will be returned to below.
CEVFs allow inclusion of dimensions of value that might not be measurable in the commensurate units “required” by EWEVFs or WEVFs (Reference Oortwijn, Sampietro-Colom and Habens49). Because they can take into account informal analysis during an iterative process of deliberation, CEVFs could prove highly relevant for decision makers (Reference Schultze, Hinrichs and Taylor50). This might involve, for instance, (a) confirming, doubting, or disproving the suitability of standard health-economic outcome metrics for the technology in question, partly through questioning the assumptions underlying information outputs from other EVFs, and (b) establishing additional objectives or outcome measurements relevant for the technology in question (Reference Enthoven and Tucker51, p. 149). For item (a), deliberative analysis may assist in identifying any need to supplement other EVFs, because it is probable that no single overriding “efficiency” principle meets all the desiderata for allocation, and there may be good reasons to consider multiple prioritization principles (Reference Daniels and van der Wilt29). For instance, some opportunity costs may not be quantifiable (Reference Marsh, Sculpher, Caro and Tervonen52) and might lend themselves only to deliberation, as in the case of rights-based deontological or paternalistic considerations (Reference Culyer and Bombard53). In addition, with regard to item (b), for some technologies there may be little pertinent quantitative information available from formal analysis, and stakeholders may hold diverse, conflicting views (Reference Williams and Williams54). The appraisal process may embody a range of considerations that might not all be well-defined prior to, or even during, economic evaluation. There are numerous situations in which deliberative analysis via CEVFs may provide a useful extension that improves on purely formal analysis, and a variety of evidentiary inputs may be used, as necessary, on a case-by-case basis (Reference Culyer1;55).
In general, although analytic endeavors within EWEVFs or WEVFs can reveal some of the implications of particular choices (Reference Mooney and Lee33), CEVFs may add a platform that stimulates discussion of more communitarian values (e.g., (Reference Sandel56;Reference Mooney57)). With CEVFs, the aim is what some have called “higher-level efficiency,” rather than efficiency in the more neo-classical sense found in the more formal approaches of EWHEE and WHEE (Reference Hitch and McKean58, p. 125).
Discussion
Each mode of economic thinking outlined in this paper can offer useful information for priority setting processes, even though each EVF involves its own particular aims, assumptions, and value judgments. Whichever EVF is applied, evaluating opportunity cost requires some valuation of what is given up (Reference Wildavsky59); hence, the aim here is not to denigrate or promote any particular mode of economic evaluation but to promote solid awareness of the information that each can provide. In all cases, it should be acknowledged that economic approaches to assessing opportunity costs are information-intensive in their input requirements and that their use often suffers from a lack of appropriate information (Reference Mooney60), especially as pathways to health are often quite complex (Reference Birch61). One should also bear in mind that any method which gives consistent or accountable answers in a systematic manner is unlikely to yield truly comprehensive evaluation (Reference Stone62). There are many circumstances wherein measurements fail to cover relevant aspects of the changes in “states of health” (Reference Hurley, Barer, Getzen and Stoddart32;Reference Brazier, Rowen, Lloyd and Karimi63) or do not capture changes in capabilities or in patient-reported experiences, not to mention the fact that “social valuations” of such changes in the health status do not fully capture society's values (Reference Coast5). On account of the measurement issues surrounding WTP, there may be many situations in which no valid and reliable methods of operationalizing WEVFs exist (Reference Gafni41;Reference Culyer and Chalkidou64).
Problems with the EVF Lexicon
Although choice processes for allocating health-care resources should lead to transparent mechanisms for valuation of the various options and their opportunity costs (Reference Mooney, Russell and Weir65, p. 138), terminology can make economic evaluation more opaque. This is evident from the declining use of terminology relating to intangibles and incommensurability, which could be seen as arrogant in a sub-discipline that often preaches humility. On account of space restrictions, the discussion here focuses on the terms “cost,” “threshold,” “decision rule,” and “value for money.”
“Cost” has multiple meanings in both lay and specialist use, as Culyer noted when deeming it naïve to employ the term “cost” for undesirable attributes (Reference Culyer1). An alternative interpretation to that offered by Culyer is to take the undesirable attributes of an intervention as also representing a cost. Of course, at the level of valuing what may be forgone through using a specific input to the production process, or “resource opportunity cost,” that is, at the level of building the pool from Alchian's and Culyer's examples, then “undesirable attributes” should not be referred to as costs. On the other hand, the use of the term “cost” for an undesirable attribute, a harm, or a negative benefit, could legitimately be used to refer to its part in an estimate of higher-level opportunity cost, that is, when assessing the value of the pool per se. Indeed, at the portfolio-of-technology level, such undesirable attributes can be seen as an essential component of any EVF. Undesirable attributes are important when forming a valuation; Alchian expresses it thus: “The decision maker must choose among events that are amalgams of goods and bads” (Reference Coase10). Therefore, in addition to the things forgone, such as the financial costs and the resources tied up, other aspects of the value forgone, the “costs” in terms of harms to health will also have a legitimate place in economic evaluations' definitions of (opportunity) costs (Reference O'Donnell66). In practice, economic evaluations do typically include undesirable attributes in their analysis; for instance, EWEVFs do tend to utilize something akin to Alchian's amalgam approach when they promote a metric expressing the estimated cost divided by the estimated incremental overall population-“health impact.” For the purposes of HTA, it seems reasonable to suggest that any sound economic evaluation involves taking both pros and cons into account: focusing on both the undesirable and the desirable attributes of technology, in line with the foundations of technology assessment (Reference Daddario67). Although, obviously, pain and suffering need not involve resources per se, the principle of opportunity cost encompasses the benefit forgone, so any robust measurement of higher-level opportunity cost should also take the “cost,” in terms of related pain and suffering, into account.
Some extra-welfarist economists and even some HTA practitioners take the perspective that “thresholds” can and should be quantified. However, economizing in line with these assumptions may be less intuitive for others involved in prioritization processes and seem rather perfunctory with respect to “societal values” (Reference Macfie68;Reference Marseille, Larson and Kazi69). As is noted above, defining opportunity cost as a single threshold estimate can be seen as a typical economic abstraction. Although economic evaluation must always operate at some level of abstraction in practice, the fairy tale of a single threshold (CERT), or threshold range, can be regarded as unhelpful. As no such one-size-fits-all threshold exists in reality, even within a well-bounded single jurisdiction, employing the term “threshold” seems to oversell EWEVFs. The problematic terminology is compounded by the use of connected phrasings such as “decision rules” and “value for money.” For instance, the real-world applicability of so-called decision rules of EWEVFs is crucially reliant on the framework's inherent value judgments and assumptions. Indeed, these “rules” are typically valid only within the confines of the EWEVF in question, and there is a danger that the term “decision rules” could be construed to carry a similar meaning beyond this arcane hypothetical setting. Furthermore, claims of ICERs revealing “value for money” seem quite arrogant, in that EWEVFs often offer only a highly abstracted indicator of value. Although the concise term “value for money” may be much easier to sell to HTA decision makers than, for example, “estimated mean valuation of estimated change in mean health status divided by the estimated change in mean health-care costs,” the former loses too much in precision; it seems much less honest. Because loose language could result in dire consequences of economic evaluation being oversold to the HTA community, it should be avoided at all costs.
Conclusions
Rather than economists holding a uniform, all-encompassing view, there are three main approaches to economic thinking for HTA, accompanied by a multitude of ways to implement each of these. Instead of a single notion of economics embodied by one EVF, the study found EWEVFs, WEVFs, and CEVFs, each with the corresponding problems and potential. Therefore, all approaches to economic evaluation should be checked for quality and relevance before being used to inform prioritization processes. Applying more precise vocabulary, coupled with greater understanding of the limits to analysis of any kind, should help decision makers engage in appropriate deliberation and interpretation in their HTA endeavors. The ways in which notions of opportunity cost are translated into practice and interpreted are likely to have great importance, not only for priority setting but also for the long-term health and sustainability of health-care systems.
In a recent commentary piece in this journal (Reference Culyer1), Professor Culyer usefully highlighted many of the issues in economics surrounding costs and context in health-economic evaluation for health-technology assessment (HTA). Although making appropriate reference to both health economics and economics in general, Culyer neglected to mention that economics for HTA can encompass more than the extra-welfarist approach and that other economic value frameworks (EVFs) exist. Although his commentary helps to demystify the topic, still greater clarity and humility with regard to “economic” perspectives on valuation could contribute to improved HTA processes. Indeed, assessing the quality and relevance of EVF outputs as information for priority setting processes may become easier once the fundamental assumptions and value judgments related to EVFs are clarified.
This paper highlights two main alternatives to extra-welfarist economic value frameworks (EWEVFs)—the welfarist (WEVF) and the classical (CEVF)—and it describes how both can inform HTA decision making processes. Each of the three economic approaches here depends on particular sets of premises (in essence, “political” judgments) as to which sorts of value count and the extent to which those dimensions of value are covered. Hence, as Culyer and Jönsson note (Reference Culyer and Jönsson2, p. 2), these can be seen as vital for correctly judging the applicability or relevance of any given EVF.
Theoretical Understanding of Opportunity Cost
This paper supplements earlier studies by clarifying several factors related to economic evaluation for HTA. Conceptual clarity is especially important both when defining opportunity costs and when actually carrying out any corresponding economic evaluation, on account of the implicit or explicit assumptions made, the limitations and uncertainties surrounding the measurement instruments, and the challenges involved in estimating any form of “economic” efficiency. A clear, transparent approach is important also with regard to terminology: as Williams argued several decades ago, the role of economic evaluation in setting priorities for health technologies is easily oversold (Reference Williams3), and the relevance of this has been reaffirmed many times since (Reference Birch and Gafni4;Reference Coast5). Another important reason to strive for clarity lies in a shift witnessed in economic evaluation away from more welfarist views (Reference Drummond6, p. 64) and toward more narrowly focused extra-welfarist EVFs (Reference Drummond, Sculpher and Claxton7). Although the Culyer piece offers a textbook parable related to opportunity cost, it bears remembering that economists have utilized the concept at least as far back as Adam Smith's day (Reference Smith8, Book I, Chapter VI, p. 1). The term “opportunity cost” itself was coined by Green, with the thrust of his definition already involving “the opportunities foregone in accepting a certain line of action” (Reference Green9). Differences between schools of economic thought notwithstanding, Green's definition seems to have been reinforced—by, among others, both Alchian (Reference Coase10) and Buchanan, with the latter stating that “opportunity cost is the evaluation placed on the most highly valued of the rejected alternatives or opportunities” (Reference Buchanan11). Though there is fairly widespread agreement that economic evaluation is intended to inform HTA decision making processes, how this principle gets applied in practical analysis of opportunity costs will reflect both the policy problems facing decision makers and the research questions involved, along with the specific EVF chosen (Reference Culyer1;Reference McIntosh, Donaldson and Ryan12).
At the conceptual level, identifying opportunity costs entails a two-part approach: first, the value of the “new” technology at issue is estimated or defined; then, the estimate obtained is compared with the value placed on the class of all “practicable” alternative technologies, however specified. The first of the two evaluative components assigns a value to the given health technology relative to at least one other way of serving the same group. This valuation addresses not only the estimated additional resource requirements of the new technology, but also takes into account its effectiveness; that is, this first valuation reports or estimates a value for at least one of the outcomes produced by the health technology. The second component places a value on what would have to be forgone for to supply the resources needed for the chosen technology. The objective of any reputable economic evaluation is therefore to provide evidence on whether the technology's economic value (ascertained in the first component) outweighs the economic value of what is foregone (ascertained in the second component). The likely utility of economic evaluation for decision making purposes is markedly lower when either of the two evaluative components lacks plausibility. Accordingly, this paper focuses on clarifying the nature of economic evaluations' information inputs to priority setting processes. From this perspective, it outlines the orientation of three EVFs, which, to varying extents, can address policy problems and identify different forms of opportunity cost (Reference Culyer1). The aim is a critical review of economists' attempts to adopt and operationalize these concepts, bundled as they are with particular aspirations, conditions, and premises.
Concepts of Opportunity Cost in Practice
There are three main “economics”-based approaches to determining whether a given technology's economic value exceeds the value of any action forgone. Each type of EVF—the extra-welfarist, the welfarist, or the classical—imposes its own boundaries on how the valuation is undertaken. For each of the two components described above, the frameworks typically identify (or tacitly accepts) their own sources of “value” and/or metrics thereof. These differences between EVFs stem predominantly from what is deemed to be of value, though EVFs also diverge in how the valuation is conducted.
For a backdrop to examination of differences between EVFs, it is useful to outline the scope of investigations that are possible as part of the economic evaluation of health-care technologies. There are at least five distinct levels at which concepts of opportunity cost can be considered: (i) choices from among particular portfolios of public expenditure (Reference Sloan and Hsieh13); (ii) choices from among the technology portfolios that constitute the basket of publicly provided services (Reference Drummond, Sculpher and Claxton7); (iii) choices between treatments within the limits set for total disease-specific expenditure (Reference Tianviwat, Chongsuvivatwong and Birch14); (iv) choices between mutually exclusive treatments (Reference Edlin, McCabe and Hulme15); and (v) estimates of what may be forgone through using a specific input to the production process, or “resource opportunity cost” (Reference Brent16). The focus here is on level (ii), because the portfolio-of-technology level represents the most prevalent scope adopted by economic evaluations aimed at informing processes of health-care resource allocation (Reference Neumann, Ganiats and Russell17).
Differences between EVFs
The objective for extra-welfarist approaches is often characterized as being to “maximize health” (Reference Culyer18), where the matter of how “health” is defined can be considered very important because of proxying; typically in EWEVFs, rather than “health” per se being maximized, only an indicator of health is maximized. Under EWEVFs, “health” usually refers to the amalgam of (i) an indicator reflecting some dimensions of perceived health status with (ii) “health-state valuations” connected with that indicator (Reference Karimi and Brazier19). Both many of the indicators, and many of the valuations thereof, are typically engineered by health economists themselves. Although extra-welfarist approaches do not dictate a given maximand, most EWEVF applications center on maximizing a combination of precisely this sort of “social valuation” of states of health with estimates of length-of-life impacts, normally operationalized in the form of quality adjusted life-years (QALYs). Under EWEVFs, the first evaluative component's output, typically a cost-per-QALY estimate, is compared with the second “output,” which represents “opportunity cost” (an estimated mean cost per unit of health forgone through diverting resources from other activities). Thus, in principle, EWEVFs address whether total “health” will increase if the new technology is introduced, but do so with an implicit assumption that both the new technology and the activities from which resources are diverted are, as economic theory suggests, perfectly divisible with constant returns to scale. However, as noted by Drummond (Reference Drummond6) and illustrated by Birch and Donaldson (Reference Birch and Donaldson20), ascertaining the new technology's impact on efficiency (net impact on health) in a theoretically well-grounded manner requires avoiding such strict assumptions, which demands a mathematical-programing approach.
The aim with welfarist approaches to economic evaluation is to maximize “welfare,” where analysis is undertaken to identify the improvements in the aggregate welfare of individuals (Reference Sendi, Gafni and Birch21). Valuation using WEVFs is based on the utility individuals gain from how the available resources are used, inclusive of any welfare impacts arising from the way commodities or outcomes are distributed within the population in connection with different uses of resources (Reference Birch, Gafni, Rosen, Israeli and Shortell22). “Social welfare” or “well-being” can be defined in terms of total net willingness to pay (WTP) (Reference Nyborg23), with contingent valuation methods constituting the main source of valuations in WEVFs (Reference Shackley and Donaldson24). In more general terms, WEVF-based analysis compares the additional well-being produced by the new technology with that forgone through diversion of the required resources from elsewhere to support the new technology.
Finally, in classical approaches to economic evaluation, one of the central objectives is to supplement EWEVFs and WEVFs by accounting for preferences or values that are ascertainable only via deliberative methods. The label “classical” refers to the long history of valuation in economics before such developments as the marginal revolution (Reference Quade25). With CEVFs, the goal is to identify and assess, rather than to define and maximize, “health” or “well-being.” That is, in place of a formalized maximand, the targets in a classical approach (Reference Franklin26, p. 136) might involve satisficing (Reference Simon27) or sufficiency (Reference Ilias, Joanna and Ed28), in addition to interpreting, for example, some EWEVF- or WEVF-derived indicator of “economic” efficiency. Often, CEVFs operate with other, non-quantitative information too, and typically encompass deliberation (Reference Daniels and van der Wilt29). Perhaps their most important element is an attempt to avoid being constrained to focus on formal economic efficiency, that is, on the type of neo-classical economic efficiency which is the result of quantitative or mathematical analysis.
EVFs, Opportunity Cost, and the Two Components of Valuation
As the name “economic value framework” suggests, each EVF has its own approach to valuation embedded within it. Under EWEVFs, one frequent approach to judging what is forgone is to assume, both in principle and practice, that it is possible to quantify an opportunity cost and that this quantity is invariant to the size of the program being evaluated, that is, that there can be a fixed “cost per QALY” (Reference Claxton30). However, this is inconsistent with the economic notion of resource scarcity and the general finding that the marginal utility of a good or service decreases as consumption increases. When EVFs employ comparison to some fixed monetary valuation of opportunity cost, they tend to ignore factors such as the potential budgetary impact of the intervention and the “lumpiness” of health technologies (Reference Paulden31;Reference Hurley, Barer, Getzen and Stoddart32).
Although all three EVFs entail estimating cost and effect differences for a new technology relative to a comparator, the discussion above should render it clear that there may be little deeper commonality in how EVFs assign value to alternative health technologies that might be displaced. The onus is generally on the user of the research to identify the possible implications of the chosen value system for the decision making process it is purported to serve (Reference Mooney and Lee33). The discussion below attempts to make the relevant implications clearer for each of the three main EVFs.
Valuation and Opportunity Cost in EWEVFs
Under EWEVFs, the first evaluative component in defining opportunity cost is generally based on cost-effectiveness analysis, which yields an estimate of the mean cost-per-unit health benefit produced by the chosen intervention—that is, an incremental cost-effectiveness ratio (ICER). In EWEVFs, this ratio, an estimate of the inverse of the mean rate of return on the additional investment required to fund the technology, is typically employed in an economic-efficiency metric entailing comparison with some predetermined benchmark ICER, that is, some cost-effectiveness-ratio threshold (CERT) (Reference Culyer34). The latter is usually exogenous to the study at hand. Only rarely under EWEVFs do the activities displaced by the additional investment of resources in the technology get identified, or be valued, on a case-by-case basis. Although some CERTs involve estimates from econometric analysis of possible relationships between current resource use and health-related outputs (Reference Thokala, Ochalek, Leech and Tong35;Reference Claxton, Martin and Soares36), they may also simply represent an arbitrary figure or diktat (Reference Birch and Gafni37). Indeed, CERTs will generally fail to fully reflect the actual displacement resulting from the technology's adoption (Reference Caro38). Many researchers continue to propose CERTs, of various types, despite evidence suggesting that thresholds are merely an economic abstraction and that a single appropriate CERT is likely to remain elusive in most contexts (Reference Cleemput, Neyt and Thiry39).
WEVF-Related Valuation and Opportunity Cost
Under WEVFs, analysis focuses on individuals' preferences and technologies are evaluated for their impacts on “well-being” (Reference Birch and Donaldson20). In some of these frameworks, the two evaluative components are brought together in a single model for analysis of portfolio choice through mathematical optimization. By incorporating resource constraints into the model explicitly, thereby focusing attention on the well-being generated from the entire resource budget as opposed to a single program's share of that budget, the approach addresses opportunity cost considerations directly without requiring the separate valuation of the foregone alternatives that is typical under EWEVFs (Reference Birch, Gafni, Ethgen and Staginnus40). Hence, the emphasis in WEVFs is on comparing across the well-being generated by various combinations (or portfolios) of “health technologies” that the available resources can sustain, and on determining which combinations could improve “welfare.” In addition, the approach can accommodate any other concrete constraints on preferences, in line with policy considerations related to equity, need, and so on. (Reference Birch, Gafni, Ethgen and Staginnus40). It is also important to note here that, in practice, WEVF utilizes WTP estimates which typically rely on methods such as contingent valuation to compare WTP between the new technology in aggregate and whatever must be forgone (Reference Gafni41).
Valuation and Opportunity Cost in Classical Economic Approaches
CEVFs can be viewed as a reaction to various limitations of EWEVFs and WEVFs in practice, especially as the latter are designed to “maximize” via an objective function of one type or another. CEVFs represent an alternative approach, one that need not focus on a single maximand (as EWEVFs typically do) or on a single source of preferences (as is typical under WEVFs, the source being individuals) yet CEVFs can still be in line with conventional interpretations of opportunity cost (Reference Coast5).
How CEVF Approaches can Help in HTA
In light of the above, CEVFs are proposed as an alternative that affords wider scope than either “health maximization” under EWEVFs or “maximization of economic welfare” under WEVFs, as they allow for qualitative use of preferences from groups of individuals, or directly from other stakeholders. Rather than rejecting use of the other EVFs, the CEVF approach supplements them with further information or deliberative analysis, such as incorporating community values (Reference Mooney42) canvassed through various evidence-gathering processes (Reference Vickers43–Reference Garrison, Neumann and Willke45).
A CEVF approach can help inform HTA in three main ways. First, CEVFs can add information to evidence provided by EWEVF and WEVF approaches on the relative efficiency with which “health” and “welfare” are produced, respectively. Although WEVFs may include strong evidence about budget or resource impacts, additional, related information (with either a short or a long time horizon) can still be produced or utilized within a CEVF (Reference Mauskopf46). Second, CEVFs can identify any qualifications or caveats to the EWEVF or WEVF findings, aiming to ensure that the information they provide is interpreted correctly, through an appropriate appraisal of their quality. Although such appraisal is already addressed by many existing HTA processes, it could have greater value due to being integral to a CEVF approach, in line with an iterative, classical vision of valuation (Reference Franklin26). The third main advantage would be that CEVFs can provide fuller awareness of the nature of the research question and its connection with the policy problem, as well as of the types and levels of uncertainty and relevance carried by information from other EVFs (Reference Deaton and Cartwright47;Reference Manski48). One major contribution that CEVFs can make to HTA processes is to force more clarity into the terminology surrounding EVFs. This point will be returned to below.
CEVFs allow inclusion of dimensions of value that might not be measurable in the commensurate units “required” by EWEVFs or WEVFs (Reference Oortwijn, Sampietro-Colom and Habens49). Because they can take into account informal analysis during an iterative process of deliberation, CEVFs could prove highly relevant for decision makers (Reference Schultze, Hinrichs and Taylor50). This might involve, for instance, (a) confirming, doubting, or disproving the suitability of standard health-economic outcome metrics for the technology in question, partly through questioning the assumptions underlying information outputs from other EVFs, and (b) establishing additional objectives or outcome measurements relevant for the technology in question (Reference Enthoven and Tucker51, p. 149). For item (a), deliberative analysis may assist in identifying any need to supplement other EVFs, because it is probable that no single overriding “efficiency” principle meets all the desiderata for allocation, and there may be good reasons to consider multiple prioritization principles (Reference Daniels and van der Wilt29). For instance, some opportunity costs may not be quantifiable (Reference Marsh, Sculpher, Caro and Tervonen52) and might lend themselves only to deliberation, as in the case of rights-based deontological or paternalistic considerations (Reference Culyer and Bombard53). In addition, with regard to item (b), for some technologies there may be little pertinent quantitative information available from formal analysis, and stakeholders may hold diverse, conflicting views (Reference Williams and Williams54). The appraisal process may embody a range of considerations that might not all be well-defined prior to, or even during, economic evaluation. There are numerous situations in which deliberative analysis via CEVFs may provide a useful extension that improves on purely formal analysis, and a variety of evidentiary inputs may be used, as necessary, on a case-by-case basis (Reference Culyer1;55).
In general, although analytic endeavors within EWEVFs or WEVFs can reveal some of the implications of particular choices (Reference Mooney and Lee33), CEVFs may add a platform that stimulates discussion of more communitarian values (e.g., (Reference Sandel56;Reference Mooney57)). With CEVFs, the aim is what some have called “higher-level efficiency,” rather than efficiency in the more neo-classical sense found in the more formal approaches of EWHEE and WHEE (Reference Hitch and McKean58, p. 125).
Discussion
Each mode of economic thinking outlined in this paper can offer useful information for priority setting processes, even though each EVF involves its own particular aims, assumptions, and value judgments. Whichever EVF is applied, evaluating opportunity cost requires some valuation of what is given up (Reference Wildavsky59); hence, the aim here is not to denigrate or promote any particular mode of economic evaluation but to promote solid awareness of the information that each can provide. In all cases, it should be acknowledged that economic approaches to assessing opportunity costs are information-intensive in their input requirements and that their use often suffers from a lack of appropriate information (Reference Mooney60), especially as pathways to health are often quite complex (Reference Birch61). One should also bear in mind that any method which gives consistent or accountable answers in a systematic manner is unlikely to yield truly comprehensive evaluation (Reference Stone62). There are many circumstances wherein measurements fail to cover relevant aspects of the changes in “states of health” (Reference Hurley, Barer, Getzen and Stoddart32;Reference Brazier, Rowen, Lloyd and Karimi63) or do not capture changes in capabilities or in patient-reported experiences, not to mention the fact that “social valuations” of such changes in the health status do not fully capture society's values (Reference Coast5). On account of the measurement issues surrounding WTP, there may be many situations in which no valid and reliable methods of operationalizing WEVFs exist (Reference Gafni41;Reference Culyer and Chalkidou64).
Problems with the EVF Lexicon
Although choice processes for allocating health-care resources should lead to transparent mechanisms for valuation of the various options and their opportunity costs (Reference Mooney, Russell and Weir65, p. 138), terminology can make economic evaluation more opaque. This is evident from the declining use of terminology relating to intangibles and incommensurability, which could be seen as arrogant in a sub-discipline that often preaches humility. On account of space restrictions, the discussion here focuses on the terms “cost,” “threshold,” “decision rule,” and “value for money.”
“Cost” has multiple meanings in both lay and specialist use, as Culyer noted when deeming it naïve to employ the term “cost” for undesirable attributes (Reference Culyer1). An alternative interpretation to that offered by Culyer is to take the undesirable attributes of an intervention as also representing a cost. Of course, at the level of valuing what may be forgone through using a specific input to the production process, or “resource opportunity cost,” that is, at the level of building the pool from Alchian's and Culyer's examples, then “undesirable attributes” should not be referred to as costs. On the other hand, the use of the term “cost” for an undesirable attribute, a harm, or a negative benefit, could legitimately be used to refer to its part in an estimate of higher-level opportunity cost, that is, when assessing the value of the pool per se. Indeed, at the portfolio-of-technology level, such undesirable attributes can be seen as an essential component of any EVF. Undesirable attributes are important when forming a valuation; Alchian expresses it thus: “The decision maker must choose among events that are amalgams of goods and bads” (Reference Coase10). Therefore, in addition to the things forgone, such as the financial costs and the resources tied up, other aspects of the value forgone, the “costs” in terms of harms to health will also have a legitimate place in economic evaluations' definitions of (opportunity) costs (Reference O'Donnell66). In practice, economic evaluations do typically include undesirable attributes in their analysis; for instance, EWEVFs do tend to utilize something akin to Alchian's amalgam approach when they promote a metric expressing the estimated cost divided by the estimated incremental overall population-“health impact.” For the purposes of HTA, it seems reasonable to suggest that any sound economic evaluation involves taking both pros and cons into account: focusing on both the undesirable and the desirable attributes of technology, in line with the foundations of technology assessment (Reference Daddario67). Although, obviously, pain and suffering need not involve resources per se, the principle of opportunity cost encompasses the benefit forgone, so any robust measurement of higher-level opportunity cost should also take the “cost,” in terms of related pain and suffering, into account.
Some extra-welfarist economists and even some HTA practitioners take the perspective that “thresholds” can and should be quantified. However, economizing in line with these assumptions may be less intuitive for others involved in prioritization processes and seem rather perfunctory with respect to “societal values” (Reference Macfie68;Reference Marseille, Larson and Kazi69). As is noted above, defining opportunity cost as a single threshold estimate can be seen as a typical economic abstraction. Although economic evaluation must always operate at some level of abstraction in practice, the fairy tale of a single threshold (CERT), or threshold range, can be regarded as unhelpful. As no such one-size-fits-all threshold exists in reality, even within a well-bounded single jurisdiction, employing the term “threshold” seems to oversell EWEVFs. The problematic terminology is compounded by the use of connected phrasings such as “decision rules” and “value for money.” For instance, the real-world applicability of so-called decision rules of EWEVFs is crucially reliant on the framework's inherent value judgments and assumptions. Indeed, these “rules” are typically valid only within the confines of the EWEVF in question, and there is a danger that the term “decision rules” could be construed to carry a similar meaning beyond this arcane hypothetical setting. Furthermore, claims of ICERs revealing “value for money” seem quite arrogant, in that EWEVFs often offer only a highly abstracted indicator of value. Although the concise term “value for money” may be much easier to sell to HTA decision makers than, for example, “estimated mean valuation of estimated change in mean health status divided by the estimated change in mean health-care costs,” the former loses too much in precision; it seems much less honest. Because loose language could result in dire consequences of economic evaluation being oversold to the HTA community, it should be avoided at all costs.
Conclusions
Rather than economists holding a uniform, all-encompassing view, there are three main approaches to economic thinking for HTA, accompanied by a multitude of ways to implement each of these. Instead of a single notion of economics embodied by one EVF, the study found EWEVFs, WEVFs, and CEVFs, each with the corresponding problems and potential. Therefore, all approaches to economic evaluation should be checked for quality and relevance before being used to inform prioritization processes. Applying more precise vocabulary, coupled with greater understanding of the limits to analysis of any kind, should help decision makers engage in appropriate deliberation and interpretation in their HTA endeavors. The ways in which notions of opportunity cost are translated into practice and interpreted are likely to have great importance, not only for priority setting but also for the long-term health and sustainability of health-care systems.
Acknowledgments
The author is indebted to Professor Steve Birch for his assistance with useful content for earlier drafts of the manuscript, along with valuable discussions. Professor Pekka Rissanen also deserves special thanks for helping provide the time and space necessary for undertaking the study. Of course, any mistakes that remain are entirely the responsibility of the author.
Financial Support
This work was supported by an unconditional grant from the Yrjö Jahnsson Foundation (grant number 6572), with the funder having no other role in the study itself, in interpretation of the results, or in the writing of the manuscript.
Conflict of Interest
The author has nothing to disclose.