The paper by Drummond et al. in this issue of the Journal discusses how the principles that have been proposed in previous papers by these authors might be used as the basis for benchmarking health technology assessment (HTA) organizations. This raises a number of important issues, some acknowledged and discussed by the authors, and some not. The commentary by Sampietro provides an analysis of many of these. Two issues strike me as fundamental in this debate: the need for an agreed and objective approach to describing health system decision-making systems; and the need for an agreed and objective approach to the assessment of the quality of HTA reports to support specific decisions.

HTA-based health system decision making comprises two elements: assessment—the process of collecting, synthesizing, assessing, and interpreting all relevant available evidence on effectiveness, costs, system impact and acceptability in a systematic, unbiased and transparent manner; and decision making—the appraisal of information from an assessment, together with other relevant information, to arrive at a decision that reflects the values and priorities of the system to which the decision makers are accountable.

Decision makers are required to work within the remit set for them by the system in which they are making decisions. Utilitarian economics might propose that all health systems should require their decision makers to maximize the common good. The values of a consumer-based Western democracy might emphasize the importance of transparency and public accountability. But not all might agree. A private insurance company, for example, might not accept that its responsibilities extend beyond its subscribers (other than where the law requires otherwise). At the national level, countries need to decide through their political processes how to balance the common good with individuals’ rights to health care where these are in competition. And not all public and private sector organizations—or the publics they serve—might agree that transparency and accountability should necessarily prevail if they lead to excessive costs and/or time to make decisions. These are matters for governments, voters, businesses, shareholders, customers, and patients to determine through the processes set out in their constitutions, laws, manifestos, and business strategies. They are not for HTA experts to dictate.

HTA experts could, however, help to develop a clear framework for describing decision-making systems. This would help those developing and operating decision-making systems or affected by them to understand the various options for such systems and make informed choices that reflect their values and preferences. A framework for describing decision-making systems could be developed from the principles proposed by Drummond et al, but it would need to be non-judgmental and not make assumptions about what is the right way to make decisions.

One of the most important challenges facing all decision makers wishing to use HTA to inform their decisions is assessing the quality of the HTA reports available to them—in particular the extent to which reports address the issues the decision makers must consider in a systematic, comprehensive, unbiased, and transparent manner. The issue here is not the nature of the agency or organization doing the assessment; it is the quality of the particular assessment(s) available to inform a decision. These assessments may come from a government or public sector agency, a university, a private consultancy, or industry—indeed, many organizations responsible for coverage decisions depend primarily on reports from industry submitted in support of the product they are seeking coverage or reimbursement for. HTA experts need to join in existing work to develop quality standards for HTA reports, such as INAHTA and EUnetHTA are pursuing. Agreement and adoption by all parties of a set of quality standards for the conduct and reporting of HTA would be of real practical value to all those seeking to use HTA reports in decisions about health care.

CONFLICT OF INTEREST

Chris Henshall has received funding from several public sector and charitable bodies for work on HTA, consultancy fees from several medical companies for chairing Advisory Board on specific technologies, and advice on global HTA developments and strategy.