We're getting there.

Since the origin of technology assessment (TA) in the 1960s, including TAs of certain medical interventions, and formal recognition and institutionalization of health technology assessment (HTA) in the 1970s, the field of HTA has evolved in programs large and small in the public and private sectors. It has diffused into an increasing number of nations in a range of policy-making and decision-support contexts. Since its inception, the development and diffusion of HTA have been strongly characterized by international collaboration involving sharing of expertise and practical experience, and joint efforts to advance the state of the art. Also during that time, there have been periodic collaborations to identify or develop standard frameworks, good practices, guiding principles, checklists, and other normative, “how-to” documentation for the field. Recognizing the diversity and ongoing evolution of HTA, these efforts have generally sought to balance the appeal of standardization with that of flexibility to accommodate the ranges of HTA program remits and contexts.

A series of efforts funded by the European Commission starting in the 1990s has contributed to improved practices and standards for conducting and reporting on HTA. These efforts have also served to promote international collaboration in the field. First, the EUR-ASSESS project addressed standardized methods for HTA priority setting, conducting HTAs, and reporting HTA findings (Reference Busse, Orvain and Velasco2). Growing out of EUR-ASSESS, the HTA Europe project provided an overview of HTA institutions and implementation in Europe (4). The European Collaboration of Health Technology Assessment (ECHTA) examined avenues for institutionalizing HTA across Europe, including shared assessments and education and describing good practices for undertaking and reporting HTAs (5;Reference Jonsson, Banta, Henshall and Sampietro-Colom8).

Subsequently, among its series of work packages for developing tools for HTA collaboration, the European network on Health Technology Assessment (EUnetHTA) produced a core HTA model for medical and surgical interventions (7) and a handbook on building capacity in HTA (6). This core model is designed to serve as a generic framework to enable international collaboration for producing and sharing the results of HTAs. The core model comprises detailed sets of HTA domains and production phases (7;Reference Lampe, Mäkelä and Garrido9).

Drawing from the experience and processes of its member HTA organizations, the International Network of Agencies in Health Technology Assessment (INAHTA) developed an annotated, fourteen-item checklist for HTA reports, noting that it is not intended to be a scorecard to rate HTA reports, which may be valid and useful without meeting all of the listed criteria (Reference Banta and Oortwijn1). These are among the more noteworthy and well documented of a larger number of such efforts around the world. Of course, these are not of the same level or scope; they address, variously, HTA programs, conducting HTA, reporting on HTA, establishing an HTA program, and other HTA-related endeavors.

Appearing across these collaborations to establish good HTA practices are certain common elements or attributes. These are derived in part from years of trial-and-error, experiences from alternative HTA models, cross-national learning, joint international HTA projects, methods and practices adapted from other fields, and external mandates (e.g., for transparency or accountability). Based on a distillation across these collaborative efforts and my own experiences with HTA programs and stakeholders, I offer the following set (not the set) of international good practices for HTA programs.

1. 1. Explicit mission or purpose of HTA program, including its mandate or other origins and how and by whom its reports and other products are to be used.

2. 2. Transparent, adequate, and stable funding.

3. 3. Explicit provisions and processes for governance, for example, in bylaws and related documentation, appointment and roles of governing board members or other oversight.

4. 4. Explicit provisions and processes for hiring and ongoing training of properly qualified staff.

5. 5. Explicit provisions and processes for engaging outside expert consultants, advisors, and reviewers appropriate for HTA topics.

6. 6. Provisions to minimize scientific biases, for example, pertaining to evidence gathering and interpretation, and to disclose and neutralize potential conflicts of interest, for example, among board and committee members, staff, and reviewers.

7. 7. Ongoing participation in international HTA collaboration and networks.

8. 8. Explicit processes and criteria for priority setting, topic selection, and determining assessment questions, using, for example, horizon scanning, priority criteria, and stakeholder input.

9. 9. Explicit, transparent, consistently implemented, and documented processes for conducting HTA.

10. 10. Explicit, valid processes, methods, and standards for identifying and assessing evidence.

11. 11. Explicit, valid processes, methods, and standards for conducting clinical, economic, and other analyses.

12. 12. Explicit provisions for independent review of draft reports.

13. 13. Explicit provisions for input by stakeholders, for example, to governance, priority setting, and HTA report review.

14. 14. Explicit process for dissemination or transfer of HTA reports to policy-makers, decision-makers, and other target groups, including via appropriate media for the respective groups.

15. 15. Provisions for outside appeals of HTA findings.

16. 16. Explicit process and criteria for reassessment, that is, updating or revising assessments.

17. 17. Independent review of HTA program performance and impacts.

“Good practices” here does not suggest “best” for all or an exclusive or comprehensive set that is applicable in all instances. Certainly, not all successful HTA programs exhibit all of these good practices; successful programs may exhibit additional ones; and unsuccessful programs may exhibit some of these. Yet, many of these practices tend to appear among diverse successful HTA programs. Even on such matters as transparency, stakeholder involvement, and independent review, which private sector HTA programs might be expected to avoid, such programs are exhibiting these practices to some extents. In any case, HTA programs across the globe, in different sectors, and more proximal or distal to policy making are exhibiting, or at least aspiring to, practices that look like these.

In this issue of the Journal, Drummond et al. (Reference Drummond, Neumann and Jönsson3) offer the latest addition to proposed standards of practice for HTA programs, based on a previously published set of fifteen principles for good HTA practice. In a further twist, the authors now augment each principle with two to eight well-conceived “audit questions,” packaged into a proposed benchmarking tool for quantitative scoring of HTA programs. Befitting an international collaboration, the seven authors are affiliated with a mix of public and private sector organizations from four countries, and each author has extensive international experience in HTA. The potential advantages cited by the authors of a quantitative benchmarking tool include providing an objective means of assessing program performance, identifying opportunities for improvement, and promoting the accountability of HTA programs.

The authors also cite potential confounders and other drawbacks, such as varying relevance of the set of principles across diverse programs and remits in different jurisdictions, the related challenges of arriving at an objective weighting system for generating an overall performance score, and determining who would administer the tool and enter the scores. As the authors acknowledge, further research, testing, and revisions of this benchmarking tool would be warranted before deploying it in earnest. Accompanying commentaries in this issue address several of these matters. Regardless of whether the suggested tools are used for quantitative benchmarking and summary scores, those principles and audit questions comprise a useful contribution toward lifting the performance of HTA programs and the state of the art.

There is no single “right way” to do HTA, yet there are right ways. And we are getting there as long as we identify evolving good practices that help us to know right HTA when we see it.

CONFLICTS OF INTEREST

Clifford Goodman's institution has received funding for his educational presentations about the field of HTA at meetings organized by industry associations and commercial conference organizers.