Three Phases of HTA

HTA has evolved through three distinct phases: the machine, the clinical outcomes, and the delivery modes, with the third of these still under way. As the focus has shifted from a single machine to choosing among interventions for specific disease conditions to service delivery approaches, HTA has drawn on research and modes of discourse from a growing variety of disciplines. HTA remains, at its core, both multidisciplinary and pragmatic. HTA's strengths arise from integrating the efforts of colleagues in multiple, diverse disciplines to produce knowledge that will assist decision makers.

The machine phase was marked by a focus on the technical performance of health technologies, often embodying innovative approaches to diagnosis or treatment of human illness. Given the newness and costliness of many technologies selected for assessment, a significant emphasis was placed on assessing the safety and efficacy of these devices. Imaging technologies were the subject of assessment in many settings, perhaps in part because devices such as the computed tomography (CT) scanner produced remarkable visual results that were perceived as affording breakthroughs in diagnosis and treatment. One need only look through the programs of early HTA conferences to see the emphasis on high-cost, infrastructure-intensive health technologies that was the hallmark of the machine period.

With the growing search for health benefits as a consequence of technology use and the rise of evidence-based medicine, HTA's focus shifted to disease conditions and clinical outcomes—in a nutshell, moving from diagnosis alone (does this technology do what its promoters say it will do?) to prognosis (which technologies, including pharmaceuticals, yield desirable changes in health outcomes and how can these be compared?). Focusing on outcomes of technology use in clinical settings expanded the ranks of contributing disciplines as epidemiologists, economists, social scientists, and ethicists all became important contributors to HTA products. Furthermore, as HTA products became more complex, their potential contribution to decision making also became more complex, with impacts across multiple levels of decision making. Thus, whereas HTA appears to have had its greatest impact at the macro or policy level (e.g., assessing the relative effectiveness and costs of one imaging technology over another as an input to a planning process for capital budgeting at a health system level), its emphasis on outcomes should be as important to institutional and practitioner decisions (e.g., comparing drugs, different forms of angioplasty and surgery in the treatment of patients with coronary artery disease).

The move to the third phase—assessing how health services are organized and delivered—still ongoing, appears to be driven, in part, by the uncompleted work of knowledge transfer (i.e., increasing the use of HTA results in decision making at all levels). HTA's policy-level impact has been demonstrated in several studies, particularly so where the HTA organization is requested to complete an assessment whose target audience is the same as that making the request (1;2;16;18;19;25). Influencing institutional decisions (14;22) and professional practices (3;6;10;15) has been more challenging and the growing HTA focus on how services and technologies are delivered can be understood as a response to the recognition that the bulk of resource allocation decision making occurs outside the policy realm, within regional health authorities, hospitals, departments, and in examining rooms, clinic offices, and operating theaters. Part of this challenge reflects the fundamental differences between informing a primed policy maker and changing the behavior of a large, diffuse group of managers and health professionals.

The complexity of HTA has increased in part because the evolution from evaluating a single machine through comparing technologies' impacts on disease conditions to ways of organizing services has not closed off the importance of further investigation of the early phases. For example, a comprehensive evaluation of delivery models of effective and efficient oncological services, subsumes the evaluation of diagnostic and treatment interventions. Thus, the narrower focus of the machine phase, stressing technical performance of health technologies, often embodying innovative approaches to diagnosis or treatment of human illness, must continue to inform broader assessments of approaches to service delivery.

Research Agenda for HTA

Since its inception, HTA has focused on doing assessments of evidence. As a result, the theoretical foundation for the field remains underdeveloped. It is high time for HTA to bring together aspects of conceptual and theoretical works from other fields to strengthen the foundation of HTA. Many challenges await the further development of HTA. They can be captured around three research themes: adapting HTA to an evolving analysis object; translating HTA results into policy, management, and practice decisions; and, evaluating organizational models of HTA.

Adapting HTA

HTA's evolution from machine to service delivery highlights the first of the three research themes: the expanding breadth of the technologies to be evaluated. Put practically, can the tools used to assess the early CT scanner be applied to evaluate the options for organizing diagnostic imaging services for a health region of 4 million people? Indeed, it is critical to HTA's ongoing impact that its tools evolve to meet its tasks. Furthermore, this growing breadth brings particular attention to the context of policy making. This attention to context draws HTA closer to the political environment in which decisions are made, rather than remaining distant from it and, in some cases, may force real trade-offs between relevance and autonomy for HTA organizations.

These tasks are particularly demanding of fresh thinking as the boundaries between health and disease and between diagnosis, treatment, and prognosis blur. This blur is a consequence of the transformation of the notion of health with the wellness industry, on one hand, creating a new domain of market transactions where clients may “purchase health,” and probabilistic future outcomes identified through genetic technologies on the other hand. This expansion of the concept of health, coupled with information technologies that have reduced the status of the expert, lead to several areas of potentially productive research activity.

How do changing notions of health define the objects of evaluation for HTA? Where “health” can be purchased in the market like other consumer goods, what role if any does HTA play? For example, what are the determinants of a consumer decision to purchase over-the-counter diagnostic or screening kits and what information delivered in what ways would inform consumer behavior? For publicly funded health systems, an important challenge for assessment arises from marketplace technologies that create follow-on effects for the health system (e.g., an over-the-counter diagnostic test used at home whose results, both true positives and false positives, require further investigation within the health system).

Using the example of genetic technologies, where direct-to-consumer marketing in North America is already under way, what lessons can be drawn from the “first wave” of genetic technologies about how the objects of inquiry and methods for HTA can provide information most useful to decision makers? Examining the impact of new developments in genetics on health services, public policy, and citizens' information level and participation is crucial and will occur ideally with the participation of a broad array of stakeholders as active designers of the research questions (4;7;8).

What are the policy implications of pharmacogenetics and what methods are best-suited to evaluating a pharmaceutical product bundled with a genetic test that seeks to identify persons most likely to respond to the drug or most likely to experience adverse effects? What are the implications for practitioners and patients if some therapeutic technologies are provided only to patients with genetic profiles known to optimize effectiveness—so-called “rationing by genes”?

The “consumer as king” has come to dominate the marketplace for many goods and services across more and more of the globe, yet consumer sovereignty alone is insufficient in markets where information is not available, poorly understood, or distributed, or where some costs or benefits are borne by parties not present in the marketplace. However, “command and control” approaches to regulation of such novel technologies such as genetic testing, would be misguided as the costs of relying entirely on these relatively blunt instruments can only increase and worse, engender a brutal awakening at some future point that could be avoided with engagement now. Asking questions such as these that are pertinent to decision making and investigating them through multidisciplinary approaches will not only provide valuable insights to inform technology management but also provide an opportunity to reflect on how HTA is done and how that can evolve to maintain credibility and relevance (24).

Translating HTA

Whereas the technologies that HTA must consider are evolving rapidly, the need to translate HTA results into policy, management, and practice decisions is an enduring theme from the field's earliest days. In its early days, HTA adopted a simple diffusion model, perhaps best summarized as “if you publish it, it will be read and appropriate action taken.” The limits of this approach were quickly identified and have been well-described in both the academic and managerial literature (11;13;20). The challenge of implementing more effective alternatives remains unfinished work for HTA organizations and their partners, as more people recognize that “…evidence-based decision making is a social process, not a technical task.” (21).

Social process may seem a rather vague description of the way decisions are made. However, without seeking to understand and influence such processes, HTA results risk not only being left on the outside looking in on decision making, but also being misused if evaluators do not participate in their contextual interpretation. In addition, although decision-making processes and how evidence is used have received much analytic attention, the next step is field testing interventions to increase the impact of HTA results in decision making.

When speaking of decision making, many commentators have adopted the categories of macro (policy makers), meso (institutions), and micro (practitioners and patients) decision making. If we take stock of what is known today about each of these categories, there would be little debate that it is the macro level that has been the most comfortable for HTA. The success of INAHTA—the International Network of Agencies for Health Technology Assessment—suggests that some similarities exist in macro-level decision making across health systems, that is, that HTA practitioners are using a common language and seeking to learn from others' experiences. By contrast, decision making by managers, practitioners, and their patients, and how HTA results may be involved is an important research frontier, from which findings are much less likely to be generalizable within a single health system let alone across cultures, yet critically important to maximizing health outcomes and allocating resources equitably. Thus, research questions arise for each category of decision making.

At the Macro Level

For macro decisions, the challenge can be viewed in terms of creating HTA policy exchange forums to set priorities, complete assessments, and translate those assessment results into policy. More specifically, research questions in this area include the following: What are the essential features of macro-level knowledge networks for success, and how is that success to be defined? How can theoretical work and experience in other areas of policy making be pragmatically applied to health and inform field testing to identify effective approaches for HTA? Partnerships within and across health systems will be essential to progress in this area and the strength of HTAi (Health Technology Assessment International) and INAHTA are encouraging signs that the foundation is already well established.

At the Meso Level

Moving from the macro level and surveying HTA's knowledge translation efforts to date, there is a growing gap between the emphasis and success at the policy level on one hand, and the paucity of knowledge translation effort directed at healthcare institutions and organizations such as regional health authorities, hospitals, and long-term care facilities.

To be sure institutional management is in part a response, perhaps often purely reflexive, to the demands of the micro level subject to the constraints imposed by the macro level, but institutions are not rudderless boats on a sea of chaos. In practice, internal debates and decisions about resource allocation are made at multiple moments and need to be more fully understood if HTA is not to vanish in the critical institutional juncture between macro and micro level decision making.

Multiple research questions thus arise in this realm, including the following: What training and linkages are necessary to increase the uptake of HTA results in institutional decision making? What features of the institutional decision-making context are amenable to modification by policy or practitioner-focused efforts to increase the use of HTA results and are these preferable to direct intervention involving institutions and their managers? Can practical instruments be developed and tested to increase the use of HTA results by managers? What role could be played by academic health sciences centers as leaders in institutional adoption of evidence-based decision making that explicitly draws on HTA? Such initiatives are on-going or in development in Canada, Italy, Denmark, Australia, ant Switzerland, where HTA efforts within an academic health sciences center not only inform decisions but act as a seed for professionals whose training will be informed by the ideas driving HTA.

At the Micro Level

Finally, it is the micro level, at bedsides and in offices and clinics, where many of the decisions that determine technology use are made. Despite increasing information parity between health professionals and their clients, facilitated by the Internet, the professional's scope for framing the discussion remains a potent determinant of outcome. And yet, much of what happens in these human encounters is only dimly if at all considered in efforts to manage health technologies. Experiences and perspectives from other disciplines will be vitally important in breaking out of the generally passive diffusion efforts that, to date, have been used to change practitioner behavior.

Furthermore, the rise of disease management, coupled with increasing vertical integration, bundling technologies such as Pharmaceuticals with a disease management program, creates new opportunities for intervention research on effective approaches to changing practitioner behavior to incorporate evidence and HTA results (9;12;23) . Also, HTA increasingly reaches the public directly, engaging citizens whether explicitly as in the NICE (National Institute for Clinical Excellence) efforts in the United Kingdom or implicitly as part of a growing accessible body of evidence that shapes decisions that affect citizens and in which they may participate.

Research questions in this area include the following: How can information technology be introduced in practitioner work environments to capture accurately what practitioners do, compare it with guidelines and provide real-time feedback that will enhance the practitioner-patient relationship and improve outcomes for patients? What changes in the training of healthcare practitioners would facilitate the maintenance and deepening of evidence-based practice and enhance the use of HTA results? And last but not least, how are public participation in HTA and delivery of HTA results to citizens as recipients of health technologies to be developed to enhance the overall HTA effort rather than merely provide window dressing, or worse, additional confusion and uncertainty?

Evaluating HTA

The third of the three areas calling for research attention as HTA reaches midlife is the HTA organization itself. Given HTA's focus on evaluation, it would be particularly misguided to exempt the HTA organization and HTA processes themselves from evaluation (17). Some of these research inquiries can be pursued in the “natural experiment” that arises from the more than forty member organizations of INAHTA, operating in a variety of cultural and health system contexts, leading to studies of international comparisons. Identifying determinants of success for HTA organizations will create a “public good” available to all HTA organizations as they continue organizational learning and development.

An additional area of evaluation would seek answers to the question of how the organizational approach to HTA within a health system influences technology diffusion. Distinct from the results of HTA, the particular organizational features of the body producing those results may influence diffusion, whether through context (e.g., a government department within a publicly funded system) or through governance and linkages to technology developers and users.

These contextual factors are critically important, yet they have received little attention. For example, HTA organizations can provide a quasi-neutral forum for those who produce technologies, those who evaluate technologies, and those who make decisions about technologies. However, as engagement with such partners deepens, it will be important to examine and explore how HTA bodies make trade-offs among divergent interests.

In addition, HTA organizations will be required to play increasingly prominent roles in decisions about resource allocation for health technologies and health services. This role cannot be other than political and will often be politically charged, highlighting the value of investigating and understanding the very concept of an effective HTA organization. What will be the impact of this changing role on the scientific work of the organization? What are the organizational characteristics of a viable equilibrium between the necessary autonomy of HTA organizations to ensure methodological rigor and a demonstrable impact on policy making? Case studies of organizations adopting different roles regarding resource allocation should be examined to identify characteristics of viable equilibria.