Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-02-05T05:55:11.195Z Has data issue: false hasContentIssue false
  • English
  • Français

End-user involvement in health technology assessment (HTA) development: A way to increase impact

Published online by Cambridge University Press:  26 April 2005

Maurice McGregor  and
James M. Brophy
Maurice McGregor
Affiliation:
McGill University and McGill University Health Centre
James M. Brophy
Affiliation:
McGill University and McGill University Health Centre
Rights & Permissions [Opens in a new window]

Abstract

Objectives: A mechanism to increase the influence of Health Technology Assessments (HTAs) on hospital policy decisions was developed.

Methods: We describe the process and results of an experiment in which a local in-hospital HTA unit was created to provide sound evidence on technology acquisition issues, and to formulate locally appropriate policy recommendations. The Unit consists of a small technical staff that accesses and synthesizes the evidence incorporating local health and economic data, and a Policy Committee that develops policy recommendations based on this evidence. It represents administration, health-care professionals, patients, and representatives of the clinical disciplines affected by each issue. The level of success of the Unit was independently evaluated.

Results: To date, 16 reports have been completed, each within 2–4 months. Five recommended unrestricted use, seven recommended rejection, and four recommended very limited use of the technology in question. All have been incorporated into hospital policy. Budget impact is estimated at approximately $3 million of savings per year.

Conclusions: This local in-house HTA agency has had a major impact on the adoption of new technology. Probable reasons for success are (i) relevance (selection of topics by administration with on-site production of HTAs allowing them to incorporate local data and reflect local needs), (ii) timeliness, and (iii) formulation of policy reflecting community values by a local representative committee. Because over one third of all health-care costs are incurred in the hospital, diffusion of this model could have a significant effect on the quantity and quality of health-care spending.

Keywords

Health technology assessmentHTAHealth policyHospital policyHealth-care costsPrioritization
Type
RESEARCH REPORTS
Information
International Journal of Technology Assessment in Health Care , Volume 21 , Issue 2 , April 2005 , pp. 263 - 267
Copyright
© 2005 Cambridge University Press

The overall purpose of Health Technology Assessment (HTA) is to inform health policy decision making through provision of analyses of efficacy, safety, costs, ethics, and legal issues, related to the acquisition and use of health technologies. Recognition of their potential to maximize the health benefits that can be realized with limited resources has resulted in a substantial proliferation of national and regional HTA agencies. However, there has been little such activity at the hospital or end-user level.

Furthermore, little effort has been spent on evaluating whether HTAs do in fact influence health policy decisions (7), and the available evidence of impact is not encouraging (4,6). Possible explanations for this finding are that the HTA process lacks the necessary mechanisms to translate evidence into policy (4) and that there is inadequate contact of HTA producers with the decision makers who use them (5).

Based on the hypothesis that locally developed HTAs would have greater influence on health policy, our hospital group recently has developed an in-house HTA unit to assist the hospital administration in difficult technology acquisition issues. Because similar units might be useful to others, its design, function, and initial results are described in this article.

PURPOSE

The initiating authority was the McGill University Health Centre (MUHC), a complex of five University Teaching Hospitals, totaling approximately 1,200 beds, functioning within the Quebec universal coverage health-care system. The principal reason for this initiative was to manage the stresses created by the ever increasing availability of expensive new technologies (procedures, drugs, devices) in the context of relatively fixed resources. Whereas the hospital is expected to provide all necessary services to Canadians, only rarely is the acquisition and use of one of these new technologies covered by a corresponding increase in hospital funding. Thus, new technologies must usually be paid for from existing funds.

Furthermore, in Canada, the traditional method of financing new expenditure through reduction of existing services (e.g., closing beds or extending summer breaks) is no longer a feasible option. This is because, over the past decade, there has been a substantial reduction in the level of hospital funding, with a resulting shortage of beds, an increase in waiting times, and an increase in congestion of emergency departments.

As a result, each new technology that increases hospital expenditure must now usually be acquired through reallocation of existing resources. Accordingly, before any new technology is acquired, it is imperative that careful consideration first be given to its efficacy, safety, budget impact, cost-effectiveness, and opportunity costs (i.e., that which the institution must give up to finance the new expenditure). The role of the hospital HTA unit, therefore, is to assist the administration to develop policy responses to local questions using a process that is transparent, fair, and consistent. However, in addition to presenting the scientific evidence relating to a technology, the Unit also develops policy recommendations based on the evidence that are sensitive to local circumstances and reflect community values.

Structure and Process

The Unit consists of two separate entities, a professional staff and a Policy Committee representing the hospital community.

Professional Staff

The scientific evidence is developed by a small staff with expertise in technology assessment (part-time Director, part-time Research Scientist, two full-time Research Associates and a Secretary). Their role is to access all relevant published information, especially HTAs developed elsewhere in the world, to update this evidence as necessary, to critically evaluate it, synthesize it, and ensure that the findings are locally pertinent, using local data when available, and when necessary carry out original research to achieve these goals.

Policy Committee

The role of the committee is to develop policy recommendations consistent with community values (on what should be done in the light of the evidence in the technical reports produced by the professional staff). The committee (honorary) consists of ten institutional representatives (nurses, medical doctors, other health professionals, and patients, each nominated by their respective Associations, and an administrator nominated by the Executive Director). To acquire the necessary knowledge of technology evaluation and hospital and health-care issues, members serve for 2 to 4 years.

The Committee is there both to act as the conscience of the institution, ensuring that decisions are consistent with the institution's values and with local economic realities, and to ensure that recommendations do not reflect parochial allegiances to specific hospital departments. In addition to the evidence relating to efficacy and costs, the Committee considers such issues as the institution's role, its budgetary status, the budget impact, the possibility of special funding, and the opportunity costs. The involvement of administrators in the process ensures that reports, while retaining full scientific rigor, will be practical and “user friendly.”

Topic Choice

While requests for a technology assessment may be accepted from any source in the hospital community, the primary objective is to provide timely policy advice to the Administration on technology acquisition issues that confront them. Thus, most analyses are undertaken at their request, prioritized according to their urgency and feasibility. The reasons for selection of a technology for such review include the probability of significant budget impact, doubts as to the quantity of the health benefits to be expected, doubts as to the level of proof of the purported health benefits, and the presence of significant ethical or legal complicating issues.

Consultants

In developing policy recommendations, the Committee is assisted by subject consultants and, when necessary, ethicists and health economists both from within the institution and externally. For a health policy decision to be effective, at least in the Canadian context, the support or “buy-in” of the health professionals most affected by the decision is important. For example, if the institution's nurses do not agree with a decision to not use a needlestick safety device, or the cardiologists reject a recommendation limiting the use of coated coronary stents, it can be difficult to put such measures into effect. Accordingly, for each study, influential representatives of the affected disciplines or departments are co-opted onto the Committee where they take an active role in synthesizing the evidence and formulating the subsequent policy recommendation. Furthermore, this participation is a two-way street. Not only does it ensure optimal “buy-in,” but it also ensures that the reports are locally relevant, with full appreciation of all clinical nuances.

Responsibility

The institution's normal decision-making process is unchanged. The Committee develops policy recommendations, not policy decisions. However, the Committee's reports are made public, and if rejected, the administration would be expected to give its reasons for rejection, a circumstance that may add weight to the recommendations.

RESULTS

The Unit was established in June 2001 and entered into production early in the following year. It has now submitted sixteen reports, each requiring 3 to 4 months to complete. At the end of 2003, an independent body, the Quality Management Team of the Health Centre, carried out an evaluation of the impact of the first eleven of these reports on institutional policy and budget. The full reports are available on the Web site: www.mcgill.ca/tau/

Policy Impact

The reports are listed in Table 1. The evaluation found that all recommendations of the first eleven reports had been accepted by the institution and incorporated into policy. The remaining five reports have now also been accepted and are in the process of implementation.

The recommendations made in these reports can be summarized as follows:

“Accepted”

Five reports (nos. 2,5,6,8,11) recommended acquisition and unrestricted use of technologies because of good evidence that they brought substantial health benefits at relatively low cost or that they caused an increase in productivity or in some cases even cost savings.

“Rejected”

In seven reports (nos. 1,3,7,12,13,15,16) use of the technologies was not recommended, on grounds either of insufficient proof of benefit or insufficient quantity of health benefit to justify the costs.

“Restricted”

Four reports (nos. 4,9,10,14) recommended that use of technologies be strictly limited. In these cases, although proven clinical benefits were recognized, it was believed that high opportunity costs would result in too great a negative impact on existing services. Thus, in these cases a utilization algorithm was specified that was considerably more restrictive than the estimated demand, while ensuring that the technology would be allocated to those most likely to receive a health benefit.

Budget Impact

Estimation of the net budget impact of these reports must of necessity be arbitrary. For those reports that accepted or rejected a technology, comparison was made between costs that were incurred as a consequence of the recommendations and an estimate of the costs that would have been incurred had the reports not been adopted.

When restricted use was recommended of a technology that was already in use, we recorded as a saving the estimated costs of the restricted use compared with the predicted expenditure that would have resulted from continued unrestricted use. However, when a technology had not yet been introduced, we recorded the anticipated cost of its use at the recommended level as an increased expenditure, despite that the cost of its unrestricted use would have been even greater. With these conservative assumptions, there was an estimated overall net annually recurring budget saving of $3,022,980 million, with a range of $2,259,530 to $3.778,404 million, depending on the underlying assumptions. But at least as important as the budget savings is that the institution, when given clear unbiased evidence of the extent of the health benefits that would accrue and the reasonableness of the associated costs, has been convinced to rapidly initiate use of some effective technologies.

Diffusion

To promote maximal use of the reports, they are circulated to all Quebec hospitals larger than 100 beds and to the Quebec Ministry of Health and Social Services and are published on the Web. The Web site has been downloaded over 20,000 times in the last year, and there is increasing evidence of the usefulness of these reports, both to the hospitals, to the Ministry, and to HTA agencies in more distant jurisdictions.

DISCUSSION

The extent to which hospital policy is currently evidence-based varies between institutions and countries. In Canada, many hospitals use some form of committee structure to guide their decisions on technology acquisition (1). However, this process frequently consists of an ad hoc group convened by the Administration, drawing heavily on the expertise of the discipline that is advocating the technology acquisition and often supported by HTAs developed by the vendor. As a result, the decisions of such groups are subject to potential bias in favor of technology acquisition. Furthermore, when each issue is considered by a different ad hoc committee, decisions tend to be inconsistent from case to case. The creation of a group that represents all sectors of the institution and consistently applies the same principles to guide policy decisions has obvious advantages.

The value of the structure and process described above may be limited in jurisdictions with a highly active, central, authoritarian HTA structure, such as the United Kingdom (7). However, in countries where hospitals have the power to determine what services they will provide within their allocated budgets, a substantial amount of health policy is determined at the hospital level. In Canada, over one third of all health-care spending takes place in hospitals, and the cumulative effect of hospitals' decisions is, therefore, an important determinant of overall health-care costs. Thus, it is essential that the hospitals base policy decisions on unbiased information such as that provided by HTAs. The high impact of the reports developed in the current experiment can be attributed to the relatively rapid response to requests, taking 3–4 months rather than the usual 12–18 months (5), and that they were developed in the hospital setting in collaboration with the hospital's administration, health-care professionals, and patients, all of whom would be affected by decisions.

There are other benefits associated with the on-site development of HTAs compared with those developed by more distant agencies. The greatest of these benefits is the ability to produce HTAs that go beyond pure data analysis to make actual policy recommendations. A policy decision on what should be done requires far more than an objective analysis of the evidence. For example, a decision to adopt a safety device to protect health personnel from needlestick injuries (2) depends, in addition to the cost, on a knowledge of the local rates of HIV, hepatitis C, and hepatitis B, the local institution's budget status, the potential for additional funding, the potential effects of the decision on local nursing morale, and on the value judgments of the community. Such issues are best estimated locally.

Similarly, in deciding whether to authorize the use of implantable cardiac defibrillators (3), in the presence of a fixed budget, the weight given to the opportunity cost becomes critical. Which would produce the greater benefit to patients, to approve a policy to insert 100 additional devices per year or to close the approximately 30 beds that would be necessary to finance the cost of such a decision? Whereas the opportunity costs can easily be calculated by a distant agency and may be generalizable, the weight given to them in decision-making is a local matter that cannot easily be estimated by a remote agency.

The development of HTAs at sites close to the end-user has one potential disadvantage, namely, that it may permit the development of local differences in the services provided by different institutions. This in turn creates the possibility of patients seeking elsewhere those services that are not provided in their own institution, so-called postal-code prescribing. Overall, however, the clear advantages of local HTA development are sufficient to outweigh this theoretical concern.

Policy Implications

Although small may be beautiful, big is also essential. An optimal system might consist of a coordinated network consisting of large central agencies, affiliated with numerous small peripheral agencies situated in close proximity to end users. The large agencies would have two functions. As at present, they would develop both analyses and policy recommendations for governments and supraregional health-care delivery organizations. They would also produce technology assessments on any health topics that could be generalizable. The small, hospital-level, HTA agencies would develop locally relevant policy advice based both on the assessments provided by the central agencies, with incorporation of locally relevant data.

Whether such a coordinated system ever eventuates or not, the experience reported here suggests that small HTA units in close proximity to decision-makers can substantially promote the impact of HTAs on health policy, and as a consequence, can have a significant influence on overall health benefits and costs.

CONTACT INFORMATION

Maurice McGregor, MD (), Professor Emeritus of Medicine, McGill University; Senior Physician, Department of Medicine, Division of Cardiology, McGill University Health Center, Royal Victoria Hospital (Room M4, 76), 687 Pine Avenue West, Montreal, Quebec H3A 1A1, Canada

James M. Brophy, MD, PhD (), Associate Professor, Department of Medicine, Divisions of Cardiology and Clinical Epidemiology, McGill University; Director, Technology Assessment Unit, McGill University Health Centre, Royal Victoria Hospital (R4, 12) 687 Pine Avenue West, Montreal, Quebec H3A 1A1, Canada

This project was totally funded by the McGill University Health Centre. The authors are grateful to Dr. Hugh Scott and Dr. Denis Roy, Director General and Associate Director General, respectively, at the time the Unit was created, and Mr. Victor Simon, Chief Operating Officer, all of the McGill University Health Centre, for their support of this initiative, and to the members of the Committee for the many hours of thoughtful input that have been essential for its success.

References

Marshall D, Menon D. 1990. Technology assessment in hospitals: A Canadian perspective. In: Proceedings of the sixth annual meeting of the International Society for Technology Assessment in Health Care. Houston, TX. May 20
McGregor M. 2002 Should the McGill University Health Center replace the Jelco/cathlon catheter by the ProtectIVPlus catheter for intravenous infusions. The Technology Assessment Unit of the McGill University Health Centre. Available at: www.mcgill.ca/tau/.
McGregor M, Chen J. 2003 Use of the implantable cardiac defibrillator (ICD) at the McGill University Health Centre. The Technology Assessment Unit of the McGill University Health Centre. Available at: www.mcgill.ca/tau/.
Oliver A, Mossialos E, Robinson R. 2004 Health technology assessment and its influence on health-priority setting. Int J Technol Assess Health Care. 20: 1– 10.Google Scholar
Roehrig C, Kargus K. 2003 Health technology assessment in Canada and in the G-7 countries: A comparative analysis of the role of HTA agencies in the policy-making process. Ottawa: Health Canada. http://hc-sc.gc.ca/english/care/hcsd_wp_ hta.htm.
Romanow RJ. 2003 Building on values. The future of health care in Canada. Ottawa: Health Canada; 83– 85. Available at: www.hc-sc.gc.ca/english/pdf/romanow/pdfs/HCC_Final_Report.pd.
Stevens A, Milne R. 2004 Health technology assessment in England and Wales. Int J Technol Assess Health Care. 20: 11– 24.Google Scholar
Figure 0

Summary of Topic Titles, Recommendations, and Budget Impact