Cross-national comparison of technology assessment processes

Anna García-Altés; Silvia Ondategui-Parra; Peter J. Neumann

doi:10.1017/S0266462304001126

Cross-national comparison of technology assessment processes

Published online by Cambridge University Press: 01 August 2004

Anna García-Altés ,

Silvia Ondategui-Parra and

Peter J. Neumann

Show author details

Anna García-Altés: Affiliation:
Harvard School of Public Health Universitat Autònoma de Barcelona
Silvia Ondategui-Parra: Affiliation:
Harvard School of Public Health Brigham and Women's Hospital
Peter J. Neumann: Affiliation:
Harvard School of Public Health

Article contents

Abstract
METHODS
RESULTS
DISCUSSION
POLICY IMPLICATIONS
References

Rights & Permissions

Abstract

Objectives: To compare methods and results among four health technology assessment organizations in different countries.

Methods: All assessment reports published between 1999 and 2001 by VATAP (United States), NICE (United Kingdom), CCOHTA (Canada), and AETS (Spain), were reviewed. Detailed information about the organization, the technology assessed, the methods used, and the recommendations made were collected. A descriptive analysis of the variables, as well as comparisons of means and proportions, was performed.

Results: Sixty-one reports assessing seventy-six technologies were published: nine (11.8 percent) by VATAP, thirty-nine (51.3 percent) by NICE, twenty (26.3 percent) by CCOHTA, and eight (10.5 percent) by AETS. A total of 64.5 percent of the technologies assessed were related to a high prevalence disease in the corresponding country. Most of the assessments addressed treatments (73.7 percent) and were mostly drugs (56.6 percent) and devices (23.7 percent). Most organizations used reviews of effectiveness and economic evaluations (64.5 percent), systematic reviews (21.1 percent), and original economic evaluations (36.7 percent). In 38.1 percent, the technology was recommended; the rest of the cases had no formal recommendations.

Conclusions: Critical issues for future technology assessment efforts are making assessment processes more consistent, transparent, and evidence-based; formalizing the inclusion of economic and ethical considerations; and making more explicit the prioritization process for selecting technologies for assessment and reassessment.

Keywords

Assessment process International comparison Coverage decisions

Type: GENERAL ESSAYS
Information: International Journal of Technology Assessment in Health Care , Volume 20 , Issue 3 , August 2004 , pp. 300 - 310

DOI: https://doi.org/10.1017/S0266462304001126 [Opens in a new window]
Copyright: © 2004 Cambridge University Press

Formal health technology assessment (HTA) offers an appealing, evidence-based approach to help inform coverage and reimbursement decisions about medical advances. HTA is defined as the evaluation of a medical technology for evidence of its safety, efficacy, cost, and cost-effectiveness, and its ethical and legal implications, both in absolute terms and in comparison with other competing technologies (63). In recent years, the number of organizations conducting technology assessment worldwide has proliferated (35;36;64;66;67). In the United States, rapid growth of health technology assessment activities has occurred in the private sector (64;67). In Europe, HTA activities started in the 1980s, with the creation of formal assessment groups directly related to government decision making, and have been growing continuously (14;58).

Whereas previous investigators have reviewed HTA activities in the United States and abroad (27;64;67), little empirical research has been conducted at the technology assessment level to understand the nature or impacts of different policies. Although investigators have examined aspects of the process in Australia (31;32;65), Europe (12;22), and Canada (37), there is little in the way of cross-national comparisons.

One might expect national technology assessment organizations to have similar assessment processes in terms of the types of technologies assessed and the methods used. The objective of this study was to analyze four health technology assessment organizations in the United States and abroad to investigate the extent to which this is true. In particular, we examined: (i) the types of technologies assessed, (ii) the methods used for the assessment, (iii) the reasons for the assessment, (iv) the degree of stakeholder participation, and (v) the recommendations made. We also discussed health policy implications.

METHODS

A data collection form was designed to obtain systematically the information of interest. The form included variables regarding the technology under assessment, and the assessment process (Table 1). The form was pilot tested twice. In the pilot tests, two trained readers, each with graduate education in technology assessment and economic evaluation, read the same thirteen reports, respectively, using a draft form, and then convened to review discrepancies in their findings and to improve the form's clarity.

Each technology assessed was characterized in terms of the disease category covered (coded using ICD-9 codification), the type of technology (drug, device, medical procedure, surgery procedure, or educational/behavior), its function (prevention, diagnostic, treatment, or rehabilitation), and its novelty (innovation, advance over an already existing technology, use of an already existing technology in a new indication, experimental, not yet allowed for its use, or already existing technology; Table 1).

We also coded the explicit mention of the reason for assessment, the assessment method used (e.g., randomized controlled trial, systematic review, economic evaluation, etc.), the decision (recommended, recommended with conditions, not recommended), an explicit mention of stakeholders' participation in the report, and mention of funding sources for the project.

We reviewed all reports published between 1999 and 2001 by Veterans Administration–Technology Assessment Program (VATAP, USA), National Institute for Clinical Excellence (NICE, United Kingdom), Canadian Coordinating Office for Health Technology Assessment (CCOHTA, Canada), and Agencia de Evaluación de Tecnologías Sanitarias (AETS, Spain). The time period of analysis was chosen to include all NICE's reports since its creation (1999), and the last complete year before starting the study (2001). The organizations were selected to reflect geographical distribution, and health policy relevance, while maintaining a degree of homogeneity in terms of including publicly funded agencies, with similar missions (Table 2).

Two readers independently read each report and completed the data collection form. A consensus meeting was held for readers to reach agreement about areas of disagreement. The organizations produced a total of sixty-seven reports during this time period: six from VATAP (83–88), thirty-one from NICE (13;17;19;20;23;26;38;40–44;47;49;50;55;56;59;61;62;69;71;76–78;80;81;89–92), eighteen from CCOHTA (16;24;34;45;46;48;51–54;60;68;72–75;93;94), and eleven from AETS (1–11). Almost all were available through their Web pages; one report was not possible to download (11); two were requested by mail (51;53); and two were excluded, because they were not technology assessment reports (i.e., catalog of publications [9], and guidelines for the elaboration of technology assessment reports [10]). The final sample comprised sixty-one reports. Because some reports (20;41;44;54;69;86;88;91;93) contained the assessment of more than one technology (e.g., drugs for Alzheimer's disease [20]) or the same technology applied to different conditions (i.e., predictive genetic testing for breast and prostate [54]), or updated information of previous reports (56;77), the unit of analysis considered was the technology rather than the report, per se, resulting in a final sample of eighty units of analysis.

We conducted descriptive analyses of the variables, as well as comparisons of means and proportions (analysis of variance, Chi-statistic). Data were stored and analyzed with SPSS 10.1 for Windows.

RESULTS

Table 3 shows the technologies assessed by each organization between 1999 and 2001. Only one, Zanamivir for the treatment of influenza, was analyzed by more than one agency (16;17). Assessments were mostly directed to technologies covering neoplasms (31 percent) and mental disorders (14 percent; Table 4).

The organizations most commonly assessed drugs (58.7 percent) and devices (22.5 percent), although there were significant differences in the types of technologies examined across organizations (p=.000). Most assessments focused on treatments (75 percent). In terms of novelty, assessments focused primarily on existing technologies (51 percent) as opposed to innovations or new uses of existing technologies (36 percent; Table 4).

The nature of the process differed across organizations in terms of whether the assessment resulted from a formal prioritization process, whether it included an economic evaluation, and the extent to which stakeholders participated (Table 5): VATAP and NICE always stated the reason for their assessments; NICE and CCOHTA mostly used economic evaluation methods; and CCOHTA made the participation of stakeholders explicit in their reports.

The funding of the project was seldom mentioned: only CCOHTA mentioned funding from public grants, as well as authors with associations with pharmaceutical companies, and in one case that no conflict of interest existed (Table 5). All these differences among organizations were statistically significant.

Organizations also differed in terms of the frequency with which they recommended a technology after an assessment. VATAP, NICE, and AETS recommended the technology or recommended with conditions 33 percent, 51 percent, and 38 percent, respectively, for example. CCOHTA made general comments in 50 percent of cases and recommended against in 25 percent.

DISCUSSION

The process of HTA typically includes the identification and prioritization of the technologies for assessment; search, review, synthesis, and production of the scientific evidence; context analysis, including the analysis of the effectiveness, efficiency, and equity and legal aspects of the application of the technology in a specific context; elaboration of policy recommendations; dissemination activities; and impact analysis (29).

However, our analysis reveals significant differences in assessment processes across four large organizations. In particular, there are differences in the diseases covered, the types of technologies assessed, the technology's function and novelty, the assessment methods used, the recommendations made, and the funding of the projects.

First, the results suggest that the types of technologies assessed do not typically depend on the specific characteristics of each country and organization. For example, NICE has assessed many drugs for neoplasms, although age-standardized cancer incidence and mortality rates in the United Kingdom are not higher than those of other countries; the same is true for mental disorders in Canada (95). On the other hand, very few assessments in all four countries have targeted diseases of the circulatory or respiratory systems, that are important causes of death (95). Similarly, the types of technologies assessed also differ across organizations (e.g., VATAP and AETS assess mainly devices, while NICE and CCOHTA assess mainly drugs), for reasons that are not readily apparent.

We only found one matching assessment among organizations in the time period analyzed. This finding may, in part, reflect attempts at coordination among European technology assessment organizations through the International Network of Agencies for Health Technology Assessment (INAHTA) a body that, among other things, tries to ensure no duplication of assessment efforts.

Second, the data highlight the different way in which recommendations are made, with some organizations issuing general guidance, rather than mandatory decisions.

Third, the organizations generally lack explicit processes for prioritization, and they do not make explicit both why they assess what they are assessing and who participates in the assessment. NICE notes that the basis of selection includes criteria such as health benefit, significant impact on other health-related government policies (i.e. reduction in health inequality), significant impact on NHS resources, and adding value by issuing a national guideline (79). VATAP mentions the uncertainty regarding the worthiness of the technology by financing and planning bodies as the reason for the assessment. However, in general terms, there is little in the way of explicit, quantitative methods to inform the prioritization process of technologies to be assessed using societal criteria such as burden of disease, uncertainty about the effectiveness and cost-effectiveness of the intervention, and potential benefits and impact of the assessment (33;57;70). On the same line, there is not explicit mention about any political deliberation that leads to the assessment of certain technologies and the participation of stakeholders in any step of the process, steps that are key for an open, systematic, and unbiased decision making (30;33).

Fourth, organizations differ in the extent to which they include economic evaluation. The idea of using cost-effectiveness to inform coverage and reimbursement decisions has gained popularity (21). But our results showed continued variation in the methods used (18). In particular, only NICE and CCOHTA regularly use economic evaluation studies in their assessments.

The main limitation of this analysis is the small sample of organizations used. Organizations were selected to reflect geographical distribution and health policy relevance, while maintaining a degree of homogeneity in terms of including publicly funded agencies, with similar missions. They are not representative of the entire health technology assessment community, although they are well known and play an important role in coverage decisions in their respective countries. Nonetheless, the sample is big enough to show a lot of variability in a process—technology assessment—that, apart from the adaptation of the technology to the local context, is supposed to be standard, and lack of explicitness, something that is so important in a process related to the inclusion of new technologies in a health-care system.

Researchers have identified a series of relevant issues in the dissemination of HTA results such as barriers to change, timing, assessment of target groups, and credibility of both the message and the messenger (28). There is evidence suggesting that the simple diffusion of information is not sufficient to promote the application of research results in clinical practice (15) and that more research is needed on the effectiveness of different dissemination tools among citizens, politicians, and mass media (28).

Others have emphasized the importance of social, political, and ethical aspects of health technology (39). Often, policy decisions will be made on this basis of a trade-off between the evidence available on clinical and cost-effectiveness, and several other considerations, including political pressures, availability of funding, or patient and caregiver opinion. The challenge under these circumstances is to maintain transparency and consistency of the decision making process in the face of these factors, in both the public and private sector (25;29).

POLICY IMPLICATIONS

We recommend that decision-makers make explicit why a particular technology is assessed, who participates in the assessment process, what determines the decisions, the sources of founding of each project the prioritization process, and recommendations for further research. Medicare officials in the United States in particular should consider these issues as they seek to improve the coverage process, in terms of length of time required to make coverage decisions and the explicitness and openness of the process (82).

This study was funded in part by the Robert Wood Johnson Foundation under grant 046071.

References

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2001. Tomografía por emisión de positrones (PET) con 18fdg en oncología clínica (revisión sistemática). Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2001. Efectividad de los apósitos especiales en el tratamiento de las úlceras. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2001. Efectividad de las bombas de infusión de insulina. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2001. Prótesis de hombro en indicaciones de procesos degenerativos o traumatológicos. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2000. Braquiterapia para el tratamiento del cáncer ginecológico y de otras localizaciones. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 2000. Riesgos para la salud causados por implantes de silicona en general, con atención especial a los implantes mamarios. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 1999. Radioterapia Intraoperatoria. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 1999. Tomografía por Emisión de Positrones con Fluordeoxiglucosa FDG-PET en Neurología. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 1999. Catálogo 2000: Informes y Publicaciones de las Agencias Españolas de ETS. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 1999. Guía para la elaboración de Informes de Evaluación de Tecnologías Sanitarias. Madrid: AETS-Instituto de Salud Carlos III

Agencia de evaluación de Tecnologías Sanitarias (AETS), Instituto de Salud Carlos III-Ministerio de Sanidad y Consumo. 1999. Diagnóstico de lesiones mamarias detectadas en cribado poblacional de cáncer de mama mediante mamografía. Madrid: AETS-Instituto de Salud Carlos III

Alban A. The role of economic appraisal in Denmark. Soc Sci Med. 1994 38: 1647– 1652.Google Scholar

Bagnall AM, Forbes C, Lewis R, Golder S, Kleijnen J. 2001. An update of a rapid and systematic review of the effectiveness and cost-effectiveness of the taxanes used in the treatment of advanced breast cancer. York: NHS Centre for Reviews and Dissemination, University of York

Banta HD, Perry S. A history of ISTAHC. A personal perspective on its first 10 years. Int J Technol Assess Health Care. 1997 13: 430– 453.Google Scholar

Bero LA, Grilli R, Grimshaw JM, et al. Closing the gap between research and practice: An overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998 327: 465– 468.Google Scholar

Brady B, McAuley L, Shukla VK. 2001. Economic evaluation of zanamivir for the treatment of influenza. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 13.

Burls A, Clark W, Stewart T, et al. 2000. Zanamivir for the treatment of influenza in adults. Birmingham: Department of Public Health and Epidemiology, University of Birmingham

Busse R, Orvain J, Velasco M, et al. Best practice in undertaking and reporting health technology assessments: Working group 4 report. Int J Technol Assess Health Care. 1997 18: 361– 422.Google Scholar

Chilcott J, Wight J, Lloyd Jones M, Tappenden P. 2001. The clinical effectiveness and cost-effectiveness of pioglitazone for type 2 diabetes mellitus. Sheffield: University of Sheffield School of Health and Related Research, University of Sheffield

Clegg A, Bryant J, Nicholson T, et al. 2000. Clinical and cost-effectiveness of donepezil, rivastigmine and galantamine for Alzheimer's disease. Southampton: Rapid Reviews Team, Wessex Institute for Health Research and Development, University of Southampton

Cranovsky R, Matillon Y, Banta D. EUR-ASSESS project subgroup report on coverage. Int J Technol Assess Health Care. 1997 13: 287– 332.Google Scholar

Davies L, Coyle D, Drummond M, et al. The current status of economic appraisal of health technology in the European Community: Report of the network. Soc Sci Med. 1994 38: 1601– 1607.Google Scholar

Dinnes J, Cave C, Huang S, Major K, Milne R. 2000. The effectiveness and cost-effectiveness of temozolomide for the treatment of recurrent malignant glioma. Southampton: Wessex Institute for Health Research and Development, University of Southampton

Einarson TR, Iskedjian M. 2001. Novel antipsychotics for patients with attention-deficit hyperactivity disorder: A systematic review. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 17.

Eisenberg JM, Zarin D. Health technology assessment in the United States. Past, present, and future. Int J Technol Assess Health Care. 2002 18: 192– 198.Google Scholar

Forbes C, Shirran L, Bagnall AM, Duffy S, ter Riet G. 2001. A rapid and systematic review of the clinical effectiveness and cost effectiveness of topotecan for ovarian cancer. York: NHS Centre for Reviews and Dissemination, University of York

Garber S, Ridgely MS, Taylor RS, Meili R. 2000. Managed care and the evaluation and adoption of emerging medical technologies. Santa Monica: Rand

Granados A, Jonsson E, Banta HD, et al. EUR-ASSESS project subgroup on dissemination and evaluation of impact. Int J Technol Assess Health Care. 1997 13: 220– 286.Google Scholar

Granados A. [The evaluation of medical technologies]. Med Clin (Barc). 1995 104: 581– 585.Google Scholar

Hagenfeldt K, Asua J, Bellucci S, et al. Systems for routine information sharing in HTA: Working group 2 report. Int J Technol Assess Health Care. 2002 18: 273– 320.Google Scholar

Hailey DM, Crowe BL. The influence of health technology assessment on the diffusion of MRI in Australia. Int J Technol Assess Health Care. 1993 9: 522– 529.Google Scholar

Hailey DM. The influence of technology assessment by advisory bodies on health policy and practice. Health Policy. 1993 25: 243– 253.Google Scholar

Henshall C, Oortwijn W, Stevens A, Granados A, Banta D. Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting subgroup of the EUR-ASSESS Project. Int J Technol Assess Health Care. 1997 13: 144– 185.Google Scholar

Husereau DR, Brady B, McGeer A. 2001. Oseltamivir for the treatment of suspected influenza: A clinical and economic assessment. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 21.

Institute of Medicine (IOM: Goodman C, ed.). 1988. Medical technology assessment directory. Washington: National Academy Press

Institute of Medicine (IOM: Mosteller F, ed.). 1985. Assessing medical technology. Washington: National Academy Press

Jacob R, McGregor M. Assessing the impact of health technology assessment. Int J Technol Assess Health Care. 1997 13: 68– 80.Google Scholar

Jobanputra P, Parry D, Fry-Smith A, Burls A. 2000. Effectiveness of autologous chondrocyte transplantation for hyaline cartilage defects in knees. Birmingham: West Midlands Development and Evaluation Service, The University of Birmingham

Lehoux P, Blume S. Technology assessment and the sociopolitics of health technologies. J Health Polit Policy Law. 2000 25: 1083– 1120.Google Scholar

Lewis R, Whiting P, ter Riet G, O'Meara S, Glanville J. 2000. A rapid systematic review of the clinical effectiveness and cost-effectiveness of debriding agents in treating surgical wounds healing by secondary intention. York: NHS Centre for Reviews and Dissemination, University of York

Lister-Sharp D, McDonagh M, Saeed Khan K, Kleijnen J. 2000. A systematic review of the effectiveness and cost-effectiveness of the taxanes used in the treatment of advanced breast and ovarian cancer. York: NHS Centre for Reviews and Dissemination, University of York

Lord J, Paisley S, Taylor R. 2000. The clinical effectiveness and cost-effectiveness of rosiglitazone for type 2 diabetes mellitus. London: National Institute for Clinical Excellence

Lord J, Paisley S. 2000. The clinical effectiveness and cost-effectiveness of methylphenidate for hyperactivity in childhood. London: NICE Appraisals Group

McDonagh MS, Bachmann LM, Golder S, Kleijnen J, ver Riet G. 2000. A systematic review of the clinical effectiveness and cost effectiveness of glycoprotein IIb/IIIa antagonists in the medical management of unstable angina. York: NHS Centre for Reviews and Dissemination, The University of York

McGahan L, Noorani HZ. 2000. Surveillance mammography after treatment for primary breast cancer. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

McGahan L. 2001. Behavioural interventions for preschool children with autism. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 18.

Meads C, Cummins C, Jolly K, Hyde C, Burls A. 1999. Coronary artery stents in the treatment of ischaemic heart disease. Birmingham: West Midlands Development and Evaluation Service, Department of Public Health & Epidemiology, University of Birmingham

Metge CJ, Vercaigne LM, Carrie A, Sarveiya V, Zhanel GG. 2001. The new fluoroquinolones in community-acquired pneumonia: Clinical and economic perspectives. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 20.

NICE. 2000. The appropriate use of proton pump inhibitors in the treatment of dyspepsia. London: National Institute for Clinical Excellence

NICE. 2000. The clinical effectiveness and cost-effectiveness of celecoxib, rofecoxib, meloxicam and etodolac (cox-II inhibitors) for rheumatoid arthritis and osteoarthritis. London: NICE Appraisal Team

Noorani H, McGahan L. 1999. Criteria for selection of adult recipients for heart, cadaveric kidney, and liver transplantation. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

Noorani H, Picot J. 2001. Assessment of videoconferencing in telehealth in Canada. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 14.

Noorani HZ, Connolly SJ, Talajic M, et al. 2000. Implantable cardioverter defibrillator (ICD) therapy for sudden cardiac death. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

Noorani HZ, McGahan L. 1999. Predictive genetic testing for breast and prostate cancer. Ottawa, Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

O'Meara S, Riemsma R, Shirran L, Mather L, ter Riet G. 2000. A systematic review of the clinical effectiveness and cost-effectiveness of sibutramine in the management of obesity. York: NHS Centre for Reviews and Dissemination, University of York

O'Meara S, Riemsma R, Shirran L, Mather L, ter Riet S. 2001. A systematic review of the clinical effectiveness and cost-effectiveness of orlistat in the management of obesity. York: NHS Centre for Reviews and Dissemination, University of York

Oortwijn WJ, Banta D, Vondeling H, Bouter L. Identification and priority setting for health technology assessment in The Netherlands: Actors and activities. Health Policy. 1999 47: 241– 253.Google Scholar

Oortwijn WJ, Vondeling H, Bouter L. The use of societal criteria in priority setting for health technology assessment. Int J Technol Assess Health Care. 1998 14: 226– 236.Google Scholar

Parkes J, Milne R, Bryant J. 2000. Implantable cardioverter defibrillator for cardiac arrhythmias. Southampton: The Rapid Reviews Team, National Coordinating Centre for Health Technology Assessment, University of Southampton

Paszat L, Grunfeld E, van Walraven C, et al. 2001. A population-based cohort study of surveillance mammography after treatment of primary breast cancer. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 15.

Payne N, Beard S, Brocklebank D, et al. 2000. Clinical and cost effectiveness of inhaler devices for children with chronic asthma. Sheffield: Trent Institute for Health Services Research, School of Health & Related Research (ScHARR), The University of Sheffield

Payne N, Chilcott J, McGoogan E. 2000. Liquid-based cytology in cervical screening. Sheffield: The University of Sheffield

Perry S, Eliastam M. The National Center for Health Care Technology. JAMA. 1981 245: 2510– 2511.Google Scholar

Perry S, Gardner E, Thamer M. Status of health technology assessment worldwide. Int J Technol Assess Health Care. 1997 13: 81– 98.Google Scholar

Perry S, Hanft R, Chrzanowski R. Report from the Australian Institute of Health and Welfare, Canberra, Health Technology division. Perceptions of Australian Health Technology Assessments: Report of a survey. Int J Technol Assess Health Care. 1993 9: 588– 590.Google Scholar

Perry S, Thamer M. Medical innovation and the critical role of health technology assessment. JAMA. 1999 282: 1869– 18672.Google Scholar

Rettig RA. 1997. Health care in transition: Technology assessment in the private sector. Santa Monica: Rand

Schachter HM, Kovesi T, Ducharme F, et al. 2001. The challenges of early assessment: Leukotriene receptor antagonists. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 19.

Scott DA, Clegg A, Sidhu M, Hewitson P, Waugh N. 2001. Clinical and cost-effectiveness of paclitaxel, docetaxel, gemcitabine and vinorelbine in lung cancer. Southampton: Rapid Reviews Team, Wessex Institute for Health Research and Development, University of Southampton

Shani S, Siebzehner MI, Luxenburg O, Shemer J. Setting priorities for the adoption of health technologies on a national level—the Israeli experience. Health Policy. 2000 54: 169– 185.Google Scholar

Shepherd J, Waugh N, Hewitson P. 2000. Combination therapy (interferon alfa and ribavirin) in the treatment of chronic hepatitis C. Southampton: Rapid Reviews Team, Wessex Institute for Health Research and Development, University of Southampton

Shukla VK, McAuley L. 2001. Novel antipsychotics in patients with bipolar disorder: A systematic review. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 16.

Shukla VK, Otten N, Coyle D. 2000. Drug treatments for Alzheimer's disease. III. A review of pharmacoeconomic evaluations. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA) Technology report no 11.

Shukla VK, Otten N, Dubé C, Moher D. 2000. Use of cisapride in patients with non-ulcer dyspepsia: A meta-analysis of randomized trials. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

Shukla VK, Otten N. 1999. Insulin lispro: A critical evaluation. Otawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

Song F, O'Meara S, Wilson P, Kleijnen J, Golder S. 1999. The effectiveness and cost-effectiveness of the prophylactic removal of wisdom teeth. York: NHS Centre for Reviews and Dissemination, University of York

Stewart A, Sandercock J, Bryan S, et al. 2000. The clinical effectiveness and cost-effectiveness of riluzole for motor neurone disease. Birmingham: West Midlands Development and Evaluation Service, The University of Birmingham

Tappenden P, Chilcott J, O'Hagan T, et al. 2001. Cost effectiveness of beta interferons and glatiramer acetate in the management of multiple sclerosis. Sheffield: Centre for Bayesian Statistics in Health Economics, School of Health and Related Research (ScHARR), University of Sheffield

Taylor R. Pharmaceutical regulation: The early experience of the NHS National Institute for Clinical Excellence (NICE) appraisal process—where are we heading? Value Health. 2001 4: 8– 11.Google Scholar

Taylor R. 2000. The effectiveness and cost-effectiveness of different prostheses for primary total hip replacement. London: NICE Appraisal Group

Taylor RS, Paisley S. 2000. The clinical and cost effectiveness of advances in hearing aid technology. London: National Institute for Clinical Excellence

Tunis SR, Kang JL. Improvements in Medicare coverage of new technology. Health Affairs. 2001 20: 83– 85.Google Scholar

VA Technology Assessment Program (VATAP). 2000. Computerized lower limb prostheses (Short Report). Boston: VA Technology Assessment Program

VA Technology Assessment Program (VATAP). 2000. Impacts of case management programs (Short Report). Boston: VA Technology Assessment Program

VA Technology Assessment Program (VATAP). 2000. Optical lens fabrication system (Short Report). Boston: VA Technology Assessment Program

VA Technology Assessment Program (VATAP). 1999. Positron emission tomography (Update). Boston: VA Technology Assessment Program

VA Technology Assessment Program (VATAP). 2000. Tablet Splitting (Short Report). Boston: VA Technology Assessment Program

VA Technology Assessment Program (VATAP). 2000. Treatment options for male erectile dysfunction: A systematic review of published studies of effectiveness. Boston: VA Technology Assessment Program

Vale L, McCormack K, Scott N, Grant A. 2000. Systematic review of the effectiveness and cost-effectiveness of laparoscopic versus open repair of inguinal hernia. Aberdeen: Health Services Research Unit and Health Economics Research Unit, University of Aberdeen

Vardulaki KA, Bennett-Lloyd BD, Parfitt J, et al. 2000. A systematic review of the effectiveness and cost-effectiveness of laparoscopic surgery for colorectal cancer. London: National Institute for Clinical Excellence

Wake B, Bryan S, Barton P, et al. 2001. Fluradarabine as second line therapy for b-cell chronic lymphocytic leukaemia. Birmingham: Department of Public Health and Epidemiology, and The Health Economics Facility, University of Birmingham

Ward S, Bansback N, Morris E, Calvert N. 2000. A review of the clinical and cost-effectiveness of gemcitabine for the treatment of pancreatic cancer. Sheffield: The School of Health and Related Research (ScHARR), University of Sheffield

Wolfson C, Moride Y, Perrault A, et al. 2000. Drug treatments for Alzheimer's disease. I. A comparative analysis of clinical trials. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

Wolfson C, Moride Y, Perrault A, et al. 2000. Drug treatments for Alzheimer's disease. 2. A review of outcome measures in clinical trials. Ottawa: Canadian Coordinating Office for Health Technology Assessment (CCOHTA)

World Health Organization. 1997–1999 World Health Statistics Annual, 25 February 2003. Available at: http://www3.who.int/whosis/mort/text/description.cfm?path=statistics,whsa,mort_info&language=english