Health technology assessment (HTA) is a form of multidisciplinary research that typically consists of the systematic examination of the safety, clinical efficacy and effectiveness, and cost-effectiveness of a technology, but that may also include the organizational implications, social consequences, and legal and ethical implications of its adoption and implementation (Reference Banta1;Reference Busse, Orvain and Velasco2). Although the completeness and rigor of HTAs vary in practice, the purpose of all HTAs is to inform policy decisions regarding the efficient, appropriate, and fair allocation of healthcare resources. Although there is a consensus that ethics analysis is important in HTA, there are questions about how to appropriately address the ethical issues that arise both in the conduct of an HTA and with respect to the technology being evaluated (Reference Arellano, Willett and Borry3).
Most large HTA agencies (both producers and users) declare they include ethical issues in the scope of their evaluative criteria for assessment (Reference Lehoux and Williams-Jones4). Nevertheless, researchers in countries with developed HTA programs have shown a discrepancy between these declarations and the presence of meaningful identification and analysis of ethical issues in HTA reports (Reference Arellano, Willett and Borry3–Reference Duthie and Bond6). Reasons for this discrepancy have included the difficulty of defining ethics in the context of HTA (Reference Ashcroft7), the problem of generalizing the findings of an ethics analysis (Reference Ashcroft7), lack of clear analytic methods (Reference Hofmann8), controversy regarding who has the authority to offer ethical expertise (Reference Ashcroft7), and agencies’ constraints on time and resources that prevent them from taking on the “additional” task of ethics analysis (Reference Lehoux, Tailliez, Denis and Hivon9).
Some researchers (Reference Lehoux and Williams-Jones4–Reference Duthie and Bond6) have proposed that the recognition of relevant expertise in ethics analysis and the continuing development of methods for analyzing ethical issues are crucial to conducting ethics analyses in HTAs more consistently, systematically, and rigorously than is done currently. However, HTA researchers and agencies have reported difficulty locating relevant experts and securing their participation in the HTA process (Reference DeJean, Giacomini, Schwartz and Miller5;10). In addition, even among HTA reports that have purported to provide a consideration of ethical issues, none included a primary analysis by an author reported to have ethical expertise (Reference DeJean, Giacomini, Schwartz and Miller5), despite many in the HTA community believing that such expertise ought to be involved (Reference Arellano, Willett and Borry3).
To help explore the difficulty in identifying and enlisting ethics experts in one jurisdiction (Canada), we aimed to identify individuals with expertise in applied ethics who might contribute to ethics analysis in HTA and to gauge these individuals’ familiarity with and experience participating in the production of HTA.
METHODS
The presence of HTA-relevant ethics expertise was assessed using a national cross-sectional study. The target population consisted of individuals with academic or other training and experience in applied or practical ethics who work in academic or other (e.g., government, consultancy) settings in Canada.
In conceptualizing the sampling frame, we took an expansive view of ethics expertise. Given several similarities between HTA and bioethics, which is one of the most developed fields of applied ethics, we assumed that bioethicists might have the expertise needed for HTA. In Canada, bioethics is not a profession with educational requirements for admission; rather, it is a multidisciplinary field that draws interested individuals from a variety of backgrounds, including philosophy, science and technology studies, medical humanities, theology, anthropology, sociology, medicine, nursing, and public health. In the absence of national databases, lists, or registers of potential ethics experts, we developed a sampling frame using the 2011 membership list of the Canadian Bioethics Society and by hand-searching listings of faculty members on Web sites of Canadian academic institutions, centers for applied ethics, and federal/provincial/territorial ministries of health. Individuals were added to the contact list if their title or online biographical sketch indicated potential expertise or research interests in ethics. Names and contact information of potential experts were also collected from HTA agencies, other researchers, and from survey participants.
The questionnaire (Table 1) was developed using questionnaires from similar surveys, and input from co-authors and collaborators (see Acknowledgments). The survey was piloted by a 5 percent sample of potential ethics experts (selected based on their affiliation with an established bioethics center or previous involvement in ethics analysis) and revised accordingly.
Table 1. Survey Questionnaire
The survey was conducted according to standard methods (Reference Dillman11). An initial contact email describing the survey content and its purpose was sent on December 5, 2011, to all potential participants (excluding those who had completed the pre-test survey). The first survey email was sent on December 27, 2011, with three reminder emails sent at regular intervals in January 2012. The questionnaire was accessed by means of Survey Monkey®, an online, self-administered survey software. Respondents working in the province of Québec (in which French is the primary language) were sent both French- and English-language versions of all survey materials.
All available data (including responses to the pilot survey) were summarized, regardless of the completeness of an individual survey. Categorical data were reported as percentages and continuous data were reported as medians and interquartile ranges. Free-text responses were summarized by grouping similar answers. A post hoc exploratory analysis was conducted to compare the level of education, location (province), and place of employment of those who had conducted ethics analysis for HTA with those who had not yet contributed, but who believed they might contribute to ethics analysis. Odds ratios and 95 percent confidence intervals were calculated using R (version 2.13.1, The R Foundation for Statistical Computing; 2011).
The survey protocol and questionnaire were approved by the University of Alberta Health Research Ethics Board.
RESULTS
Usable survey data were provided by 350 of 663 survey recipients (52.8 percent participation rate) and thirteen pilot survey respondents (Table 2).
Table 2. Respondent Characteristics: Employer, Education, and Training
a Twenty-four respondents (6.7%) indicated more than one highest degree.
Employment
Over half of the respondents reported working primarily in academic institutions (50.4 percent) or hospitals (15.4 percent), with fewer than 8 percent of respondents working in health authorities, government (federal or provincial) or quasi-government organizations, or religious organizations (including churches and religious-affiliated hospitals and universities).
Education and Work-Related Experience
Respondents indicated a wide variety of academic training for their highest degree. Over one quarter of respondents had their highest training in philosophy (28.2 percent), followed by medicine (14.0 percent), nursing (9.8 percent), sociology (8.9 percent), science and technology studies (5.3 percent), theology (4.7 percent), or anthropology (2.2 percent). One-third of the respondents (33.5 percent) obtained their highest degree in one of a variety of other health or non-health related disciplines.
Many respondents (83.1 percent) indicated having some form of education, formal training or work-related experience in health ethics. Most respondents indicated having a doctorate (34.5 percent) or “other formal training” (24.1 percent), for example, post-doctoral training, internships, coursework, work-related research, or residencies in clinical ethics.
Respondents reported a median of 10 years (interquartile range: 5 to 15) of work-related experience in applied ethics, including full-time work as an ethicist, university teaching or research in ethics, participating on research ethics boards, case consultation, or being a member of a hospital ethics committee.
Involvement in HTA
Some respondents (15.7 percent; n = 99) indicated that they had been involved in aspects of HTA other than ethics analysis as consultants (26.3 percent), reviewers (24.2 percent), primary researchers (17.2 percent), secondary researchers (16.2 percent), research assistants (4.0 percent), or “other” (12.1 percent).
Conducting Ethics Analysis
Almost one quarter (24.5 percent; n = 87) of respondents indicated having been involved in the analysis of ethical issues for HTA, 33.3 percent of whom had also been involved in an analysis for other aspects of HTA. Almost half (49.4 percent) of those who were involved in the analysis of ethical issues in HTA had doctoral training in ethics (62.7 percent of whom had received a doctorate in philosophy). Respondents had been involved predominantly as advisory committee or expert panel members (47.1 percent), as consultants (44.8 percent), peer reviewers (34.5 percent), or as primary researchers (21.8 percent) for HTAs to inform decision making by provincial (44.7 percent; n = 38) or federal governments (36.5 percent; n = 31), as well as for local institutions (28.2 percent; n = 24) and regional health authorities (23.5 percent; n = 20).
Potential Contributions to Analysis of Ethical Issues
Based on the description of HTA that was provided in the questionnaire, 65.4 percent (n = 165) of respondents believed their education, training, or work experience would enable them to usefully contribute to an analysis of ethical issues in HTA, even though they had not already done so. Few respondents indicated the nature of their potential contribution, for example, philosophical analysis or empirical study. Some respondents indicated their expertise might contribute to areas important to, but separate from, ethics analysis, such as social and legal issues, or facilitating communication between participants when discussing ethical issues.
Respondents who conducted ethics analysis had more than twice the odds of being formally educated and trained in applied ethics (odds ratio [OR], 2.63; 95 percent confidence interval [CI], 1.01–8.13) compared with persons who indicated they could contribute to ethics analysis. As well, respondents who had conducted ethics analysis had twice the odds of possessing doctoral level training (OR, 2.0; 95 percent CI, 1.16–3.61) compared with persons who indicated they could contribute to ethics analysis.
DISCUSSION
Our findings are a necessary first step toward improving the rigor, relevance, and comprehensiveness of HTA, through an attempt to identify potential ethics experts in Canada who are able and willing to work with HTA organizations to conduct ethics analyses. The results of this survey suggest there is a potentially large number of experts available to do ethics work in HTA. This sits in contrast with the difficulty HTA organizations have typically reported in enlisting the resources required to conduct ethics analysis. Another apparent inconsistency lies in the frequency with which ethics analysis is conducted for HTA. Almost one quarter of respondents indicated having been involved in the analysis of ethical issues for HTA, which suggests that there is more ethics analysis in HTA than is evident from the HTA reports themselves. Although disagreement about what constitutes ethics analysis in HTA may help explain the finding (that is, respondents may have had a more expansive view of what qualifies as ethics analysis than we did), it may also be that, for whatever reason, ethics analyses are not being made public alongside the clinical and economic assessments.
To our knowledge, this is the largest survey of its kind. The participation rate of this survey, 52.8 percent, was lower than the 72 percent response rate of a similar pan-Canadian survey (Reference Coughlin and Watts12) of 350 healthcare ethics consultants in Canada. Although the target population of that earlier survey was broad, the multidisciplinary nature of HTA reflected in the range of expertise sought in this survey resulted in an even broader target population, which likely explains the lower response rate.
Previous research has found that even when HTA agencies are genuinely interested in integrating ethics analysis, their ability to do so in a coordinated and coherent manner is undermined by a lack of human resources (Reference Lehoux, Tailliez, Denis and Hivon9). The comparison of the characteristics of those who have conducted ethics analysis with those who indicated they could contribute to such an analysis shows these potential experts are located in regions and institutions familiar to HTA agencies. The high proportion of those who are already participating in aspects of HTA other than ethics analysis, and who indicated they could contribute to ethics analysis, suggests that future capacity for ethics analysis could build on existing HTA capacity. Although these results reflect the expertise available in a single country, we suspect HTA producers in other countries with developed HTA programs may find themselves in similar situations (Reference Arellano, Willett and Borry3;10).
The fact that those with ethics expertise might contribute to areas important to, but separate from, ethics analysis suggests that recruiting ethics experts has the potential to increase capacity in other areas of HTA as well. At the very least, the ability for many with experience and training in ethics to facilitate communication between those involved in the HTA process when discussing complex and contentious ethical issues could be of benefit for improving the quality and rigor of HTA (Reference Duthie and Bond6).
Allowing respondents to self-identify as being able to contribute to ethics analysis has resulted in variability in respondents’ characteristics. We believe an important outcome of this study is a better understanding of the degree of variation among those who consider themselves able to contribute in an informed way to ethics analysis in HTA. Two of us (K.B., K.D.) have proposed elsewhere that “expertise” ought to be understood as a capacity to provide strong justifications for claims in a domain (Reference Duthie and Bond6). In this conception, “ethics experts” are individuals who possess particular content and methodological expertise related to the domain of ethics (Reference DeGracia, Beauchamp, Sugarman and Sulmasy13;Reference Rasmussen14). Ethics experts in HTA ought to be seen as “geographers of morally relevant facts and values” (Reference Kovács15), possessing specialized conceptual tools and a broad knowledge of the potential issues that allow such experts to provide extensive and deep (rich) ethics analyses. This expertise may also help to ensure that an HTA's results will be valid and useful to stakeholders (Reference Duthie and Bond6). Indeed, the expertise needed to adequately conduct ethics analysis requires, at a minimum, a knowledge of the relevant policy context and a facility with methods of ethical reasoning. This is not to suggest that this knowledge and skill is sufficient for all cases, only that it is necessary. To do either clinical ethics or ethics in HTA requires the ethicist to become familiar with subject matter other than ethics and moral philosophy. Just as the clinical ethicist must become broadly familiar with medicine and health care; the ethicist working in HTA must understand and be familiar with various aspects of health research and technology assessment. Furthermore, the ethicist is often called upon to show the relevance of ethics to the task at hand, and must be able to demonstrate how ethics processes and methods can fit in with the clinical or HTA work. This latter piece requires that the ethicist develop an additional set of skills. Additionally, it may very well be useful or methodologically necessary, for example, for primary empirical research, to enlist additional expertise. This latter point is especially important given the division in the HTA community about whether HTA should provide a normative or descriptive ethics analysis (Reference Arellano, Willett and Borry3).
The results of our study place the issue of expertise into sharp relief. The range of formal ethics education among respondents was quite wide, from doctoral-level education in a clearly relevant discipline to individuals whose knowledge is gained on the job or through workshops or seminars. It is likely that in other countries, as in Canada, bioethicists have varied backgrounds and do not have educational requirements for admission to the profession. Hence, the information provided here regarding the location of experts and the issues in assessing the relevance of expertise will hold for those in other jurisdictions.
One cause for concern is that many who have little to no formal education in ethics believe they could contribute to ethics analysis in HTA. In light of this fact, self-report of ability to contribute to ethics analysis is unlikely to be a sufficient indicator of ability, and a method is needed to identify those who have the ethics expertise necessary for HTA. Before such a method can be created, however, there needs to be a broad consensus on what competencies are required. The general competency in ethics analysis is already identified. Other competencies that have not been discussed, but for which some consensus might be desirable, include a basic understanding of the relevant scientific, clinical, and economic literature used in HTA, familiarity with policy and policy processes, ability to engage with the ethics literature, an understanding of the connections between normative and empirical ethics, and an ability to produce concise philosophical writing within short timelines. Until a consensus about the full set of competencies is achieved, we believe it is reasonable for agencies to use formal education in ethics as an indicator of competence, albeit a fallible one, in ethics analysis.
Some comfort may be taken from the fact that a large majority of those who have participated in ethics analysis in HTA have considerable formal education, with nearly half possessing a PhD in philosophy. Also, experts who have conducted ethics analysis had more than twice the odds of having formal education and training in applied ethics and doctoral level training than those who said they might contribute to ethics analysis. This finding suggests that agencies conducting HTA do scrutinize the qualifications of self-professed experts. Nevertheless, the question remains regarding the level of education and training that is sufficient to reliably produce a high-quality ethics analysis. This question seems especially salient given the many master's level programs aiming to impart expertise in bioethics.
Our survey was concerned with the pool of people who might be drawn upon for ethics analysis, not with identifying an ideal set of expertise. Even so, our study results highlight the question of the competencies needed for ethics analysis in HTA. The question of what constitutes “expertise” in ethics analysis mirrors a similar question regarding clinical ethics consultants (Reference Rasmussen14) and bioethicists more generally. As the field of bioethics and the role of health ethicists continues to evolve, the question of core competencies generates dialogue in Canada (Reference Coughlin and Watts12;16) and internationally (17;Reference Sacchini18).
Once standards of competence for ethics analysis in HTA are developed, HTA agencies might use them to identify the relevant expertise. In the meantime, HTA agencies might first look to bioethics associations and begin to identify the relevant expertise.
Strengths and Limitations
We made minimal assumptions about the expertise and experience required to contribute to an ethics analysis. This is both a strength and weakness. The Canadian Bioethics Society, the membership list of which constituted the primary source of potential study participants, is composed of individuals interested in sharing ideas related to bioethics and is not a professional organization; there is no guarantee that a given member has expertise in ethics. Nevertheless, as our survey aimed to capture ethicists whose primary vocational engagements are academic scholarly work, post-graduate education, or administration of Research Ethics Boards, as well as practicing healthcare ethicists, our broadly inclusive contact list is more likely to have identified the range of people working in applied ethics, or who may be able to contribute to future ethics analyses in HTA than is one based on more narrowly defined expertise. Moreover, even if one accepts a restrictive definition of expertise, there still seem to be many experts in Canada available for conducting ethics analysis in HTA.
The lack of a paper-based questionnaire may also be seen by some as a potential weakness of this survey. However, the ease of replying to email may have been an encouragement to participate and, given the short length of time required to complete the survey, it is unlikely that a paper-based questionnaire would have dramatically improved participation (Reference Shih and Fan19). Finally, the questionnaire did not ask specifically whether a potential expert's contribution could be to philosophical or empirical ethics work, and few respondents indicated the nature of their potential contribution. As a result, we have an incomplete picture of the potential contributions respondents might make to ethics analysis. Acknowledging these potential limitations, we believe this study provides a useful model for others to assess the potential capacity for ethics analysis in their own jurisdictions and to promote discussion of the requisite core competencies for ethics analysis in HTA.
CONCLUSIONS
A lack of available expertise is unlikely to be a barrier to conducting ethics analysis in HTA in the Canadian context, although identifying expertise is likely to be a continuing problem. Future research could focus on identifying and addressing barriers to enlisting qualified ethics experts for HTA and on identifying core competencies for conducting ethics analysis in HTA. In addition, given the absence of a reliable credential for ethics expertise, HTA agencies should exercise caution when enlisting ethics experts.
CONTACT INFORMATION
Kenneth Bond, MA (kbond@ihe.ca), Research Associate, Institute of Health Economics, #1200, 10405 Jasper Ave., Edmonton, Alberta, Canada T5J 3N4
Mark Oremus, PhD, McLaughlin Foundation and Associate Professor, Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4L8
Katherine M. Duthie, PhD, Leader Clinical Ethics, Fraser Health Ethics Services, #400, 13450 102nd Ave., Surrey, British Columbia, Canada, V3T 0H1
Glenn G. Griener, PhD, Associate Professor, Department of Philosophy, University of Alberta, Assiniboia Hall 2-40, Edmonton, Alberta, Canada, T6G 2E7
CONFLICTS OF INTEREST
K. Bond declares Grant to institution from Canadian Agency for Drugs and Technologies in Health. Funding provided through the Canadian HTA Exchange for methods work related to HTA. Declared in “Financial support” section of manuscript. The other authors declare no financial or non-financial conflicts of interest.
