Current developments in heath technology assessment (HTA) reflect the demand to bring a more contextual perspective to the carrying out of HTA processes and analyses. This would include taking account of patients’, citizens’, and stakeholders’ own perspectives, and ethical evaluations. There is a consensus on the importance of ethical analysis as a part of HTA (Reference ten Have1–Reference Hofmann3). Integrating ethical issues into HTA can be useful in dissemination, decision making, and policy making (Reference Hofmann4). Yet, analysis of ethical issues rarely figures in HTA reports. Dejean et al. (Reference DeJean, Giacomini, Schwartz and Miller5) reported that only 17 percent of reports (n = 680) produced by six Canadian agencies between 1997 and 2006 addressed ethical issues in their analyses.
A survey of 223 reports produced by nine agencies (Canada, United Kingdom, Denmark, and United States) showed that only 5 percent of reports consider ethical, social and organizational issues in their HTA production (Reference Lavis, Wilson and Grimshaw6). It is known, as well, that novel technologies may give rise to specific ethical and moral issues. HTA reports may thus be less useful for decision making if they fail to consider the ethical issues that might underlie certain kinds of decisions or if they do not take account of the moral conflicts that could have an impact on the dissemination and implementation of technologies (Reference Burls, Caron and Cleret de Langavant7).
Despite the recognized importance of integrating ethics into HTA, the multiplicity of methods of ethical analysis and the need for them to be comprehensive to be applied systematically into HTA remains challenging for HTA producers (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree8;Reference Hofmann9). In 2014, Assasi et al. worked to identify existing guidances for the integration of ethics into HTA (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10). Their systematic review identified forty-three conceptual frameworks or practical guidelines. They found significant variation and diversity in terms of philosophical approach, structure, methodology, and comprehensiveness. These variations seemed to reflect the differences among healthcare systems and differences in the degree of proximity between agencies and decision-making bodies (Reference Garrido, Amaro and Cichietti11). There is a lack of consensus on a practical method for assessing ethical issues in HTA (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10;Reference Saarni, Hofmann and Lampe12–Reference Arellano, Willett and Borry14), as well as limited information about the appropriate scope and level of detailing in ethical issues analysis (Reference Saarni, Hofmann and Lampe12;Reference Duthie and Bond15). Methodological choices for collecting and analyzing ethical data seemed to depend on context, the purpose of the analysis, and the availability of resources (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10). While some frameworks for analysis were designed to fit particular contexts, others appeared more generic (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10).
Ethical analyses using existing frameworks are generally done by a professional ethicist or an HTA practitioner with skills and knowledge in ethics (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10). In practice, integrating ethics into the scientific process of HTA remains challenging. Few guidelines, meta-methodologies, frameworks, checklists, or stakeholder-engagement procedures address methodological issues related to ethical evaluation in HTA (Reference Hofmann4;16–Reference Hofmann20). These documents provide no clear guidance. For example, Hofmann (Reference Hofmann4) proposes a set of thirty-three questions for examining moral issues in HTA. Those questions are neither exclusive nor exhaustive. They can be used as a checklist for highlighting moral issues during the HTA process; but Hofmann proposes no operational methods for conducting an ethical analysis, whether with or without the involvement of ethical experts. The main criticisms HTA practitioners direct at all these different approaches have to do with difficulties in operationalization and a lack of organizational expertise and resources (Reference Assasi, Tarride, O'Reilly and Schwartz21).
In response to these problems, in 2016, Assasi et al. (Reference Assasi, Tarride, O'Reilly and Schwartz21) proposed a new framework consisting of three components: a decisional algorithm, a stepwise guide, and a list of tools recommended for use in ethical analysis. While this framework represents a promising avenue to support and promote good practices for including ethics in HTA processes, Assasi et al. conclude that it will require further validation through case studies and expert consultations to establish its practical utility.
Other concerns regarding the best conceptual and methodological approaches to integrating ethics into HTA have been raised by various authors. Hofmann et al. (Reference Hofmann, Oortwijn, Bakke Lysdahl and Schwartz22) tried to identify approaches appropriate for integration into HTA. They found the literature features a wide range of stances, methods, and approaches to how ethical issues should be considered in HTA. They concluded there is no universal method for integrating ethical issues into HTA. Saarni et al. (Reference Saarni, Hofmann and Lampe12) and Assasi et al. (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10) presented approaches such as Casuistry, coherence analysis, Principlism, participatory HTA approaches, social shaping of technology and wide reflective equilibrium. They did not, however, find a way of identifying a single shared concept between these approaches.
Finally, our literature review found no analysis comparing the impact of the diversity of the various approaches’ ethical concepts and ethical background theories.
As reported by Hofmann et al. 2014 (Reference Hofmann9), ethics can be integrated into HTA in four ways: (1) ethical analysis constitutes a subsidiary activity requiring a separate chapter of the HTA report; (2) ethical analysis constitutes a separate part or process of the HTA, on an equal footing with the other dimensions (e.g., efficacy, safety, effectiveness); (3) ethical analysis constitutes a separate part or process of the HTA, but its role and importance may depend on the context; and (4) ethical analysis actively interacts with and alters other parts of the HTA process. Those four possible forms of integration were considered in our analysis.
Uncertainty remains about what constitutes a suitable scope and level of details for an ethical framework that is to be applied in HTA (Reference Saarni, Hofmann and Lampe12;Reference Duthie and Bond15;Reference Lehoux and Williams-Jones23;Reference Saarni, Braunack-Mayer, Hofmann and van der Wilt24). Several published studies have aimed to assess methods of ethical analysis applied in HTA (Reference Saarni, Hofmann and Lampe12;Reference Duthie and Bond15;Reference Lehoux and Williams-Jones23;Reference Potter, Avard and Graham25). While these studies frequently use specific methodological approaches in considering ethical issues, none of them has critically evaluated the characteristics (e.g., frameworks, resource requirements) of such methods (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10).
As a starting point for our study entitled “Theoretical and practical operationalization of the integration of ethics into the process of Health Technology Assessment (HTA)” (CIHR no. 142187), we decided to systematically examine the literature to: (1) better understand the reasons for integrating ethics into HTA, which we referred to as a diagnosis of the social needs for integrating ethics into HTA; (2) determine the obstacles to ethical integration; (3) identify the concepts and processes deployed in ethical evaluation, and more specifically, value judgments. The main objective of this study is to identify issues challenging the integration of ethics into HTA. That is why we systematically reviewed critical analyses of formal experimentations of ethical evaluation in HTA.
METHODS
We conducted a systematic review using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (Reference Moher, Liberati, Tetzlaff, Altman and Grp26). A research protocol was established before the start of our study but was not registered in a database of systematic reviews (e.g., The National Institute for Health Research Database PROSPERO).
Screening of citations, full-text articles, and references of eligible articles, as well as data extraction and analysis, were completed by four independent reviewers. Two subgroups of two reviewers were created. The first group was composed of HTA producers (C.A.B. and S.K.B.), while the second consisted of ethics/philosophy experts (J.P. and G.A.L.). Each methodological step was performed by both groups independently. Results within groups were then compared and discussed until a consensus was reached. The comparison exercise was repeated between the groups. Discordances within and between groups were arbitrated by a referee (P.D.).
Search Strategy
The search strategy was designed and implemented by one information scientist and validated by another. No limitations were applied for date and language. Search terms included “ethic,” “technology assessment,” and “HTA”. The literature search was performed to November 21, 2016 in Medline/Ovid, SCOPUS, CINAHL, PsycINFO, and Canadian and the international HTA Databases (http://www.crd.york.ac.uk/PanHTA/homepage.asp). The result was imported into the bibliographic management software EndNote X7.0.1 (Bld 7212) and duplicates were removed. A detailed search strategy including MeSH terms for each database is presented in Supplementary Search Strategies.
Screening of Citations
The bank of citations obtained according to the search strategy was reviewed. Criteria applied to confirm the inclusion of a citation was relevance to ethics in the field of health technology assessment. Citations of case analysis, technology assessment outside healthcare, and HTA with legal assessment without specific ethical assessment were discarded.
Screening of Full-Text Articles
Full-text articles pertaining to the citations retained were obtained and fully reviewed. Articles were included according to the same criteria applied in the first screening of citations.
Thus, to be included, articles were required to provide data addressing at least one of the categories of information described in our objectives. Articles were excluded with reasons and a final list of eligible articles was retained. Conflicts between reviewers were discussed and consensus was reached as previously described. Eligible articles’ references were screened for citations not obtained from database searches. Retained citations and full-text articles were screened using the same methods.
Quality Assessment of Eligible Articles
Given the aim of this review, which focused on descriptive studies, narrative and philosophical documents, and the qualitative nature of this exercise, neither specific nor generic quality assessment tools were considered relevant or useful for this systematic review. No criteria were used to reject or retain authors’ opinions and perspectives as we were interested in the full diversity of concepts and ideas related to our research objective.
Data Extraction and Analysis
Data extraction from eligible articles was performed in two steps by two independent reviewers.
The first step consisted in extracting from each article text sections or paragraphs corresponding to the data we wished to collect and grouping them into distinct working documents. Then, a second extraction was performed using an analytical qualitative method known as the general inductive method (Reference Thomas27). This generated codification categories that were reported in tables as previously described (Reference Pedersen, Dagenais and Lehoux28).
RESULTS
Study Flow
The literature-search strategy revealed 1,646 references after the removal of duplicates. Of this number, 132 references were selected for complete eligibility assessment. Of these, sixty-seven were selected for analysis (Supplementary Included Studies [n = 67]). Sixty-five studies were excluded from the final selection for the following reasons (Supplementary Excluded Studies with reasons [n = 65]): absence of reference to integration of ethics in HTA (n = 12); HTA topics without an ethical dimension (n = 2); technology assessment not in health (n = 4); case analysis (n = 2); article not in HTA (n = 6); basic description of HTA (n = 1); language: German (n = 8), Dutch (n = 1), Spanish (n = 4); and lack of information for data extraction (n = 25). Conflicts between the researchers concerning exclusions were discussed and a consensus achieved without the need for a referee. A summary of the selection of articles is provided in a PRISMA flow diagram shown in Supplementary Figure 1.
Tables 1–3 present the main findings grouped by research issues, while Table 4 presents findings about critical studies on formal experimentation of ethical evaluation in HTA. Their complete reference/citation list (n = 67) is shown in the Supplementary Included Studies file and is numbered 1–67 independently from references cited herein.
Table 1. Reasons for Integrating Ethics into HTA
a The complete reference/citation list is shown in the Supplementary Included Studies (n = 67) file and is numbered 1–67 independently from references cited herein.
Table 2. Obstacles to Ethical Integration into HTA
a The complete reference/citation list is shown in the Supplementary Included Studies (n = 67) file and is numbered 1–67 independently from references cited herein.
HTA, health technology assessment.
Table 3. Concepts and Processes Deployed in Ethical Evaluation in HTA (Value Judgments)
a The complete reference/citation list is shown in the Supplementary Included Studies (n = 67) file and is numbered 1–67 independently from references cited herein.
HTA, health technology assessment.
Table 4. Critical Studies on Experimentation of Ethical Evaluation in HTA
a The complete reference/citation list is shown in the Supplementary Included Studies (n = 67) file and is numbered 1–67 independently from references cited herein.
HTA, health technology assessment.
Reasons for Integrating Ethics into HTA (Social Needs)
As shown in Table 1, the most frequently reported social need for ethics in HTA is to inform policy decision making, as these decisions should be based on ethical principles aimed at improving the general health and wellbeing of society. In several more recent studies, the role played in empowerment for individual health decisions by the act of informing patients is also reported as an ethical consideration, one related to the autonomy principle of Principlism. Transparency and accountability, a governance issue with an impact on decision makers, was another important ethical dimension of HTA identified as a social need.
Obstacles to Ethical Integration into HTA
Table 2 presents the most frequently cited barriers to including ethical analysis into HTA. Our results showed that the major barrier resides in the nature of ethical analysis, because there are no agreed procedures for ethical analysis and there is a lack of consensus on the role of ethical theory and ethical expertise. The qualitative nature of ethical analysis based on value judgments raises difficulties and contrasts to the largely shared experimental paradigm. These findings are in line with the aforementioned social needs, as we also identified the absence of a clear-cut policy for ethics and the problem of translating the results of ethical analysis into decision making process. Feasibility issues related to HTA producers’ competency to carry out ethical analyses were also identified. HTA professionals’ typical predisposition in favor of the paradigm of the experimental sciences, along with certain prejudices regarding ethical analysis, a practice anchored in the human sciences' qualitative paradigm, were also reported as possible obstacles to the integration of ethics into HTA.
Concepts and Processes Deployed in Ethical Evaluation (Value Judgments)
Our systematic review yielded similar results to the findings of Assasi et al. concerning the nine approaches to ethical analysis found in HTAs: Principlism, Casuistry, coherence analysis, wide reflexive equilibrium, axiology, Socratic approach, triangular method, constructive technology assessment, and the social shaping of technology. In Table 3, we list only the principal values that were referred to in explicit value judgments in HTA analyses. Taking into account the context of utterance, in the comments section of the table we clarify the normative force of the values referred to. Fairness/equity were the values most mentioned; they relate to one of the expectations that underlie the social needs for ethical analysis. Values embedded in Principlism were also often invoked in ethical evaluation: autonomy, beneficence and nonmaleficence, and justice. We found no specific processes explicating how value judgments are elaborated and applied to a particular case.
Critical Analyses of Experimentation of Ethical Evaluation in HTA
Only three critical analysis studies on experimentations of ethical evaluation in HTA were found in the literature (Table 4). The Evidence and Value: Impact on Decision Making (EVIDEM) framework was applied by HTA producers in the assessment of growth hormone use in Turner's syndrome. Panelist found the framework to be an appropriate tool for ensuring awareness and explicitness regarding all historical, political, system-related, and ethical elements that may impact the decision (Reference Goetghebeur, Wagner, Khoury, Rindress, Gregoire and Deal29).
The Aristotelian-Thomistic ethical approach was used for decision making regarding the use of the Electronic Medical Diary (EMD) (Reference Sacchini, Refolo and Virdis30). According to the authors this approach was not adequate to address the complexities of EMD implementation.
Another tool for integrating ethics was tested and critically analyzed in an international collaborative study and was not found useful for selecting assessment topics, the formulation of the research questions, or the organization of the overall health technology assessment process (Reference Saarni, Hofmann and Lampe12).
DISCUSSION
The aim of this systematic review was to identify issues challenging the integration of ethics into HTA. According to our research protocol, we systematically recorded from the scientific literature the reasons for integrating ethics into HTA, the obstacles to ethical integration into HTA, the concepts and processes deployed in ethical evaluation, and more specifically, value judgments, as well as critical analyses of formal experimentations of ethical evaluation in HTA.
Among reasons or social needs for integrating ethics into HTA, “to inform policy decision making” was the most frequently reported. This is an important component of the HTA concept which is considered a value-laden activity because its development is defined as: “a multidisciplinary process that summarizes information about the medical, social, economic, and ethical issues related to the use of a health technology in a systematic, transparent, unbiased, and robust manner. Its aim is to inform the formulation of safe, effective, health policies that are patient focused and seek to achieve best value” (31). Decision making suggests value judgments, but could also mean weighting conflicting value judgments
Transparency was also among the most often reported reasons for the use of ethics. When we link “transparency” to a social need, we refer to different democratic expectations about health technology decisions: that is, the reasons underlying the relative weighting of conflicting value judgments must be made explicit. This is corroborated by the requirement of accountability for reasonableness (Reference Daniels, Porteny and Urrutia32;Reference Daniels and van der Wilt33). These expectations impact the way assessment reporting is done. If an assessment is destined exclusively for a governmental agency's decision-making body, the need for transparency may differ from that entailed by reporting to patients and citizens for their use in their own individual health-decision making. For citizens, transparency means knowing on what grounds socioeconomic investments in health are being made; this is a condition of accountability to the general public. To meet the expectations of transparency for patients and citizens, an assessment must broaden its scope and integrate more contextual elements than when it is based solely on safety, clinical effectiveness, and cost-benefit analysis.
The main difficulty reported in integrating of ethics into HTA was the seeming absence of a shared standard model for ethical analysis in the HTA community and a certain lack of consensus on the role of ethical theory and ethical expertise. A wide diversity of methods was found in the literature. The nine most commonly cited being Principlism, Casuistry, coherence analysis, wide reflexive equilibrium, axiology, Socratic approach, triangular method, constructive technology assessment, and social shaping of technology. Furthermore, there exists no shared underlying model, a situation that is revealed at two levels. For some, the process of ethical analysis must follow the same methodological procedure as the assessment of safety or clinical effectiveness based on systematic reviews of the literature.
Moreover, identifying and analyzing ethical issues are considered part of HTA, as a complementary domain. For others, ethical analysis is an evaluative process incorporated into HTA as a requirement for formulating recommendations. As well at a theoretical level, ethical analyses vary in the ethical concepts they convey, as shown by the diversity of approaches. Authors do not identify the exact nature of these concepts (e.g., whether they consist of social values, moral norms, or value judgments). In absence of clarity about their exact nature, it becomes difficult to achieve transparency in ethical analysis.
Another obstacle identified was the scarcity of information concerning an operationalized procedure that would clarify how an ethical evaluation is carried out for a given technology. To apply a general principle to a particular case involves a practical ethical (or moral) reasoning. These limitations of ethical analysis may have an impact on the fulfillment of the expectations associated with the social needs that underlie ethical integration, namely transparency and accountability.
Only three published formal studies were found regarding critical analysis of experimentations of ethical evaluation in HTA. The first used the EVIDEM framework in an assessment of growth hormone for use in Turner's syndrome. Goetghebeur et al. 2010 (Reference Goetghebeur, Wagner, Khoury, Rindress, Gregoire and Deal29) found that greater awareness and explicitness regarding all the historical, political, systems-related, and ethical elements that may impact a decision can be achieved with this model. The second study, the Aristotelian-Thomistic ethical approach was not found to be useful for improving ethical analysis for an information technology implementation. In the third, ethical evaluation was tested in an international collaborative study (Reference Saarni, Hofmann and Lampe12). The study reported that ethical considerations were not found useful in selecting assessment topics, formulating research questions, or organizing the overall health technology assessment process. Therefore, more critical studies on real cases might make it possible to highlight the practicability of current methods. The lack of critical analyses of formal experimentation with the proposed frameworks suggests that more work is needed to improve existing tools, as well as for gaining better knowledge of the needs related to decision-making processes.
When ethical analysis in HTA is evaluative and not just descriptive, it rests on value judgments. A close look at the thirteen values invoked in value judgments showed a correlation with differing expectations about ethical evaluation and the differing conceptions of ethics that underlie these values. The social need for transparency in policy making requires ethical evaluation regarding fairness/equity and justice/efficiency. These referential values are at the core of accountability for reasonableness. At a clinical level, the values commonly associated with Principlism are at the forefront: autonomy and beneficence and nonmaleficence.
Other values invoked are associated with moral theories: Good life (Aristotle), Human dignity (Kant), Human integrity (Thomas of Aquinas), Utility (Mill), Need and Satisfaction (emotivist theory of ethics). Because the quality of evaluative ethical analysis rests on the way value judgments are made and the weighting of those value judgments during assessment, it seems reasonable to expect a clear account of how they are made. The fact that few indications exist about how to clarify what constitutes a valued outcome seems to be at the core of our difficulty in adequately grounding ethical analysis in reasons; and thus satisfying the expectations associated with the social needs that drive ethical analysis: that it will support decision makers in reaching their final decision.
While Assisi et al. focus on disparities between practical concepts and methods for ethics analysis in HTA, we were more concerned with the fundamental issues of social finality of HTA and the role of value judgments during the evaluative process. In an initial publication, they have systematically examined the available methodological guidances for evaluating ethical considerations in HTA (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10). Their subsequent publication aimed at exploring, from different aspects, barriers and facilitators influencing ethical evaluation in HTA (Reference Assasi, Schwartz, Tarride, O'Reilly and Goeree8). In their systematic review, the scarcity, heterogeneity, and complexity of ethical analysis methods; challenges in translating the result from ethical analysis into knowledge that is useful for decision makers; and the lack of organizational support in terms of required expertise, time, and financial resources, were the barriers most often cited. Use of values-based appraisal methods, stakeholder and public engagement, the enhancement of practice guidelines, ethical expertise, and educational interventions were the facilitators most often cited.
We decided to perform this systematic review in complementarity with the work of Assasi and colleagues. We covered fundamental questions about the social needs that drive efforts to integrate ethics into HTA, the diversity of philosophical positioning, and the processes proposed in the literature and explored systematically the critical studies on formal experimentations of ethical evaluation in HTA. The small number of studies found in our systematic review that reported results of critical analyses indicates a real issue in terms of critical assessment of the available methods for integrating ethics into HTA. This chimes with the finding of Assasi et al. (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10) in 2014 that evidence on how methods for ethical analysis might be effectively used in HTA is limited. While they performed a thorough identification and review of existing frameworks, Assasi et al. (Reference Assasi, Schwartz, Tarride, Campbell and Goeree10) did not compare methods nor assess their operationalization; their applicability, or the impact of ethical assessment on decision making processes, and more specifically on value judgments. Our results regarding the difficulties of integrating ethics into HTA were similar to those of Assasi et al.
After analyzing our data, we hypothesize that there exists a form of interdependency between the social, philosophical and conceptual aspects of ethics integration into HTA. We think that the bases of this relationship are values and value judgments, concepts shared between these different integrative aspects.
Our systematic review was performed without any language restriction. However, studies in German, Dutch, and Spanish were excluded from data extraction. The authors are nonetheless sensitive to the fact that these articles (a total of 15 studies) may contain information relevant to our study. Because theme saturation was achieved within the final selection of the sixty-seven studies, as described in the results section, we are confident the studies discarded for language reasons would not have affected our conclusions.
Our method used two subgroups of two reviewers (health technology assessment experts and ethics and philosophy experts) and a two-steps validation process within groups. This methodological choice was adopted to arrive at a consensual qualitative analysis minimizing individual subjectivities. The strength of our multidisciplinary team's procedure, increases the inter-observer validation of the extracted data, resulting in an improved credibility and transferability for our reported results.
In conclusion, the main reasons for integrating ethics into HTA are related to the policy-decision making process in health care. On one hand, decisions makers, if they are to obtain the most complete picture of the HTA issues, must include ethical considerations in that picture, because every decision rests on value judgments. On the other hand, patients’ expectations and citizens’ concerns about resource allocation entails transparency regarding the final decision. HTA reports should take into account value judgments, during the assessment process, which constitutes a major difficulty for integrating ethics into HTA.
Our analysis leads us to believe that there exists a form of interdependency between the three issues (social needs, obstacles, concept, and processes) discussed in this work and that value judgments could be their linking concept. This dependency may explain the challenges for integrating ethics into HTA. Based on our research protocol (CIHR no. 142187), a better understanding of this relationship of dependency of the issues will allow us to step further in the development of an integrated method into HTA, addressing social needs and practitioner's expectations.
Compared with the scientific experimental paradigm, there are no settled proceedings for ethics in HTA nor consensus on the role of ethical theory and ethical expertise hindering its integration. As reported by Refolo et al. (Reference Refolo, Sacchini and Brereton34), epistemological reasons may explain the difference in interpreting ethical analysis and the difficulty of its integration into HTA. In HTA, empirically testable methods from natural sciences can be used for some dimensions of the assessment (e.g., safety, effectiveness, efficacy) while others are dealing with meaning and are not empirically testable (e.g., ethical, socio-cultural, or legal domains). At a more fundamental level, each theoretical approach yields a different set of values serving as the basis for ethical evaluation; yet the processes involved in forming a value judgment and weighting conflicting values was, in our view, never clearly addressed in the literature we examined. This may account for the fact that evaluative ethical analysis is not recognized as an essential part of the HTA process, one capable of providing decision makers with pertinent and well-founded information.
Given the importance of value judgments in decision making, a better understanding is needed of the role of value judgments in descriptive and evaluative ethical analysis. Furthermore, because HTA is values-laden, clarifying the role of value judgments in the multiple decisions that feature in the assessment process would shed light on the conjunction of factual judgments and value judgments in HTA.
SUPPLEMENTARY MATERIAL
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462318000508.
Supplementary Search Strategies:
https://doi.org/10.1017/S0266462318000508
Supplementary Figure 1:
https://doi.org/10.1017/S0266462318000508
Supplementary Included Studies (n = 67):
https://doi.org/10.1017/S0266462318000508
Supplementary Excluded Studies with reasons (n = 65):
CONFLICTS OF INTEREST
The author have nothing to disclose.
