Recent economic developments at the global level have led to both an increased focus on evidence-based approaches for rational policy making in health care and exacerbated inequities across social determinants of health. While a given health policy, health program, or coverage decision may have no direct impact on inequities, it may also reduce or intensify them. Health technology assessment (HTA) was conceived as a tool to guide decision makers in efficiently allocating limited resources within the health system and enabling timely access to appropriate technologies based on scientific knowledge. It is one of the main evidence-based tools informing coverage decisions in many countries. Furthermore, it was endorsed by the World Health Organization (WHO) in May 2014 as a vital component of decision making for countries aiming to achieve universal coverage in an equitable way (1). It is, therefore, important to consider whether HTA practice is concerned with equity questions and how such concerns could be incorporated in the development of assessments in a realistic way.

The literature on equality and equity in health and health care is vast and multifaceted. In her defining work for WHO, Whitehead (Reference Whitehead2) states that the term inequity goes beyond measurable differences in health status to include a moral and ethical dimension; inequities are differences that are unnecessary and avoidable but are also considered unfair and unjust in the given context. Equity is “therefore concerned with creating equal opportunities for health and with bringing health differentials down to the lowest level possible” (Reference Whitehead2). The WHO's Commission on Social Determinants of Health (CSDH) further specified that systematic differences in health considered to be avoidable by reasonable action are unfair and thus inequitable (Reference Marmot, Friel, Bell, Houweling and Taylor3).

Evans and Brown (Reference Evans and Brown4) first proposed the operationalization of determinants across which such differences can exist with the PROGRESS acronym, later to be extended to PROGRESS-Plus (Place of residence, Race/ethnicity, Occupation, Gender, Religion, Education, Socioeconomic status, Social capital; “Plus” includes additional elements that influence health equity such as age, sexual orientation, disability and other vulnerabilities) (Reference Kavanagh, Oliver and Lorenc5). The “community effectiveness” of a given technology, namely its impact when applied in a “real world” setting compared with the experimental environment of a clinical trial, may vary for different population groups as it takes into account contextual factors (Reference Tugwell, de Savigny, Hawker and Robinson6). Thus, to assess the real world consequences of adopting a given health technology, contextual elements need to be considered and the corresponding evidence sources and types included in evaluations (Reference Tugwell, Bennett, Sackett and Haynes7–Reference Oxman, Lavis, Lewin and Fretheim10).

The HTA Glossary (HTAglossary.net) defines equity as “the fair allocation of resources or treatments among different individuals or groups, such that they each get what they are owed or what they are entitled to. (. . .) Vertical equity means that the people in the greatest need of services receive the most services, and horizontal equity means that people who have similar needs receive similar services.” In the context of HTA for resource allocation, there has been ample discussion on equity concerns at the decision point, both at the theoretical level, for example, on value judgments and the underlying theories of distributive justice, and from a methodological perspective. Williams and Cookson's (Reference Williams and Cookson11) estimate in 2006 was that the majority of HTA programs did not have a systematized approach toward balancing equity concerns with efficiency considerations when determining value. Culyer and Bombard (Reference Culyer and Bombard12) note that to date, equity discourse among HTA agencies takes the form of either a general awareness and recognition of relevant issues, such as the differentiation between horizontal and vertical equity, or more concrete individual approaches, such as the application of equity weights in the evaluation of benefits and costs.

When it comes to evidence assessment, best practice approaches have long recognized the importance of considering sociocultural elements along with clinical effectiveness, safety, and costs. However, these elements have not been sufficiently incorporated in practice, mainly due to restrictions in time and other resources (Reference Lehoux and Williams-Jones13). Several important initiatives emerged in recent years, going beyond the general consideration of sociocultural elements to explicitly address equity issues related to HTA. The WHO Collaborating Centre for Knowledge Translation and Health Technology Assessment in Health Equity put together the first “equity-oriented toolkit” for HTA (Reference Ueffing, Tugwell and Hatcher Roberts14), while Culyer and Bombard developed a framework on equity-related issues to be considered by decision makers across the spectrum of HTA report production (Reference Culyer and Bombard12). Furthermore, extensive methodological work on incorporating equity considerations exists for individual HTA domains, such as on systematic reviews of effectiveness (Reference Oxman, Lavis, Lewin and Fretheim10;Reference Tugwell, Petticrew and Kristjansson15;Reference Welch, Petticrew and Tugwell16) or economic evaluation (Reference McDaid, Sassi, Shemilt, Mugford, Vale, Marsh and Donaldson17–Reference Cookson, Drummond and Weatherly19).

Perhaps most notable in this respect is the work of the Campbell and Cochrane Equity Methods Group. The Group responded to the CSDH endorsement of systematic reviews as one of the tools that could be used to guide political action on equity (Reference Kelly, Morgan, Bonnefoy, Butt and Bergman20) by launching a comprehensive research effort to optimize existing methods. It evaluated the incorporation of equity concerns in existing reviews and found that their consideration was sporadic, variable, and inadequate (Reference Tugwell, Maxwell and Welch21;Reference Welch, Petticrew and Ueffing22). Subsequently, they produced concrete signposts for equity-sensitive reviews, including an equity extension for the PRISMA reporting tool (Reference Tugwell, Petticrew and Kristjansson15;Reference Welch, Petticrew and Tugwell16;Reference Welch, Petticrew and O’Neill23).

While equity is an important criterion from the decision-makers’ point of view (Reference Guindo, Wagner and Baltussen24), evidence used to inform policy does not always take relevant considerations into account (Reference Cookson, Drummond and Weatherly19;Reference Tugwell, Maxwell and Welch21;Reference Mathes, Jacobs, Morfeld and Pieper25;Reference Humphreys and Ogilvie26). Very little information exists on institutional practices toward equity-related issues in HTA report production (Reference Mathes, Jacobs, Morfeld and Pieper25;Reference Panteli, Zentner, Storz-Pfennig and Busse27;Reference Clifford28). Our research aimed to systematically explore whether and how HTA agencies include equity considerations in their assessments; furthermore, to map approaches, both methodological and process related, that could accommodate equity-sensitive reports even if they are not initially adopted for that purpose; lastly, to illustrate exemplary practices in combination with methodological literature that could aid those interested in including equity elements in their work. We were not concerned with the equity discussion on theories of distributive justice adopted by individual agencies, as reflected, for example, in the setting of cost-effectiveness thresholds or the rationale behind equity weights, as these are bound to be both institution- and context specific (Reference Culyer and Bombard12).

RESULTS

Included Agencies

Of 121 agencies identified, both methodological papers and HTA reports in the working languages were available for nineteen (Supplementary Figure 1). The nineteen agencies included in the analysis were diverse in terms of remit (some only evaluated pharmaceuticals or procedures while others covered a broad range of technologies) and role in their health system (Table 1). Due to the study's technical limitations in terms of access to information and language, some countries were not represented in the sample and others were present more than once.

Table 1. Agencies Included in the Analysis, Type of Methodological Guide and Number of Included Reports

Note. 1, general methods; 2, agency methods for assessment only; 3, agency methods for assessment and appraisal; 4, guide for submission by the industry.

^a Document has legal function.

^b Appraisal process is described only for the “interim funding” approach, which falls under the “coverage with evidence development” scope.

^c One report addressed both a drug and a non-drug intervention and is counted twice in the subtotals.

Equity Considerations in Methodological Guides

As a natural consequence of the variability of agencies included in the sample, methodological guides included general educational handbooks (e.g., DACEHTA), agency methods for assessment only (e.g., AHRQ, DIMDI), agency methods for assessment and appraisal (e.g., G-BA, TLV) and specific templates outlining requirements for submissions by the industry (e.g., DERP) (Table 1). Furthermore, documents varied in terms of length and age at the time of analysis; while some agencies indicated having newer, internal manuals available at the time in their responses, our research only included the latest publicly available documents in the analysis. A full list of included methodological guides can be found in Supplementary Table 2.

Approximately half of the methodological documents included some interpretation of equity among the goals of the respective health technology assessment program or the document itself. In this context, this took the form of “fairness” (AHRQ), “social justice” (KCE), the “appropriate consideration of the interests of those afflicted, taking into account age, biological and social gender and particularities related to life circumstances” (G-BA) or “equity” verbatim (CRD, MSAC, NICE). These formulations were related to the agency's remit and the type of document examined. For example, the G-BA is responsible for decision making on health technologies in the German statutory health insurance system and its methods are described in its code of procedure (“Verfahrensordnung”). It was, therefore, not surprising that the latter included a consideration of different population characteristics to be taken into account. On the other hand, methodological guides purely focusing on assessment elements (e.g., HVB, DIMDI) did not necessarily require the description of system-level goals.

Methodological Elements

As a general approach potentially related to equity concerns, the consideration of (psycho)social issues related to the adoption of the technology in question was mentioned by several agencies (CADTH, DIMDI, GÖG, KCE, NICE). CADTH mentioned such elements under the concept of “intangible” factors that need to be accounted for and can include “patient satisfaction, individual acceptance of screening/treatment, stigmatization/anxiety and gender/family concerns” and distinguished them from issues that need to be considered under “equity-efficiency trade-offs,” which were mentioned in a separate subsection. According to the KCE, patient preferences were to be examined in the societal context, especially in case of social justice concerns.

Determinants in the PROGRESS-Plus acronym were explicitly mentioned in three ways across the included methodological guides: (a) to consider in terms of relevance of different subgroups when formulating the research questions (AHRQ, DERP, G-BA), (b) in terms of looking for or conducting subgroup analyses in included studies on effectiveness (AHRQ, CADTH, DERP), and (c) recognizing groups to examine more closely in the context of inequalities (CADTH, CRD, GÖG, IQWiG, NICE).

Table 2 shows specific approaches adopted by individual agencies that were related either to general equity considerations or the examination of effects on population subgroups. These range from the appropriate sources for evidence on psychosocial implications (DIMDI) to individual tools such as health services impact (CADTH) and the NICE Equality Scheme.

Table 2. Specific Methods Related to Health Equity

Process-Related Elements

Information on practices related to procedural equity, such as the participation of stakeholders, was extracted where available. However, the depiction of such issues was highly dependent on the nature of the methodological guide under consideration. Thus, the compiled overview of related processes is potentially incomplete both for individual agencies and the overall sample.

Examples of approaches described in the documents included the involvement of stakeholders in topic selection (AHRQ); opening the assessment protocol to public comment before embarking on the actual assessment process (IQWiG); public consultation processes on draft assessment reports (AHRQ, DERP, IQWiG); open review rounds upon publication of the report (KCE); and interaction with a body acting as a permanent advisor throughout the appraisal process (NICE Citizens Council).

Equity was mentioned as a contributing factor for topic selection in the documents of five institutions (AHRQ, DERP, G-BA, MSAC, PHARMAC). While in most cases this was kept general to include vulnerable groups and patient subgroups, PHARMAC explicitly mentioned advancing the health of the Maori/Pacific population in New Zealand as a prioritization criterion for assessments.

Evidence from HTA Reports

We identified ninety-eight reports from agencies included in the analysis, spanning drug, nondrug and population-based interventions (Table 1) and published between 2006 and 2011. Information on the prespecified domains was extracted and transferred to extraction tables (one report which addressed both drug and nondrug interventions was extracted twice).

As per the inclusion criteria, all reports included an assessment of effectiveness, while fifty-one reports (52 percent) also considered economic elements. Of those, twenty-six (51 percent) reported original economic evaluations based on a variety of methods, sixteen (31 percent) presented and discussed the results of existing economic studies, while a further nine (18 percent) provided information on direct costs with varying sources and level of detail.

Some agencies (DERP, TLV, G-BA, IQWiG for screening interventions) mention in their reports a general commitment to producing assessments that consider equity or fairness. Only one report had a term (“social inequalities”) related to equity in the topic. This report adopted a clear equity focus and “applied an ‘equity lens’ to tobacco control policies, re-examining the available evidence about the impact of policy measures and other population-level interventions to assess their role in tackling health inequalities (. . .)” in a systematic way (Reference Fayter, Main and Misso33). Six reports included some element of equity in their objectives, three of which were concerned with addressing the issue of unequal access (due to regional or socioeconomic factors) and a further two with reducing inequalities in health (see Supplementary Table 3).

Methodological Elements

Nineteen reports (19 percent) included the consideration of population subgroups among their research questions. However, only five of those did so with an explicit link to contextual or socioeconomic elements or access to care while the rest were interested in the applicability of clinical effectiveness findings across population groups. Similarly, subgroup analyses were carried out or reported mainly to test the applicability of clinical findings for different groups of patients and not in relation to community effectiveness. Only seven reports included their own subgroup analyses, while thirteen (13 percent) pointed out that subgroup analyses were not possible based on the available primary research and thirteen (13 percent) reported subgroup analyses from individual primary studies. The determinants most frequently considered were age, gender, comorbidities and disease severity (in descending order). The dearth of evidence on specific populations (such as breastfeeding mothers of low socioeconomic status, migrant populations, etc.) was particularly highlighted by three reports with increased relevance for vulnerable groups, who are underrepresented among study participants.

Ten reports (10 percent) considered ethical and sociocultural elements separately as one of their assessment domains; of those, half did not succeed in finding any primary evidence to match the corresponding research questions. Three more reports mentioned that while such issues are important, they were beyond the scope of the presented document. Only one report involved a participatory approach with patients and their families to obtain relevant information to contextualize findings from published research.

Interpretation of Results and Recommendations in Included Reports

Eighteen reports (18 percent) included equity considerations in their interpretation of results (see Supplementary Table 3). These considerations encompassed elements of coverage (e.g., technology not covered or only covered by some regional plans in a decentralized context or high copayments putting low-income groups at disadvantage), increased prevalence among vulnerable groups that would merit targeted interventions, variable uptake due to psychological elements connected to specific determinants (e.g., the higher uptake of bariatric surgery among women, reluctance to participate in cystic fibrosis screening for some ethnic groups and low awareness of sudden infant death syndrome in families with low socioeconomic status due to a different perception of danger) and organizational aspects that influence access to care.

Six reports on population-based interventions recommended the introduction or intensification of targeted interventions for vulnerable groups in their conclusions. A further six reports called for measures to ensure access, which took the form of revision of coverage plans, triage strategies for the equitable allocation of technologies, development of existing organizational structures to enable delivery and subsidies for vulnerable groups. Four reports included general statements endorsing a future societal perspective or universal access in their recommendations (see Supplementary Table 3).

Process-Related Elements

As was the case with the analysis of methodological documents, the availability of information on stakeholder involvement in included reports was variable; findings can, therefore, only be interpreted as indicative and not as a complete overview of approaches. Forty-three reports (44 percent) mentioned stakeholder involvement at some stage of the report production process. The most common form of involvement described was external review of the draft report, followed by participation in the determination of the scope and/or research questions before report production began. Input methods varied, but the online publication of documents (protocols and/or draft reports) was a common characteristic. Some institutions, like IQWiG, followed up on written public comment to the draft report with structured meetings when elements remained unclear. Eight reports mentioned that the topic for assessment originated in a nomination and/or prioritization process that incorporated public involvement.

DISCUSSION

Our findings indicate that equity was not a standard consideration in HTA reports. The nature of specific approaches, as well as the amount of resources invested into including an equity perspective, varied considerably. Where such approaches were used, they seemed to depend on the role and remit of the institution, its jurisdiction level, the legal context within the health system and finally the type of technology under consideration. For example, NICE, an institution that has long been a proponent of addressing equality issues in the evaluation of health technologies, launched a separate equality scheme in 2007. The scheme aimed to put the Institute's commitment to fighting discrimination and fostering equal opportunities in practice, both by ensuring fair and appropriate representation in advisory bodies and applying a set of equity-related questions throughout the guidance production process (34). It is unlikely that HTA agencies with fewer resources or a narrower scope would be able or required to establish similar approaches. Still, an equity-sensitive approach introduced early in the process of report production could increase the likelihood that potential issues are not overlooked (Reference Culyer and Bombard12) and that the appropriate questions are asked and methods of analysis used. It would, therefore, be preferable in comparison to an additional consideration of potential equity concerns after effectiveness or cost-related questions have been answered.

While the consideration of equity issues in the analyzed reports was sporadic, it encompassed both concerns of vertical and horizontal equity: targeted interventions were endorsed by some reports and differential outcomes or consequences for universal applications of technologies were raised in others. While prioritizing the evaluation (and funding) of targeted interventions for specific population groups to reduce health inequities is vital, it also depends on individual agency or policy agendas. It is important to recognize that health technologies not aimed at bridging the equity gap still need to be addressed in an equity-sensitive way as they may introduce, perpetuate or exacerbate inequities that are not apparent prima vista.

An equity-sensitive approach adopted in the early stages of assessment would mean that specific population groups that merit attention are taken into account while formulating the research questions, exploring the context of technology implementation, or choosing appropriate evidence types and study designs for the assessment (Reference Oxman, Schunemann and Fretheim9;Reference Oxman, Lavis, Lewin and Fretheim10;Reference Tugwell, Petticrew and Kristjansson15;Reference Welch, Petticrew and Tugwell16;Reference Welch, Petticrew and O’Neill23). However, methods used in the analysis also need to reflect these steps. Our findings show that subgroup analyses were rarely conducted by the institutions themselves. When they were discussed, it was usually in regard to the applicability of overall findings for population groups with different biological or clinical characteristics and not in connection to contextual or other equity-related factors. Analytical tools such as subgroup analyses need to be appropriately applied to avoid misinterpretation (Reference Oxman, Lavis, Lewin and Fretheim10;Reference Welch, Petticrew and Ueffing22;Reference Petticrew, Tugwell and Kristjansson35) and require sufficient and adequate information from primary studies. In fact, some included reports mentioned that subgroup analyses were not possible due to insufficient data.

Other reports in our sample concluded that more primary research tailored to enable insights into inequities was required in general. The issue of adequate primary studies as a prerequisite for equity considerations had already been raised by Whitehead (Reference Whitehead2) and the particular necessity of good data for equity-sensitive evaluations was further emphasized by Tugwell and colleagues (Reference Tugwell, Petticrew and Kristjansson15). While it is hoped that the promotion and usage of equity-oriented tools developed for evidence syntheses will lead to corresponding initiatives in primary research in the long-run (Reference Welch, Petticrew and Tugwell16), a more active endorsement would definitely be beneficial.

Procedural equity is not only important from the viewpoint of fair representation. Stakeholder involvement, when structured accordingly, can also contribute to pinpointing the questions to ask and what is important. This is particularly the case when it includes those most directly affected by a technology's implementation, namely patients and their caregivers. The type and entry point of involvement is expected to vary based on agency tradition and system context (Reference Kreis, Puhan, Schunemann and Dickersin32;Reference Cavazza and Jommi36). This was mirrored by findings from included reports, which mentioned consultation and/or participation approaches. Participation in the early stages of report production, for instance at the protocol development stage, can be particularly useful in revealing equity issues that may need to be considered throughout the assessment. Furthermore, an inclusion of stakeholder groups and especially patients in the agenda setting for assessments (Reference Nasser, Ueffing, Welch and Tugwell37) can foster the goal of evaluating technologies to address inequalities. In both respects, existing agency approaches can be expanded or modified to enable a clearer, more direct link between stakeholders and an equity-sensitive approach. At this point, it is important to note that if such approaches have an additional goal of facilitating equity-sensitive evaluations, those included need to be carefully selected to ensure that important perspectives are not missed; which parties to involve has already been found to be one of the main challenges for agencies using related processes (Reference Gagnon, Desmartis and Lepage-Savary31;Reference Kreis, Puhan, Schunemann and Dickersin32).

Ideally, a fully realized equity perspective in the assessment of health technologies would entail a multifaceted approach spanning protocol development to dissemination and using both equity-aware staff and inclusive stakeholder involvement. However, the need for swift assessment results along with resource considerations can hamper both the implementation of equity-sensitive tools that are detailed and comprehensive (Reference Burford, Welch and Waters38) and the elaboration of participatory mechanisms (Reference Gagnon, Desmartis and Lepage-Savary31). According to their task, process, and resources, HTA agencies interested in adopting a more equity-sensitive approach may need to choose their areas of focus. Figure 1 combines examples of approaches identified in our analysis (institutional names) and insight from methodological literature (references in square brackets in Table 3) to provide an overview of potential points for consideration.

Figure 1. Potential equity considerations along the evaluation process.

Table 3. Examples of Equity Considerations Depicted in Figure 1