Introduction
The Spanish Network for Health Technology Assessment of the National Health System (RedETS) consists of a secretary from the Spanish Ministry of Health and eight HTA agencies and units (Reference Serrano-Aguilar, Asua-Batarrita, Molina-López, Espallargues, Pons-Rafols and García-Armesto1): the Agency for Health Quality and Assessment of Catalonia (AQuAS), the Basque Office for Health Technology Assessment (OSTEBA), Institute of Health Carlos III (ISCIII), Andalusian Health Technology Assessment Area (AETSA), the Evaluation Unit of the Canary Islands Health Service (SESCS), the Scientific-Technical Advice Unit from the Galician Agency for Health Knowledge Management (avalia-t), the Aragon Institute of Health Sciences (IACS), and the Healthcare Technologies Evaluation Unit of Madrid (UETS). According to the Spanish legislation, one of the main tasks of the RedETS is the production of HTA reports. Candidate technologies for assessment are proposed and prioritized. An annual working plan that compiles the HTA reports that will be developed every year is made public (Reference Serrano-Aguilar, Asua-Batarrita, Molina-López, Espallargues, Pons-Rafols and García-Armesto1). RedETS agencies and units work independently in the specific reports included in the networks’ annual working plan, but they coordinate to apply a common methodological framework to ensure quality in the assessment of technologies and the elaboration of HTA products (Reference Puñal-Riobóo, Baños Álvarez, Varela Lema, Castillo Muñoz, Atienza Merino and Ubago Pérez2). As part of this common guidance, the RedETS published a methodological framework for patient involvement (PI) and a strategy for its implementation.
The RedETS' PI strategy was built by integrating results from a literature review, semi-structured interviews to HTA managers and researchers with PI experience, a consultation to patient organizations, and a consensus process among the members of the RedETS Governing Council. The strategy used involvement as an inclusive concept of three levels of participation: communication, consultation, and engagement (Reference Facey, Plough Hansen and Single3).
A three-phase strategy for PI implementation was developed, namely (i) pilot experiences in PI across RedETS agencies and units in the short term (2017), (ii) internal and external capacity building in PI for HTA researchers and patients in the medium term, and (iii) on a long-term basis, PI mainstreaming (Reference Toledo-Chávarri, Perestelo-Pérez, Álvarez-Pérez, Abt-Sacks, Santoro Domingo and Villalón4;Reference Toledo-Chávarri, Alvarez-Perez, Triñanes, Perestelo-Pérez, Espallargues and Palma5). Current challenges in the field of PI in HTA include the need to reflect on how best to combine research methodologies to increase impact and understand the appropriate role of patient-based evidence (Reference Facey, Plough Hansen and Single3).
Evaluations of PI initiatives in HTA are increasing and could promote best practices to further improve the participation processes and their impact (Reference Gagnon, Dipankui, DeJean, Facey, Ploug Hansen and Single6). Evaluations show that these methodological challenges impact on PI implementation procedures. A survey by Weeks et al. to HTA international organizations (n = 15) showed that almost half of them (7/15) evaluated their PI activities (Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey7). In the RedETS, an internal evaluation of the 2017 pilots on PI was conducted (Reference Toledo-Chávarri, Triñanes Pego, Reviriego Rodrigo, Ibargoyen Roteta, Novella-Arribas and Vicente Edo8). Both studies pointed out challenges in the implementation of PI in HTA, including the variability of HTA objectives (e.g., medical devices, medical procedures, rapid HTA, and health economics) (Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey7) and the lack of guidance to tailor PI methodology to each assessment (Reference Toledo-Chávarri, Triñanes Pego, Reviriego Rodrigo, Ibargoyen Roteta, Novella-Arribas and Vicente Edo8). The RedETS evaluation showed that agencies and units have used different methodologies for PI including direct input, primary studies, and literature reviews. The criteria for these methodological decisions were not homogenic. One of the main conclusions of the evaluation of the pilot processes was the need for a common tool that could help enhance and standardize PI methods and procedures in RedETS HTAs.
Nevertheless, there is a lack of literature on the actions that followed those evaluation processes and what answers are HTA agencies providing to the mentioned methodological challenges. The aim of this paper is to describe the development of a decisional flowchart to guide RedETS researchers’ decisions regarding PI procedures and methods in individual HTA processes.
Methods
In December 2017, during the RedETS Annual Conference, a PI Interest Group was created to support and facilitate PI in HTA within our network. With the support of the Spanish Ministry of Health and the RedETS Council, the PI Interest Group incorporated at least one member from each of the eight agencies and units. During the development of the flowchart, the PI Interest Group had twenty-two HTA researches as members. The PI Interest Group held bi-monthly teleconference meetings (phone or online) and an annual face-to-face encounter to share experiences, seek peer support to solve methodological issues, and work on advancing PI mainstreaming. The decisional flowchart for PI was developed between March and April 2019 following an iterative process, reviewed by the members of the PI Interest Group, as well as by other members of the HTA agencies/units, and discussed and validated during an online deliberative meeting.
The first version of the flowchart was based on the RedETS PI methodological framework (Reference Toledo-Chávarri, Perestelo-Pérez, Álvarez-Pérez, Abt-Sacks, Santoro Domingo and Villalón4;Reference Toledo-Chávarri, Alvarez-Perez, Triñanes, Perestelo-Pérez, Espallargues and Palma5) and the results of the evaluation of the pilot study (Reference Toledo-Chávarri, Triñanes Pego, Reviriego Rodrigo, Ibargoyen Roteta, Novella-Arribas and Vicente Edo8). The methodological framework was based on a literature review, a qualitative study, a Delphi consultation, and a consensus process with the members of the RedETS Governing Council to define the final strategy (Reference Toledo-Chávarri, Perestelo-Pérez, Álvarez-Pérez, Abt-Sacks, Santoro Domingo and Villalón4;Reference Toledo-Chávarri, Alvarez-Perez, Triñanes, Perestelo-Pérez, Espallargues and Palma5).The flowchart included the six phases of HTA identified in the methodological framework: (i) Identification and prioritization of technologies for assessment; (ii) Setting objectives and scope of the assessment and problem definition; (iii) Evidence gathering and review; (iv) Elaboration of recommendations; (v) Review and presentation of the allegations; (vi) Dissemination of HTA results. This first draft was reviewed by the director and five senior researchers of SESCS. A second version of the flowchart was produced and reviewed by two researchers from two other HTA agencies (avalia-t and OSTEBA) who had experience in PI in HTA. The third version of the flowchart was sent to the PI Interest Group members who shared it and discussed it with other HTA researchers from their agencies and units. Modification suggestions were discussed during an additional teleconference meeting in March 2019. The revised version of the flowchart was formatted with an online design tool according to the consensus reached during the meeting, and it was sent to the PI Interest Group for a last review. After minor changes, a final version of the flowchart was obtained and reviewed at an ordinary meeting of the PI Interest Group held in April 2019. Some aspects related to the use of the flowchart were discussed, but its content was not modified. The flowchart was then presented to the RedETS Council. The development of the flowchart described below was developed through a review of its different versions, as well as the e-mail communications between the members of the PI Interest Group and the notes taken during the meetings.
Results
The decision-making process on how to involve patients in RedETS HTAs was graphically represented. The final version of the flowchart is presented in Figure 1. In the flowchart, the term “patient” was defined as people living with a condition, representative of patient groups and organizations, health system users, family members, or informal caregivers. The flowchart also refers to specific examples of these different actors that can be found in the overall RedETS methodological framework for PI (Reference Toledo-Chávarri, Perestelo-Pérez, Álvarez-Pérez, Abt-Sacks, Santoro Domingo and Villalón4).
Figure 1. Flowchart to inform patient involvement in HTA reports of the RedETS.
The RedETS Annual Working Plan defined every year in January, detailing all commissioned HTA reports, should be disseminated to umbrella patient organizations with a general invitation to participate.
The flowchart indicates that patients should be involved in all HTAs, except those that assess technologies with no direct implication for patient experiences, values, and preferences. These exceptions were illustrated during the meeting with examples of previous assessments in RedETS, such as a hospital pharmacy dispensing robot (Reference Giménez, Reynolds and Espallargues9), or the one included in a footnote of the flowchart, a new clinical information system for image file management and storage (Reference Herrera-Ramos, Bermúdez-Pérez, Ojeda Cruz, Perdomo-Cabrera, González-Rodríguez and García-Pérez10). The EUnetHTA Core Model 3 (11) questions for ethical, patient, and social domains have been adapted to a checklist for decision making regarding direct PI on an HTA (Figure 2). Only when none of these questions is valued relevant in relation to the technology being assessed, patients may not be included. Patients should be involved otherwise. The plan for PI should be stated in a specific section of each HTA protocol. The results of the participation process should be reflected in the final report.
Figure 2. Checklist for decision making regarding patient direct involvement in Health Technology Assessments.
The first step to PI in specific assessments proposed in the flowchart was to identify and invite patient organizations and/or expert patients that were related to the topic of the HTA. Several recruitment sources were identified, including specific patient organizations, advisory councils, patient parliaments, and clinicians. A formal invitation should be sent to the targeted associations or individuals. A minimum of two to three patient representatives involved in each HTA report was recommended.
Second, the flowchart indicates at which stages of the HTA process these patients should be involved. Indeed, patient representatives or expert patients could participate directly in the decisions of each HTA as a part of an expert panel during the protocol development, the assessment process, and the report review. For instance, patients can suggest outcome measures that are meaningful to them and define their relative importance to decision making, and identify the populations and subgroups affected by the technology among other contributions. If patient-based evidence is not included or available, they may share their experiences, values, and preferences with respect to the health condition and the technology of interest and evaluate the impact that the technology may have in a real-life context and its acceptability, to inform the framing of the recommendations. If patient-based evidence is available, patients could comment on the contextual relevance of those findings.
Third, when the technology assessed affects, in a relevant manner, patient experiences, values, or preferences, additional information should be included. Patients must be involved in the decision-making process on the pertinence of making further efforts for the inclusion of patient perspectives by collecting or producing patient-based evidence. Specific objective(s) for these further efforts must be defined. The first option would be to conduct a systematic review of the literature if available studies provide answers to the specific research questions of the HTA. In general, qualitative or mixed-method systematic reviews could be of value. The GRADE-CERQual tool and the methodological guidance from the Cochrane Qualitative & Implementation Methods Group can support conducting such syntheses (12;13). When no relevant literature is found, a primary study should be conducted to address those specific objectives. Primary studies that aim to gather patient experiences, values, and preferences regarding their health condition and the proposed technologies could employ methods such as focus group, interviews, and surveys. When a primary study with patients is deemed necessary in the context of an HTA, ethical approval should be sought as in other clinical research studies by sending the study protocol to the Ethics Committee of reference of each agency/unit.
The content and the format in the flowchart were simplified in the successive versions to make it easier to understand. The flowchart focuses exclusively on the development of HTA reports. Therefore, the PI Interest Group excluded methods and procedures in the identification and prioritization of technologies to be evaluated. Dissemination of HTA results was also excluded as the RedETS has previously developed specific guidelines (Reference Hermosilla-Gago14). The group also discussed whether citizen participation should be mentioned. Finally, a footnote was added specifying that the term “patient” is used generically in the flowchart including a diversity of types of participants (see footnote 1). The PI Interest Group shared and discussed some considerations that ultimately did not change the flowchart. A researcher proposed including a reference to a set of tools to facilitate PI, among others, the Patient Group Submission Template for Non-medicine's HTA from the Health Technology Assessment international (HTAi) Interest Group for Patient and Citizen Involvement in HTA that was being translated to Spanish by a team from the AQuAS at the time (15). PI facilitation tools were already available in the repository of resources of the PI Interest Group, so the proposal for their inclusion in the flowchart was dismissed. Exclusion of PI in the assessment of emerging technologies was also suggested, but the PI Interest Group agreed that new technologies with little evidence may be those in which patients can specially contribute with their experiences, values, and preferences.
Discussion and Conclusions
This article shares the development process of a flowchart to guide the decision of RedETS researchers regarding PI in HTA. The flowchart may help other HTA researchers or organizations to elicit their options in PI planning or to develop their own decision-making tools.
Evaluations have been used to ameliorate PI processes in their respective organizations (Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey7). Our results add to previous contributions that have created flowcharts in order to facilitate PI in HTA and in other research studies such as clinical trials (Reference Hunter, Facey, Thomas, Haerry, Warner and Klingmann16;Reference Bagley, Short, Harman, Hickey, Gamble and Woolfall17). The European Patients’ Academy on Therapeutic Innovation (EUPATI) designed a flowchart describing the possible contributions of patients throughout the drug research and development process including HTA, but it was not aimed for planning or decision-making purposes (Reference Hunter, Facey, Thomas, Haerry, Warner and Klingmann16).
Several elements suggest that the RedETS flowchart could facilitate meaningful patient participation along HTAs. The flowchart proposes early involvement of patients in such a way that they have the capacity to influence the HTA process from its first steps (Reference Wortley, Tong and Howard18;Reference Hämeen-Anttila, Komulainen, Enlund, Mäkelä, Mäkinen and Rannanheimo19). A systematic review showed that the most relevant contributions from the public could be made during the definition of the problem and the framing of the objectives and scope (20). In our proposal, patients can contribute to the definition of the HTA protocol and therefore to its objective, population, and relevant outcome measures. In addition, the flowchart also promotes their involvement in decision making about the scope of PI in each assessment, including the call to collect or produce patient-based evidence. The RedETS flowchart presents a process to foster PI in HTA that combines direct input and patient-based evidence. This combination has been proposed previously by other authors and HTA organizations (Reference Facey, Plough Hansen and Single3;11;Reference Hämeen-Anttila, Komulainen, Enlund, Mäkelä, Mäkinen and Rannanheimo19;Reference Kristensen and Sigmund21). If no relevant patient perspective literature is available to meet the objectives pursued, primary studies are needed. These primary studies could provide information “unavailable from other sources,” which a previous evaluation of PI in HTA identified as one of the most relevant aspects of meaningful participation (Reference Rozmovits, Mai, Chambers and Chan22). Furthermore, these primary studies can collect specific and contextual contributions from broader groups, including populations that are not usually involved in advisory panels, such as people at risk of exclusion (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis23). The combination of different methods allows a triangulation of various results that can be contrasted through the use of different procedures (20;Reference Gauvin, Abelson, Giacomini, Eyles and Lavis23). Therefore, our flowchart aligns with the proposal of promoting the value of patient-based evidence along with clinical and economic inputs, producing evidence that is more robust and that can be critically assessed (Reference Staniszewska, Werkö, Facey, Plough Hansen and Single24).
The most discussed element during the elaboration of the flowchart within the PI Interest Group was the criterion that patients may not be involved when assessing technologies that do not affect their experiences, values, or preferences. The PI Interest Group finally decided that this criterion may serve as a guide for decision making as it could facilitate good practices by limiting the cases in which PI was excluded. To facilitate decision making and avoid arbitrary decisions, a checklist of questions to help determine when a technology has no implications for patient experiences, values, and preferences was included. The decision made should be justified and included in the protocol and final report in order to make it transparent.
Similarly, the Public Engagement Subcommittee from the Ontario Health Technology Advisory Committee recommended that PI be included in all assessments with two exceptions that "the topic under review concerns technologies with no direct patient interface” and/or "the focus of the review is exclusively on the technical aspects of the technology” (20). Moreover, some HTA agencies admit to avoiding PI in the preparation of brief reports of an urgent nature or when it comes to evaluating the validity and precision of diagnostic tests (Reference Gauvin, Abelson, Giacomini, Eyles and Lavis23).
This paper has many limitations. Patients were not involved in the conceptualization and design of the flowchart, and they may have proposed different solutions. Another limitation of the flowchart was the lack of feedback from PI naive users. The complementary footnote comments are written based on experience and serve as a basis to minimize this limitation. Nevertheless, the flowchart is not intended to be the sole PI guide in the RedETS, as it is a part of the actions contemplated in the networks’ long-term strategy (Reference Toledo-Chávarri, Perestelo-Pérez, Álvarez-Pérez, Abt-Sacks, Santoro Domingo and Villalón4;Reference Toledo-Chávarri, Alvarez-Perez, Triñanes, Perestelo-Pérez, Espallargues and Palma5). The use of the flowchart will be accompanied by methodological support from the PI Interest Group, which is also preparing training strategies for researchers and patients. In addition, there will be a periodic evaluation of the PI process in which the use of the flowchart will be reviewed.
The flowchart is being tested in the 2020 Annual Working Plan. Research is ongoing on the use of the flowchart in HTA activities within the RedETS and more data will be available to assess its usefulness and its impact.
The development of a flowchart for meaningful PI in HTA within the RedETS is expected to guide decision making on when and with which methods to involve patients in specific HTAs. Our flowchart contributes to an answer to the current methodological challenges in the field of PI in HTA by proposing a combination of methods that combine direct involvement with patient-based evidence. It also aims to increase transparency regarding PI and the HTA, because the resulting process and contributions will be included in the HTA protocol and in the final report.
Acknowledgments
The authors of this paper are grateful to the RedETS researchers that have participated in the review of any of the versions of the flowchart presented here.
Funding
This work was performed in the framework of the Spanish Network for Health Technology Assessment of the National Health System (RedETS) and under its funding.
Conflict of Interest
The authors have no conflict of interests to declare in relation to this work.
