During the last decade, patient involvement in health technology assessment (HTA) has benefited from attention by decision-makers, HTA professionals, patients, and academic researchers (Reference Gagnon, Desmartis, Gagnon, St-Pierre, Rhainds and Coulombe1–Reference Single, Facey, Livingstone and Silva4). Patient involvement is defined as patient participation in the HTA process and research into patient aspects (Reference Facey, Ploug Hansen and Single5). While the focus has been largely put on national HTA agencies, little attention has been paid to patient involvement in local HTA units (Reference Single, Facey, Livingstone and Silva4–Reference Nabarette7). Because these local HTA units, sometimes called hospital-based HTA units, are designed to meet the local requirements for meso- and micro-decisions in hospitals or local health care and social services facilities (Reference Sampietro-Colom and Martin8–Reference Sampietro-Colom, Lach, Pasternack, Wasserfallen, Cicchetti and Marchetti11), they are ideally positioned to involve patients in their processes and to benefit from their experiences (Reference Roy, Ganache and Dagenais12–Reference Gagnon, Candas, Desmartis, Rhainds, Coulombe and Gagnon13). However, little is known about how patient involvement spreads into these local HTA units and even less about the needs of the units to encourage it (Reference Tantchou, Gagnon and Poder14).
In this study, we describe the results of a survey conducted in local HTA units in Quebec, Canada, to provide an overview of their practices to involve patients in their process, their interest in developing this involvement, and their information needs to allow such a development. Considering that the province of Quebec has always been at the forefront of local HTA development (Reference Sampietro-Colom and Martin8), this study will provide insight into potential challenges that other more hospital or service-based HTA units may face.
Methods
A meeting about public and patient involvement in HTA activities was held on 16 May 2017 on behalf of the Community of Practice (CoP) in HTA in Quebec. This CoP has the mandate to develop HTA activities in Quebec healthcare institutions, to share knowledge about methods and experiences and to initiate collaboration between HTA units. The CoP is coordinated by a permanent representative of the Institut national d'excellence en santé et en services sociaux (INESSS—National institute for excellence in health and social services) and a representative of a Quebec local HTA unit (i.e. rotating representation between local HTA units).
The main goal of the meeting was to present the results of an updated systematic review about public and patient involvement in HTA activities (Reference Tantchou, Gagnon and Poder14), to share knowledge and experiences of Quebec local HTA units on this topic, to identify the needs of these HTA units in terms of knowledge, and their preferences for dissemination (e.g. communication channels, supports, materials) to guide subsequent activities and mobilize the CoP.
To prepare for this meeting, an online survey was sent for completion to all local HTA units and INESSS. Since INESSS is highly involved in the CoP and collaborates with many local HTA units, it was deemed important to include it in the survey. Each unit had to designate a representative to complete the survey. The survey was sent 3 weeks before the meeting and needed to be completed before the meeting could be conducted. As stated above, the aim of the survey was to build an inventory of the practices of public and patient involvement in local HTA activities in Quebec, to assess the interest of these HTA units in developing this involvement, and to identify the information needs to allow this development (i.e. what kind of information they need to support them in implementing patient and public involvement and to encourage patient and public involvement).
As in Quebec, the Canadian French terms “Patients, usagers, proches-aidants et citoyens” (PUPAC) are used to identify all potential partners, the survey was conducted using these terms for “Patients, users, caregivers, and citizens.” This acronym is used by the provincial HTA agency (https://www.inesss.qc.ca/naviguer-selon-le-profil/patients.html) and by many local HTA units. The term “user” is used by social services institutions while hospitals used the term “patient.” In both cases, it refers to the person they care. For citizen, it refers to members of the public, that is to say, anyone who is not a patient, user, or caregiver.
The survey was conceived by two of the authors, one with an extensive experience in public and patient involvement in HTA activities from an academic point of view, and the other with empirical experience as a member of a local HTA unit. Most items in the survey were developed based on the updated systematic review (Reference Tantchou, Gagnon and Poder14) and their knowledge of the local HTA units process. The survey was then revised and improved by the other authors in two rounds. In each round, the importance of each item and its univocality was considered.
After the meeting, a second survey was sent. The aim of this second survey was to collect commentaries about the meeting, to prioritize the information needs from those identified at the meeting, and to plan the next steps for the dissemination of knowledge to local HTA units. This second survey was conceived and developed in the same way as the first stated above. It was sent 6 weeks after the meeting and data were collected up to 5 months after the meeting. In order to achieve the highest response rate, two reminders were sent. The surveys were anonymous and we did not know who responded in each HTA unit. Both surveys were written in Canadian French and are available upon request to the corresponding author.
To avoid any over-representation of local HTA units, in case we received more than one response per unit, we decided to keep only the responses from one respondent per unit, completing the information with the responses from the other respondent(s) of the same unit if necessary. For questions where respondents were asked to prioritize items, we calculated the mean response (i.e. on a Likert scale from 0 to 4, we took the mean score or the upper rounding if the mean score was not an integer).
Results
At the time of the meeting, twelve local HTA units (including INESSS) were active in Quebec. All these units participated in the meeting, and all but one completed the pre- and postmeeting surveys (n = 11). The one that did not participate in the survey was a unit with less than 1 year of experience in HTA activities.
For the postmeeting survey, eight respondents out of eleven responded in 1 month or less, and reminders were sent to the three local HTA units that did not initially respond.
The participating HTA units were: seven (63.6 percent) local HTA units from an integrated university center of health and social services, three (27.3 percent) local HTA units from a university hospital, and one (9.1 percent) HTA provincial agency (INESSS). In the premeeting survey, 63.6 percent indicated involving only patients in HTA activities, 9.1 percent involving both public and patients, and 27.3 percent not involving public or patients. In the postmeeting survey, the rates were 54.5, 18.2 and 27.3 percent, respectively. This finding indicates that, during the lapse of time between surveys, one HTA unit moved from involving only patients to involving public and patients in HTA activities.
Among the eight HTA units involving public or patients in HTA activities, 75 percent indicated that public or patients were involved in committees or working groups, 62.5 percent in knowledge transfer activities, while 50 percent were involved in data collection (i.e. mostly experiential knowledge) and 50 percent in the formulation of recommendations. Other indications of involvement are provided in Table 1.
Table 1. Type of Involvement in Local HTA Activities
Note: Responses are for HTA units indicating to involve public or patients in HTA activities (n = 8).
Moreover, in an open-ended question, we explored how local HTA units determined whether patients or members of the public should be involved in HTA activities. Responses from six HTA units out of eight indicated that it depended on the issues involved and the technology evaluation needs. However, one of these six units stated that they were consistent for all their HTA work.
Respondents were also asked to indicate if patient and public involvement activities have been evaluated in their unit. None of the eleven HTA units indicated they evaluated these activities. Those who provided comments in response to this question mentioned that it was too early to conduct such evaluation, that they did not have the time and resources to do so, and that it was not up to them to evaluate these activities since evaluation was generally done by other stakeholders in their institution.
The intent of local HTA units to involve patients and the public in their activities was also explored in the pre- and postmeeting surveys. Table 2 shows that more local HTA units intended to involve patients after the meeting whereas fewer units were interested in involving both public and patients after the meeting. Moreover, after the meeting fewer respondents stated that they may involve public or patients within the next few years. This could mean that the information provided during the meeting changed their perceptions towards patient and public involvement in HTA activities, and respondents felt that involving patients in HTA activities was perhaps more feasible in the short term.
Table 2. Plan to Involve Public or Patients in Local HTA Activities (n = 11)
The premeeting survey also explored what local HTA units needed in order to involve patients and the public in their activities. Table 3 presents the identified needs in order of importance (i.e. frequency).
Table 3. Needs of Local HTA Units to Involve Public or Patients in HTA Activities (n = 11)
In the postmeeting survey, we also asked the local HTA units to prioritize their needs on a Likert scale from useless (0) to essential (4). We slightly modified the wording of the proposed needs for the second survey as a result of comments received. Notably, we provided additional precision to the items related to “Working in partnership” and “Support and resources.”
As shown in Table 4, the most highly prioritized needs postmeeting were: (i) recruitment and selection of patients, users, informal caregivers, and members of the public; (ii) integration of experiential knowledge in HTA products; (iii) knowing and implementing the best methods and practices for partnership; and (iv) training and coaching tools for patients, users, informal caregivers, and members of the public. Although the majority of respondents considered evaluation of patient and public involvement in HTA as an important need, none regarded it to be essential. Thus, the prioritization of needs after the meeting followed the pattern of the premeeting consultation, with the exception of the need for evaluating patient and public involvement in HTA.
Table 4. Prioritization Scores of the Needs (n = 11)
Note: Top 3 indicates if the item was classified as one of the three top priorities in the local HTA unit.
Discussion
The aim of this study was to collect information about patient and public involvement in the processes of local HTA units in Quebec, Canada. Although the INESSS is the provincial HTA agency in Quebec and not a local HTA unit, it was deemed important to include it in the survey since it is highly involved in the CoP and often collaborates with local HTA units.
Results of the first survey indicated that seven out of the eleven local HTA units already involve patients in their activities. Following a CoP meeting at which the survey results were presented, they will continue to do so but more systematically. These units also expressed their need for tools to help them to more effectively involve patients and members of the public in their activities. As patient involvement was relatively new for them, they did not feel adequately equipped to deal with this new aspect of their activities. Some of the questions raised by the units reflected this feeling: “How to develop abilities to understand the patients’ experiences? How to make HTA experts and decision-makers aware of the value-add of patient, user, caregiver, and citizen involvement? How to build a partnership with patients and not only asking them to participate?” These interrogations are fundamental and may lead to some difficulties in the execution of their activities if not appropriately addressed (15;16). It was thus important to clearly identify what were their most important needs in relation to patient and public involvement. The three most important ones we identified in this study were: recruiting and selecting patients; integrating experiential knowledge; and knowing and implementing the best methods and practices for partnership. Since these needs were identified as a priority, they should be given full consideration by academic researchers, as well as by healthcare institutions to help HTA units in their mandate to provide the most up to date and contextualized data to decision-makers to promote better health care and services. Indeed, it is increasingly recognized that involving patients and the public could increase the effectiveness and impact of local HTA units since it will help the units to better identify and address local issues, as well as to provide more suitable recommendations (5;10;17;18). This still needs to be more evaluated in local HTA units to support this evolution, but, paradoxically, although the majority of respondents in our survey considered evaluation of patient and public involvement in HTA as an important need, none regarded it to be essential. To evaluate this involvement and its impact on HTA processes and efficiency would help to improve this new collaboration for a better use of resources. If not, one may get the sense that local HTA units relate to “iterative, adaptive locally-developed, and pathway-dependent processes” for involvement, which fits in more with the hospital/service delivery model of quality improvement following the Plan-Do-Check-Act model as they learn and evolve (Reference Seidl and Newhouse19).
Another point that was identified in this study is that members of the public were infrequently involved in HTA activities, contrary to patients. This may be explained by the natural tendency in local HTA units to consult individuals directly concerned by the topic assessed, which is the patient. In this setting, it may be considered less important to consider the point of view of members of the public, even when it is recommended by researchers and HTA organizations (Reference Carvalho, de Sousa, Barreto and da Silva20–Reference Oxman, Lewin, Lavis and Fretheim21). Another reason may be the confusion surrounding the definitions of a patient and a member of the public or a citizen. Indeed, some respondents found it difficult to differentiate between these stakeholders, which was reflected in the discussion during the meeting and in some of the answers to the open-ended questions in the survey.
A strength of this study is that we were able to mobilize the CoP around this topic and to get a very high response rate (i.e. eleven units over twelve) to our surveys. As such, the internal validity of this study is very high. In addition, the composition of the CoP was a mix of HTA units pertaining to different sectors. While some were more oriented toward social services others only dealt with physical or mental health. This led to various points of view, which we think are representative of the diversity of local HTA activities and reinforce the external validity of this study, even if it was focused on the province of Quebec. However, it is important to indicate that this province has one of the highest densities of local HTA units in the world (Reference Sampietro-Colom and Martin8), which makes it a fertile ground for observing trends in local HTA. Also, this study is, to our knowledge, the very first to address this topic using local HTA units, thus paving the way for future research.
However, this study has some limitations. First, the participants were from the same community of practice and showed interest in the topic by participating in the meeting on patient and public involvement in local HTA, which could introduce a positive bias in their responses. Second, the survey questions were developed by the authors and were not pretested with potential respondents prior to the survey, which could have led to a lack of clarity of some questions. Third, the use of a survey limited the possibility to collect more nuanced responses, although the open-ended questions provided some complementary information. Structured focus groups or individual interviews may have provided additional information. Finally, the survey was conducted 3 years ago and only provides a snapshot of local HTA units and their activities at that time. Indeed, several local HTA units have deepened their collaboration with patients, users, caregivers, and citizens since the survey was conducted in 2017, which in some part was a consequence of the survey itself (personal communication with local HTA unit managers). Also, at the time of the survey, it was the intention of many units to strengthen their collaboration with the patient partnership service of their institution to standardize the recruitment of patients in HTA processes, which may have led to a more systematic contribution of patients' knowledge as well as the development of their own tools to better integrate these in HTA activities (12;22–23). As a consequence, it would be interesting to further assess how the practices of local HTA units changed since this survey was done.
Conclusion
This study clearly identified that local HTA units in Quebec were already investing efforts to involve patients in their processes in 2017, but that members of the public were less involved. Local HTA units also largely recognized that they needed specific information to help them better involve patients and members of the public in their processes. In this regard, the three most important needs identified were in: recruiting and selecting patients; integrating experiential knowledge; and knowing and implementing the best methods and practices for partnership. As such there is an important need to develop tools to help them more effectively involve patients and members of the public in their processes, which in turn may improve the effectiveness and impact the HTA units have in their healthcare institutions. In this setting, they will benefit from the recent initiative of the International Network of Agencies for Health Technology Assessment (INAHTA) to create a patient engagement learning group, as well as from the experiences of national HTA agencies that are already involving patients and the public in their processes (Reference Weeks, Polisena, Scott, Holtorf, Staniszewska and Facey6).
Acknowledgments
We acknowledge all of the collaborators and participants in this study. We thank anonymous reviewers for their valuable comments. TGP is fellow of the FRQS (Fonds de recherche du Québec—Santé) and member of the FRQS-funded Centre de recherche de l'IUSMM.
Financial Support
This research received financial support from the Canadian Institutes of Health Research, Planning and Dissemination Grant—Institute Community Support (#354915).
