Many jurisdictions are moving toward greater patient and public involvement in health policy and practice, including health technology assessment (HTA) (Reference Kreis and Schmidt1–Reference Abelson, Bombard, Gauvin, Simeonov and Boesveld3). In recent years, pressure to involve patients and the public in HTA has grown (Reference Menon and Stafinski4). Patients can give their perspectives on experiences, attitudes, beliefs, values, and expectations about health, illness, its effects, and concerning health technologies (Reference Facey, Boivin and Gracia5;Reference Pivik, Rode and Ward6). Thus, patient involvement in HTA should help produce care that is responsive to their needs and values (7;8). Along with providing experiential knowledge, it is believed that involving patients in health decision-making will promote a sense of empowerment and contribute to more efficient solutions as regards the distribution of scarce health resources (Reference Pivik, Rode and Ward6;Reference Segal9).
Although authors report increasing patient or caregiver engagement in the production of knowledge in the HTA process, there is little focus on evaluating the impact of such involvement (Reference Menon and Stafinski4;Reference Gagnon, Desmartis and Lepage-Savary10–Reference Crawford, Rutter and Manley12). Moreover, little is known about perceptions of key stakeholders regarding the input of patient involvement when they have participated in an activity with patients. Consequently, we sought to evaluate two strategies of patient involvement in the assessment of alternative measures to restraint and seclusion among adults in short-term hospital wards in psychiatry and in long-term care facilities for the elderly. The term “alternative measures” includes intervention strategies designed to avoid the use of restraint or seclusion (13). Restraints are measures to restrict a person's freedom of movement using physical force or mechanical devices (Reference Dimant14). Seclusion, a type of restraint, involves confining a person in a room from which the person cannot exit freely. The alternatives measures identified in this HTA were constant observation, occupational and behavioral approaches, patient evaluation, movement detection or observation systems, and equipment such as alarms, fall-prevention carpets, and low beds (Reference Gagnon, Desmartis, Dipankui, Gagnon and St-Pierre15). In this study, the term “patient” encompasses service users in psychiatry or family members for elderly people in long-term care facilities.
METHODS
Study Context
In partnership with the HTA unit of the CHU de Quebec, researchers from Laval University and from the CHU de Quebec Research Centre conducted a research project to set up and evaluate interventions that included patients in the assessment of alternatives to restraint and seclusion among adults in short-term hospital wards (in psychiatry) and long-term care facilities for the elderly (Reference Gagnon, Gagnon and St-Pierre16).
The HTA unit of the CHU de Québec is assisted by two distinct entities in every assessment: a working group with experts and healthcare professionals from the area of concern of the assessment, and a scientific committee whose mission is to endorse the assessment products (reports, informative notes) and the scientific methodology. For the assessment of alternatives measure to restraint and seclusion, two main reports were produced by the HTA unit of the CHU de Quebec, each of them leading to recommendations: one on constant observation (17) and another one on other alternatives (18). These reports were an attempt to respond to the concerns of healthcare professionals and decision makers on the efficacy, safety, and respectful nature for the patient of alternative measures used in the network (19). These reports were analyzed for this study.
In the context of this research project, three patient representatives from mental health community organizations were exceptionally involved in the working group set up for the assessment. In addition, consultation of service users (in psychiatry) and family members (for elderly people in long-term care facilities) on seclusion, restraint, and their alternatives was done through focus groups for this research project (Reference Gagnon, Desmartis, Dipankui, Gagnon and St-Pierre15). This consultation led to the production of a report and a published study by the research group that were also considered in this study.
Participants
We sought participation from members of the working group, patient representatives from mental health community organizations who participated in this HTA, members of the local HTA unit of the CHU de Quebec, researchers involved in the project, and members of the local HTA unit scientific committee. The Ethics Research Committee of the CHU of Quebec approved this research project, and all participants signed a consent form.
Data Collection Process
We conducted thirteen semi-structured interviews (eight face to face and five by phone), from September to December 2013, to explore two main questions: (i) How was the point of view of patients considered in this HTA report? (ii) In your view, what did patient involvement change in this HTA report and its recommendations?
The average length of interviews was 15 minutes and data were collected until saturation. The link to the HTA report was sent to participants few days before the interview, as well as the main questions to be discussed.
Interviews were audio-recorded and transcribed verbatim. The first author (M.T.D.) took field notes that were compiled in a diary. She also analyzed six documents that were produced in relation with this HTA: two final HTA reports with recommendations, a report and a published study from the consultation with service users in psychiatry and family members for elderly people in long-term care facilities (Reference Gagnon, Desmartis, Dipankui, Gagnon and St-Pierre15), and two PowerPoint presentations.
Data Analysis
The information from interviews and field notes was analyzed using N'Vivo 8 software. We performed a thematic content analysis according to the method described by Huberman and Miles (Reference Huberman and Miles20), comprising coding, organization, and linking. Two people codified the first five interviews independently following a list of themes and subthemes previously elaborated from the analysis of the first two interviews. The main themes of this list included: (i) input of patient experience in this HTA; (ii) integration of patient views in the report and recommendations; (iii) effects of patient involvement strategies on the report and recommendations. As the analyses proceed, themes and subthemes were added or revised mainly after discussion and agreement with the second person. Then, one person codified the remaining interviews and the two people together reviewed some codes and extracts for which doubts existed with regards to their codification.
We also extracted information from the two HTA reports produced for this assessment on two main components: “What healthcare professionals have said” and “What patients have said”. This content and the other four documents were then triangulated to validate and complete the information collected from interviews.
RESULTS
Participants' Characteristics
We interviewed thirteen people, including three patient representatives and five members from the working group, two local HTA unit members, two researchers, and two members of the local HTA unit scientific committee (one of whom was also a member of the working group).
Input Concerning Patient Involvement in the HTA Report and Recommendations
Integration of Patient Perspectives in the Report
All participants emphasized that patients' perspectives were considered in the HTA report. More specifically, the report (18) was well organized to reflect the views of healthcare professionals and services users consulted for this HTA.
In fact, from the HTA report analysis, we noticed that the distinction between what comes from patients and from healthcare professionals is clear. Sections are dedicated to patients in the methods and the results sections. Thus, the process of data collection with patients and the results of their consultation are clearly presented. However, in the report summary, the key issues arising from data collected from patients as well as some results from their consultation are integrated with what comes from healthcare professionals.
For some participants without patient involvement in this HT the main findings would have focused more on technical and organizational aspects rather than on their engagement in the care process.
If I had to say something about a scenario where there was no patient consultation or participation, I guess this assessment would be more focused on measures as such, some considerations concerning their use, and on organization, but in this instance the side of engaging users, peer support, and families really stood out [. . .] communication and listening too [. . .]. (R1)
This view was confirmed when analyzing the HTA report. Indeed, we noticed that, in addition to organizational aspects, the patients' perspective was highlighted in the main findings and in the conclusion as well. However, the transcripts are not included in the report (except one excerpt), and some sentences from the patient consultation report have been reformulated.
Inform Recommendations
For over two thirds of participants, the views of patients and their representatives also informed recommendations proposed in the HTA report, in particular through one of the twelve main findings underlying the recommendations, namely:
[. . .] that listening and communication on the part of stakeholders, as well as the involvement of families, relatives or peer-support workers in the episode of care are key elements to be explored to reduce the use of control measures. (p. 31 HTA report)
One of the recommendations to healthcare facilities focuses on the implementation of one or more inter-institutional working groups to lead a reflection and broad consultation on a greater involvement of the patient, his or her family (or relatives), and/or peer-support workers in the episode of care. According to a HTA unit member, “without users' participation, this recommendation might not have been made.”
However, despite the consideration of the views of patients in the report and recommendations, their involvement in the follow-up to the recommendations was not outlined, as noticed by this participant.
[. . .] I found nowhere in the recommendations any suggestion to include families and relatives in the follow-up to these recommendations. [. . .]. I find it interesting that they added content and it was taken into account. But in the follow-up, they are not there yet. (HM2)
The next sections focus on the input of each involvement strategy on the outcomes of this HTA.
Input of the Consultation
Proposing Alternative Strategies
From the analysis of the HTA report and the published study on patients' consultation (Reference Gagnon, Desmartis, Dipankui, Gagnon and St-Pierre15), we noticed that patient consultation led to suggesting ways to reduce the use of restraint and seclusion. Patients suggested strategies such as their involvement in the therapeutic process, the involvement of peer-support workers and family members in care, and better listening and communication from caregivers. Patients also pointed out the importance of adapting the measures to each individual and his or her situation rather than trying to implement a “one size fits all” intervention. Furthermore, the consultation of patients has made it possible to identify barriers and facilitators related to staff attitude regarding the use of alternative measures to restraint and seclusion. According to a researcher, this point of view could only be revealed through consulting patients.
From the document analysis, we could also observe that barriers and facilitators related to physical surroundings (cohabitation of service users with different problems), factors related to service users (non-compliance with medication, drug addiction), and external social factors (lack of knowledge about mental health issues, fear and stereotyping, inappropriate police interventions) emerged only from consultation with patients.
Confirming Other Findings
The analysis of the HTA report and the report from patient consultation also revealed that many of the issues arising during consultations with both healthcare professionals and patients are widespread. Findings such as the importance of adapting alternatives measures to each individual rather than trying to implement an intervention for all, the problems of constant observation by a security guard, the importance of staff training, the disadvantages associated with the use of technologies such as observation systems and motion detection systems, and the crucial role of physical surrounding were mentioned in both groups.
Input of Direct Participation of Patient Representatives in Working Group Activities
Hearing the Voice of Patient Representatives Directly
For the two members of the local HTA unit interviewed, the participation of patient representatives in the working group provided them with direct access to the patients' voice. Patient representatives were able to formulate comments in light of what they had retained from the documents they received and of what was presented during meetings. According to a researcher, their comments may have influenced some elements of discussion in the report, such as “pervasive risk management, which involves a certain tension in the practice setting as regards the desire to provide care that is of high quality, safe and respectful of the patients' needs” (p.26 HTA report).
Rephrasing Some Findings
The participation of HTA unit members in an ad hoc meeting with patient representatives allowed them to rephrase some observations to adjust to the patients' perspectives.
One of the findings did not seem to be grasped or understood, or at the very least, seemed to run counter to the views of the patient representatives. And this made us think and make changes so as to adjust this observation. Not in the sense of creating a bias, but to adjust so as to take into account what had bothered the representatives (. . .). (HTAUM1)
Indeed, an avenue suggested for further reflection in the PowerPoint presented in the ad hoc meeting focused on “changing the ‘zero restraint’ approach for the use of control measures as a last resort through adopting the ‘least possible restraint’.” According to a patient representative, changing the statement could reduce efforts by caregivers to avoid using control measures. This view has been considered in the report, as evidenced in the conclusion: “The quasi-elimination of control measures is a reachable objective in the health network, but it requires a certain shift to address solutions under the various angles of the management and organization of care.” (p.33 HTA report)
Challenges to Evaluating the Input of Direct Participation
For an HTA unit member, it was difficult to say whether the input from patient representatives changed the results of this HTA because their comments were consistent with the data already collected from the consultation with patients. In addition, a researcher pointed out the difficulty of distinguishing the effects strictly linked with direct participation from those associated with consultation, because some issues (e.g., pervasive risk management, the need for measures that are safe and respectful of patients' needs) were discussed in both of these involvement strategies. However, the direct participation of patient representatives in working group activities led participants to consider more human and patient-centered issues.
According to researchers and an HTA unit member, a main barrier to evaluating the real input of direct participation was the involvement of patient representatives in the working group once the issues of interest and the objectives of the assessment were already defined. Their participation could have been different if they had been involved earlier, because they would have been able to influence the issues and objectives of the assessment.
DISCUSSION
To the best of our knowledge, this study is the first to evaluate two strategies of patient involvement, consultation and direct participation, in a specific HTA. Globally, it showed that each strategy helped the integration of patients' perspectives in this HTA. This finding leads us to make the following principal observations.
First, the patients' inputs from consultation were presented in a separate part of the results section of the HTA report, allowing for a better consideration of patients' needs. This presentation is close to what is done in Danish HTA reports, which include a separate chapter about patient aspects that are based on secondary or primary research (Reference Hansen, Lee and Van Randwijk21;Reference Tjornhoj-Thomsen and Hansen22). Presenting patient aspects apart in these reports is explained by the fact that, in Denmark, HTAs are usually framed around four key elements: the technology, the patient, the organization, and the economy (Reference Hansen, Lee and Van Randwijk21). This framework influences how HTA is conceptualized, how the assessment process is conducted, and how knowledge is produced and presented in HTA reports (Reference Tjornhoj-Thomsen and Hansen22). This observation brings us to suggest that the Danish model could be followed to ensure that the patient is explicitly considered by HTA agencies.
Second, patient consultation in this HTA yielded alternatives that could reduce the use of restraint and seclusion other than those suggested by healthcare professionals. It also helped to validate some opinions brought forward by other stakeholders involved in the HTA process, as was the case in the study by Brennan (Reference Brennan23) on the engagement of blood services stakeholders for organ and tissue donation. In accordance with our findings, Brennan (Reference Brennan23) underscored the importance of integrating the input of patient consultation with other forms of evidence to favor more informed health technology decisions and recommendations.
Third, the direct participation of patient representatives made it possible for their comments to be directly heard by other stakeholders. It also led HTA unit members to rephrase some observations so as to better reflect their perspective. However, it is difficult to determine the real input of this involvement strategy because, due to delays in starting the research project, patient representatives did not attend the first meeting of the working group when the issues and objectives of the HTA were defined. They were therefore unable to influence the definition of alternatives to consider or the evaluation questions. Despite their involvement in an ongoing assessment, patient representatives helped focus discussions on more human and patient-centered concerns. In that regard, Boivin et al. (Reference Boivin, Currie and Fervers24) noted that some patient and consumer organizations have become quite sophisticated in their understanding of the evidence, and that patient representatives' involvement could strengthen the quality of clinical practices guidelines by expanding the range of evidence being considered and by questioning certain expert assumptions.
Our study then confirms the importance of considering research on patient and their representative involvement in mental health as a method that could influence service provision (Reference Simpson and House25). As Simpson and House (Reference Simpson and House25) point out, there is a need to allocate funding for involving patients and their representatives, and to monitor and investigate such processes so that effective projects are sustained (Reference Simpson and House25).
Study Limitations
MTD was in close contact with some members of the research team that set up this experimentation, which could have influenced her interpretation of the results. However, field notes compiled in a diary, documenting impressions and helping identify possible influences, mitigated this bias. Moreover, given that the input of direct participation is based on only one meeting of the working group, the conclusions must be nuanced.
Another limitation of this study concerns the transferability of the results with regard to other HTA topics. Restraint and seclusion is a sensitive subject, which mobilizes mental health community organizations. This limits the transferability of our results to HTA topics where there is less involvement of community groups.
CONCLUSIONS
Patient consultation was seen as having directly influenced the content of the HTA report, while direct participation made it possible to rephrase some findings. This study is one of few studies to assess the impact of patient involvement in HTA and more such studies are needed to identify the best ways to improve the input of such involvement to significantly improve HTA decisions.
CONFLICT OF INTEREST
The authors declare no conflict of interests.
