Public involvement in health policy making has been a durable and increasingly popular trend since the 1980s, having been implemented in many developed countries, including Canada, the United Kingdom (UK), Australia, Denmark, Sweden, Finland, and France. The involvement of patients and consumers in health technology assessment (HTA) is no exception, and an increasing number of successful experiences in different parts of the world have been reported (Reference Caron-Flinterman, Broerse and Bunders7;Reference Jørgensen, Hvenegaard and Kristensen19;23;Reference Pivik, Rode and Ward27). Whereas dissemination and uptake activities are thought to be intrinsic to HTA (Reference Battista and Hodge5;Reference Eisenberg8;Reference Menon and Stafinski22), little is known about how formal patient and consumer groups use HTA findings and whether these groups can be part of the dissemination process.
USE AND DISSEMINATION OF HTA ARE TWO DIFFERENT STEPS
The use and dissemination of HTA reports by patient and consumer organizations are two very different steps within the HTA process, with different outcomes. Whereas HTA use involves the utilization of findings by organizations and results in the endorsement of assessment reports by formal patient and consumer groups, HTA dissemination involves the communication of results to a targeted audience within the community at large—either individuals or groups—and results in the communication of the findings and recommendations of reports to a larger audience. In this study, we use the term “patient and consumer organizations” to designate formal groups that bring together patients with a specific condition or people with a similar health situation.
There is clearly an increase in the use of information technologies by patients seeking to be better informed when making decisions about their health and to actively participate in clinical decision making (Reference Fox, Ward and O'Rourke11;Reference Thompson30). Involving patients and consumers in HTA dissemination would facilitate such information uptake (Reference Entwistle, Renfrew, Yearley, Forrester and Lamont9;Reference Entwistle, Watt and Davis10). It can help tailor the message to the intended audience, while providing information that corresponds to patients' needs and concerns (Reference Greer, Goodwin, Freeman and Wu13). In general, research uptake by different stakeholders is better when less technical language is used and when the diffusion and dissemination channels are purposefully chosen (Reference Entwistle, Watt and Davis10;Reference Lavis20). A systematic review of consumer involvement in developing patient information material, which examined two randomized control trials (Reference Nilsen, Myrhaug, Johansen, Oliver and Oxman24), showed that “consumer consultation prior to developing patient information material resulted in material that is more relevant, readable and understandable to patients” (p. 10).
However, consumers' involvement in research dissemination is not without challenges. For instance, Royle and Oliver (Reference Royle and Oliver28) report the “disproportionate effort required” (p. 496). The Australian government recently produced a guide for health information producers on how to effectively involve consumers and communities in health information development and dissemination (23). The authors highlight the distinctive and beneficial role played by consumers and the community in the process and highlighted the importance of building partnerships with various groups. Leadership and creating a favorable context for long-term partnerships with community members are expected of HTA producers (Reference Lomas, Fulop, Gagnon and Allen21;23;Reference Pivik, Rode and Ward27).
In this study, we argue that HTA uptake and patient association involvement in developing and disseminating HTA reports also depends significantly on organizational and cultural factors intrinsic to patient and consumer groups, such as resources, information processes, values and beliefs, and the organization's stated purpose. By examining these factors, this study extends current knowledge by focusing on the perspectives of patients and consumers, and by exploring both the organizational factors required during the process and the experience of groups who are not regularly involved in policy- or research-driven initiatives. We suggest that a better understanding of these groups' interests and contingencies (Reference Hartley and Benington15) will help HTA agencies to better target, approach, and collaborate with formal patient and consumer groups.
DIFFERENT TYPES OF USE
This current study stems from a larger study that examines the various roles played by the media, doctors, and patient groups in HTA production, dissemination, and uptake in Canada. Here, we turn our attention to how and why patient and consumer organizations use HTA findings within their organization, and what factors influence how and when they communicate their findings to members and other organizations. The framework we use to support our data analyses is informed by a previous study (Reference Hivon, Lehoux, Denis and Tailliez16).
In an attempt to categorize how science knowledge is used in public policy, Hivon et al. (Reference Hivon, Lehoux, Denis and Tailliez16) used the three types of knowledge use described by Peltz (Reference Peltz, Yinger and Cutler26): instrumental, conceptual, and symbolic. In patient groups, instrumental knowledge use includes activities that incorporate research findings as a basis for action; for instance, when research findings are used to support an argument or the organizations' needs, or to build new objectives. When research results are used to inform or further thinking about an issue or debate, knowledge use is said to be conceptual. This is the case when associations make a position statement on some public policy. Finally, if research results are used to justify the organization's position vis-à-vis that of another organization such as the government, the knowledge use is symbolic. In addition, the study identified scientific and material limitations to HTA use by patient groups, such as a lack of knowledge brokers to translate research findings into clear messages, and a lack of resources and know-how for accessing and using scientific information (Reference Hivon, Lehoux, Denis and Tailliez16).
METHODS
The broader study was designed around four controversial HTA reports produced by Canadian agencies (AETMIS in Quebec [Agence d'évaluation des technologies et des modes d'interventions en santé] and ICES in Ontario [Institute for Clinical Evaluative Sciences]): prostate-specific antigen screening (PSA) in asymptomatic men (17), the effect of the Ontario drug plan on patterns of use and cost of new drugs such as COX-2s (18) (COX-2s are a class of drugs that selectively inhibit the action of cyclo-oxygenase-2, a peptide found at inflammation sites. These drugs are known to relieve pain and inflammation while limiting gastrointestinal side effects), the use of electroconvulsive therapy (ECT) (2), and first-trimester prenatal screening for Down's syndrome (3) (see Box 1 for a summary). Each of the four issues constitutes a case study. We used the reports as a basis for discussion with participants. We chose a qualitative type of inquiry because it is a useful approach when trying to understand organizational processes and the context in which they are implemented (Reference Patton25).
Box 1. Description of the case studies: Four HTA reports on controversial health technologies.
Prostate-specific antigen screening in asymptomatic men (17)
Prostate-specific antigen (PSA) screening relies on a blood test to detect growing prostate tumors. The major goal of population-based screening is to decrease mortality. However, the findings of this HTA report do not demonstrate a clear benefit to such screening because of the potential harm associated with aggressively treating men who test positive but whose tumor is indolent or who do not in fact have a tumor (false-positive). Prostate cancer treatment is associated with some major complications, including erectile dysfunction and incontinence. The authors of this study recommend continuing the approach of not paying for PSA testing in asymptomatic men and providing testing on request to patients who are fully informed about the risks associated with the test.
Effects of Ontario drug plan coverage for new drugs on patterns of use and costs (18)
This HTA report examines the use of two new classes of drugs: cyclo-oxygenase-2 inhibitors (COX-2s) and atypical neuroleptics, which are prescribed under different drug coverage policies in Ontario and British Columbia. COX-2 inhibitors are a type of nonsteroidal anti-inflammatory drug (NSAID), and are known to cause fewer gastrointestinal side effects than regular anti-inflammatory drugs. However, they are more expensive than conventional NSAIDs. In Ontario, the policy for COX-2 inhibitor coverage requires that a code be written on the prescription, while in British Columbia, the process is more complex and requires obtaining an approval beforehand. COX-2 prescriptions in Ontario almost doubled in the first year of coverage, and coverage costs almost tripled, whereas prescription rates remained unchanged in British Columbia after the new coverage policy was instituted.
Use of electroconvulsive therapy (2)
Electroconvulsive therapy (ECT) consists in applying electrical currents to the brain, under general anesthesia and while vital signs are monitored. The risks of ECT are related to the currents given and to the anesthesia, and are mostly of a cardiovascular or cognitive nature. They usually last for a short period of time. The authors of this HTA report note that efficacy varies between psychiatric conditions and, therefore, its use should be adapted accordingly. For instance, it should be an accepted treatment in cases of major depression and pernicious catatonia, whereas its use should be limited in cases of schizophrenia. AETMIS also recommends that the use of ECT be better regulated by means of clinical practice guidelines, patient information, and quality control programs.
First-trimester prenatal screening for Down syndrome and other aneuploidies (3)
Prenatal screening for aneuploidies (chromosomal anomalies) such as Down syndrome can be performed in the first and second trimesters of pregnancy. When performed during the first trimester of pregnancy, the screening includes two tests: maternal serum markers and ultrasound screening for nuchal translucency. In this HTA report, the authors note that while the efficacy of first-trimester prenatal screening is satisfactory, its effectiveness still needs to be demonstrated. Hence, AETMIS does not recommend implementing wide-scale prenatal screening during the first trimester of pregnancy.
For each of the four issues, we selected associations from the selected jurisdictions based on their mission and activities: formal organizations that directly provide support, information, or advocacy services to patients and consumers in that subject area. We excluded those who acted mainly as a research foundation or professional association. We then approached the highest ranking administrators in the selected organizations. In total, we conducted sixteen semistructured interviews with patient and consumer associations. They lasted an average of 45 minutes (30–150 min). All interviews were tape or digitally recorded and transcribed verbatim in their original language (either English or French).
We first coded transcripts by using predetermined codes and themes, based on organizational theories (Reference Germann and Wilson12;Reference Harrisson14) and the three types of knowledge use described previously. Second, we observed emerging themes across cases and within patient and consumer groups using an open coding method (Reference Strauss and Corbin29). We compiled the results in tables to better compare and contrast the cases. Once the analysis was complete, the empirical data and conclusions were validated by one informant.
RESULTS
We first present our findings on the use of HTA reports by patient and consumer organizations for each case, beginning with a brief description of the usual sources of knowledge. In Table 1, we summarize the types of knowledge use observed for each case. Finally, we look at how and why dissemination is carried out by some of these lay organizations.
Table 1. Examples of Types of Knowledge Use Exhibited by Patient and Consumer Organizations for the Four Case Studies
Use of HTA Reports by Patient and Consumer Organizations
COX-2 Case Study
Organizations concerned with COX-2s all made considerable use of HTA in their activities, either directly through reports published by the HTA agency or by reading peer-reviewed articles and articles on scientific advances in newspapers, chatting with patients or colleagues, and attending conferences on topics of interest. HTA reports published by the provincial agency were a regular, although not frequent, source of information.
These organizations used HTA findings “instrumentally” when advocating for access to a new drug, as an evidence base when writing letters to health authorities. One participant reported:
“In fact, we define our advocacy as being ‘education with persuasion.’ So we really concentrate on being as well informed as we can be in the areas that we operate. [. . .] We try to make sure that our members can incorporate this information into their own needs for advocacy.” (C2)
Such an approach is particularly common when research findings for a new medication are conclusive, or when, as is the case for COX-2s, the costs of the medication are not fully covered by Canadian drug plans.
The organizations also used biomedical information “conceptually,” to take a stance on a new drug. In the case of COX-2s, for example, organizations focused on the known benefits of these drugs to support their assertion that they should be more accessible (i.e., covered by public drug plans).
PSA Case Study
In the case of PSA, the organizations all used biomedical information taken directly from journals, retrieved using search engines, or forwarded to them by friends or colleagues. Here again, the ICES report was not the usual source of biomedical information; rather, it was mainly electronic media and newspapers.
In this case, the instrumental use of HTA was more purposeful, in the sense that information from the biomedical literature was often carefully selected, based on whether or not the conclusions were aligned with the organization's advocacy purposes.
“I used [the ICES report] to help build my case when I was making presentations to politicians and to other organizations that I was trying to develop as allies. My main goal was having PSA screening covered under the Ontario Health Insurance” (P2).
Thus, organizations may have basic assumptions about PSA screening, about which they are firmly convinced, and they will then use some precise information to provide arguments supporting their position when they are lobbying. For instance, the organization quoted above was, in fact, not satisfied with this report's conclusions and was very critical of the technology assessment process: “[. . .] If the people who are carrying out the study know what outcome is expected, I think it's impossible for the conclusion of that study to be impartial, and to say that science is not political is naive.” (P2)
Yet, this organization uses biomedical information for advocacy purposes. Is there a contradiction? This organization did use findings and experts opinions from sources other than the provincial HTA agency when seeking to support their advocacy efforts. This means they purposefully chose the information on which to take action, but the direction of their actions remained fixed.
By contrast, a representative of another national organization explained why they rely on experts' opinion: “[This report] reflects the position that we actually have regarding PSA [. . .] we are a shy organization; we will rally to what the scientific community judges to be good.” (P1). When comparing this statement with the one made above by the other organization, it is clear that a given HTA report and its recommendations can be used very differently. Thus, the instrumental use of this report, in this case, seems to be closely related to what the organization values more: better access to screening or maintenance of the organization's credibility as reflected in the quality of the information it disseminates.
With regard to the conceptual use of biomedical information, all of the organizations concerned with PSA used it to directly inform their members or the patients they represent about the use of the technology.
Down Syndrome Screening Case
For this issue, research findings were rarely taken directly from the provincial agency HTA report or even from peer-reviewed journals. As some of the respondents pointed out, “[The HTA report] is in fact very medical . . . maybe this is why. We are more ‘social’. And I am convinced that [the authors] think they are being holistic, but when you have seen it from the outside, they are not at all.” (D2). Another noted: “[The report] is so technical. . . at some point we have to circumscribe our mission, and that's why we don't go into the medical stuff” (D3). These organizations often had a psychosocial knowledge base and used the psychosocial literature. The more “medical” type of biomedical information was taken from the general media, and there was no active search of information regarding health technologies.
Not surprisingly, these organizations raised some social and ethical concerns related to the wide implementation of such screening: “Will we come to ostracize those with Down syndrome who were not caught by the ‘machine’?” (D3). In fact, no “instrumental” use was made of this specific HTA report on Down syndrome screening. An example of such instrumental use of this report would be when these groups carry out advocacy activities against large-scale screening. We explain our result by the fact that these organizations (associations of people with Down syndrome and their parents) agreed with the report's recommendation not to make available such a prenatal screening test on a large-scale.
The “conceptual” use of biomedical information was often evident; for example, when providing information to parents of children with Down syndrome, and to parents “confronted with screening tests and decisions they have to make: it provides us with information we can use when answering their questions” (D1).
ECT Case Study
In this case, biomedical information was obtained through newspapers, conversations with colleagues, or meetings with partner organizations. The psychosocial literature was used as often as the biomedical literature, followed by experiential knowledge. Experiential knowledge was described in the literature as one that is specific to patients, and that results from personal reflection following a lived experience of illness, care, or cure (Reference Caron-Flinterman, Broerse and Bunders7).
The use of knowledge was particularly blurred in the case of ECT: all three types of knowledge use were present but none was clearly predominant. For instance, an example of the instrumental use of this report would be using it to support efforts aimed at increasing access to the technology for patients who could potentially benefit from it (a recommendation of the report). This was observed in one instance, for an organization advocating for better access to alternative therapeutic treatments for mental health conditions. Therefore, the organization used other biomedical literature rather than the AETMIS report.
One striking observation was the propensity of some of the respondents (2 of 6) to use HTA reports symbolically. Usually, this stance would be favorable to the use of ECT, but what we observe here is different. These respondents used elements in the report to justify their position against the recommendations made within the report. More precisely, patient associations—who were mainly patient advocacy groups—took elements put forward in this report to support a position that was opposite to the report's recommendations. In one study in which the second author participated (P.L.), such symbolic use by patient organizations was not observed and it was suggested that such a phenomenon is unlikely because these organizations rarely make official decisions (Reference Hivon, Lehoux, Denis and Tailliez16). The data collected for this study bring more precision: advocacy organizations that are opposed to controversial technologies such as ECT may in fact be using parts of the report, taken out of context, to justify their unpopular position to the general public or governments.
The Mission and Knowledge Base of a Patient or Consumer Organization Shape Its HTA Utilization
In our interviews, we further explored the impact of a group's mission on the type of knowledge use. The mission statement and activities of organizations that did not use any HTA results were less closely related to health technologies (this was the case for some groups related to ECT and Down syndrome screening). For example, when the only approach taken by an association is prevention, or when the main service provided is support through active listening, only information about psychosocial issues or public health prevention are needed and sought. As one mentioned, “It was brought to my knowledge that there was a report on ECT, but that's all. Unless it involves promotion or prevention—we try to stick to our mission—[. . .] we won't go into treatments.” (E1).
Table 2 also shows that an organization's knowledge base is linked to how it uses knowledge. For instance, those with a biomedical knowledge base use knowledge instrumentally (noted as “B” in Table 2), and those that use only experiential knowledge as a basis for decision making and action tend to use research symbolically. Those who use psychosocial knowledge barely use HTA-related information and, if they do, they do so conceptually. Moreover, organizations whose primary aim is to advocate for better access to health services usually use biomedical information.
Table 2. Mission, Knowledge Base, and Types of Health Technology Assessment (HTA) Use for the Patient and Consumer Organizations Included
Note. “+” indicates the presence of the characteristic; “−” indicates its absence; “+/−” indicates the characteristic is present but not systematically.
aN, National; P, Provincial; C, Community-based.
bB, Biomedical literature (includes HTA); E, experiential knowledge; G, general literature diffused by the press and other media; P, psychosocial literature.
HTA Dissemination by Patient and Consumer Organizations
Sometimes the HTA findings were communicated to the association or organization's members, who are usually patients or their relatives, and consumers. Specifically, dissemination activities took place in two of the case studies: COX-2 and ECT.
Organizations concerned with COX-2s highlighted that they often “translate” HTA reports when writing to governmental instances or other stakeholders: “We have done that quite a bit with HTA reports, essentially taking some of the key findings or pieces and translating them into something that is usable by public and consumer groups, so that they understand what's coming out of that research” (C1). This knowledge-translation exercise is highly valued in these organizations, for which education about new drugs is a primary aim. This education is targeted not only at patients and consumers of pharmaceutical products, but also at decision makers and physicians, who are intermediaries with respect to drug users: “[. . .] I think that that's a downfall of the whole research aspect: taking it and making sure that the prescribers understand it as much as people who are using the product.” (C1)
In the case of ECT, dissemination activities were observed in an organization that used HTA findings to inform patients, their relatives and the public about specific treatment interventions, usually with the aim of “demystifying this bizarre thing” (E2). The importance of contextualizing research findings of health technologies for mental health conditions lies in avoiding a “one-size-fits-all” biomedical approach, against which other associations were protesting, as mentioned earlier. For instance, after the news broke that a gene responsible for a particular psychiatric condition had been identified, the patient association organized a special lecture given by a researcher, who, as noted by one respondent, “really relativized all of it. We thus helped bring a study to the public; at the same time we tried to put it in context and explain it in simple terms to people. And it was a very successful event” (E2). So for this organization too, education was a primary element in the mission statement.
DISCUSSION
Toward an HTA-Based Public Debate
This study showed key differences in the use of HTA reports for the four cases we examined, and they appear to be related to the very nature of the organizations involved. In the case of COX-2s and the quest for their coverage by drug plans, the commitment of these organizations to translating knowledge about drugs would appear to open up the debate to potential users of such medications—the public. It is to their advantage in the long run: the more patients and drug consumers know about the benefits and risks of new drugs, the more pressure they will exert on regulatory bodies to increase access to them. This is complemented by a resulting change in the patient–healthcare professional relationship that decreases the asymmetry of information present when prescriptions are made. This is consistent with what is reported in the social literature, that such health-related organizations are social actors that effectively engage in policy advocacy and transform assumptions about disease and prevention (Reference Brown and Zavestoski6).
Does a Report's Content Matter More Than Its Recommendations?
Another conclusion we can draw is that findings about effectiveness and harmfulness are used by patient and consumer groups, perhaps to an even greater extent than are the report's recommendations. The most striking example occurs with COX-2s. Positive findings about the efficacy of COX-2s are already available; many patients are using these drugs because they are the only drug that alleviates their symptoms while minimizing the risk of gastric side effects. As it is the only drug of its kind available in Canada, patient and consumer organizations promote better access to this drug (i.e., coverage by public drug plans) despite the potential risks of stroke associated with its long-term use. These organizations take the positive findings as evidence on which to base their position papers and letters demanding better drug plan coverage for COX-2s. Such activities were systematically undertaken by all the organizations studied. The approach described here is similar to what Barbot (Reference Barbot4) has described as a “detailed critique” that is, “putting forward a well-documented critique . . . [whose] aim is to illustrate delays or discriminatory practices and bring pressure to bear on those responsible” (p. 543).
A second example of the importance of content over recommendations relates to PSA screening. Although there are positive findings about PSA screening, namely, the test's sensitivity and specificity (17), the effect of PSA screening on mortality is not sufficient to justify population-based screening. Nevertheless, proponents of the test use the “positive” findings in their advocacy activities.
Policy Implications for HTA Agencies
In our study, the categories of knowledge use are not exclusive and different types of knowledge than those we observed may be used in other contexts. Despite these limitations, some practical lessons can be learned from our results. First, educational activities carried out by patient and consumer organizations can be considered a channel for disseminating HTA. In fact, organizations with an educational or informational aim used reports and other biomedical findings about health technologies to educate patients and the public about the use of the technology; some even provided information about potential risks. In doing so, they translated this information into “lay” language that is accessible to a wider population. HTA agencies that seek to collaborate with patient or consumer organizations during the dissemination phase may want to target organizations that share this goal.
Second, when disseminating HTA reports, agencies need to remember that even though patient and consumer organizations are not the primary target of their report, many of these organizations are in fact regular users of them. Furthermore, these groups will most probably be health advocacy groups that use the information contained in the report, not just the recommendations. They will likely use some of the reported findings about the efficacy, effectiveness, or safety during advocacy activities, or use this information to educate patients or the general public about the technology. In this regard, agencies may want to acknowledge the importance of this audience and be mindful of the use of technical language in their report and how the results could be interpreted by a “lay” audience. They could, for instance, publish stand-alone lay summaries.
Third, for potentially controversial technologies and especially for those that have known detractors, agencies could take greater advantage of deliberative processes such as inviting advocacy groups that represent patients “to the table” during the assessment process and listening to their social and ethical concerns about the technology before making final recommendations. Deliberation by citizens on tangible issues (such as ECT use) is described by Abelson et al. (Reference Abelson, Eyles and McLeod1) as a way of laying out potential policy scenarios and finding ways to obtain more convincing evidence regarding a set of recommendations. Such an approach is therefore much more likely to produce a report that produces a consensus amongst stakeholders and, in turn, increases its uptake. Also, our results suggest that technology assessments are not always compatible with advocacy activities: the degree of compatibility depends on the advocacy group's mission and assumptions about the technology. Meanwhile, HTA seems to be compatible with organizations that have a strong mission element of public education/information. These are the ones that might accept HTA recommendations and divulge them as is.
Finally, patient and consumer organizations could potentially be a precious help in push efforts (Reference Lavis20) made by agencies and aimed at healthcare providers and decision makers. In fact, in organizations whose primary purpose is to gain better access to services and technologies, conclusive research results are often the triggering factor for a collective action focused on health authorities, regulatory bodies and healthcare professionals. Effective communication of HTA reports to these patient and consumer groups is required, especially given the limited and indirect access they have to biomedical information.
CONTACT INFORMATION
Julie Fattal, MSc (HTA) (julie.fattal@umontreal.ca; juliefattal@hotmail.com), Graduate Student, International Master's Program in Health Technology Assessment and Management (Ulysses), Department of Health Administration, University of Montreal, P.O. Box 6128, Branch ‘Centre-ville’, Montreal, Quebec H3C 3J7, Canada
Pascale Lehoux, PhD (pascale.lehoux@umontreal.ca), Full Professor, Department of Health Administration, University of Montreal, Canada Research Chair on Innovations in Health, Researcher, Interdisciplinary Health Research Group (GRIS), University of Montreal, P.O. Box 6128, Branch ‘Centre-ville’, Montreal, Quebec H3C 3J7, Canada
