Health systems are placing more emphasis on designing services that are “patient-focused” and encouraging active patient and citizen involvement in decisions about health service design and delivery, as evidenced in the following documents: A foundation for Alberta's health system (http://www.health.alberta.ca/documents/MACH-Final-Report-2010-01-20.pdf); Rebalancing evidence-based healthcare: the central role of patients and consumers (USA) (www.evidencebasedhealthcare.org); Scottish Patients’ Rights Bill (www.scottish.parliament.uk/s3/bills/42-PatientRights/index.htm).
Health technology assessment (HTA) is a multidisciplinary form of policy analysis that seeks to determine the intended and unintended consequences of using a health technology (that is any form of health intervention: health promotion activities, vaccines, medicals devices, medicines, assistive technologies, rehabilitation, etc). In many countries and regions, HTA is still driven by assessment of quantitative evidence from controlled studies or economic modeling, which provides an explicit, transparent assessment of the clinical and cost effectiveness of a health technology. The broader social, psychological and ethical aspects related to the use of a health technology are often only considered implicitly by the researchers and decision makers who develop and use HTA.
The issue of ethics and HTA has been examined in several articles and these are useful in determining an approach to ethical issues (Reference Lampe and Mäkelä15). However, surprisingly few HTA agencies use and invest in scientifically robust methods to gather evidence about the social and psychological aspects of living with an illness or using a technology (Reference Kristensen and Sigmund13;Reference Lee and Sinding16). A key source for such evidence is patients, who can give their perspectives about: experiences; attitudes; beliefs; values; and expectations about health, illness and health technologies.
Such information may be considered by many HTA researchers as subjective and potentially biased, but patients’ perspectives can be generated in a systematic manner, and quality and validity can be evaluated so that it can inform HTA recommendations.
In addition to evidence on patients’ perspectives, there is a need for effective engagement of patients in the entire HTA/appraisal process, from scoping to define the context and research questions, through to evaluation of the evidence about the technology and communication of findings. The HTAi Interest Group on Patient/Citizen Involvement in HTA seeks to encourage and share best practice in engaging with patients and citizens throughout the HTA process and to promote methods of obtaining robust evidence for assessment of patients’ perspectives. This study seeks to dispel myths about the poor quality of evidence associated with patients’ perspectives and to assert that patient participation in HTAs should be integral to the assessment process.
In this study, we use the term “patient” as a broad definition meaning any current or potential health service user or beneficiary of a health technology. We consider a patient to be a person who has valuable experiential knowledge about a specific illness or condition (such as pregnancy) or health technology, who can provide views about living with the illness/condition or the intended and unintended consequences of a technology. We do not include the “public” or “citizens” who are members of society who have an interest in the efficiency and fairness of the health system and can provide helpful comments on social aspects of HTA, but who may not have personal experience of a specific technology or disease.
In the following sections, we outline key concepts related to two distinct but complementary ways in which HTAs could be strengthened by taking account of patients’ perspectives. First, we suggest a systematic approach to include robust evidence on patients’ perspectives and in the second part, we outline different approaches that can be used to support patient participation in the HTA process.
SCIENTIFIC PROCESSES TO GATHER EVIDENCE ON PATIENTS' PERSPECTIVES
Whatever aspect of HTA is being assessed, it must be strongly rooted in research and scientific method to provide robust results that can inform decision making.
Similar to the other aspects of HTA, the assessment of patients' perspectives should be an interdisciplinary process involving all those conducting the HTA in decisions about the evidence to be gathered, the form of methods to be used and how the results impact on the HTA conclusions. A professional experienced in social and/or humanistic sciences should ensure rigorous methodology, analysis, and reporting to produce evidence that can undergo critical appraisal of quality, as would be done by an economist in the assessment of cost effectiveness.
The first phase of gathering evidence on patients’ perspectives is to conduct a systematic review for existing secondary and primary studies. This is done to identify research questions of relevance to the particular HTA and find perspectives’ about issues already identified in scoping. If evidence is not available for the specific disease/technology in question, or not transferable to the context considered in the HTA (e.g., due to different values or culture) or is of low quality, generation of primary research data are recommended as a second phase.
Systematic Review
Evidence on patients’ perspectives can be derived from both quantitative and qualitative approaches, but this subsection focuses on the review and appraisal of qualitative research, as most HTA researchers are familiar with systematic review techniques for quantitative evidence.
A systematic review to find qualitative evidence begins with a literature search, which identifies papers that might generate or answer research questions. Papers must be systematically reviewed according to inclusion criteria, assessed for relevance and summarized. This sounds the same as the process that would be used for a systematic review of clinical effectiveness, but valuable evidence about patients’ perspectives may arise from a variety of forms of studies in the social and humanistic paradigm, including qualitative studies (e.g., anthropological/sociological/nursing studies) and qualitative evidence embedded within quantitative studies. Furthermore, there is less standardization in the presentation of qualitative evidence, so this makes their identification, review, and summarization more challenging.
Literature Searching for Identification of Qualitative Studies Including Patients’ Perspectives. As there are no databases that include only qualitative evidence, careful searching of a range of publication databases is required to identify potentially relevant papers. The principal literature databases include the standard ones of MEDLINE, PubMed, and EMBASE with the addition of PsycINFO, CINAHL, Sociological Abstracts, and Social Sciences Citation index. Some of these provide search filters to help identify qualitative research.
Abstracts related to qualitative evidence may not follow a clear structure and the titles of studies are often creative, so this can make abstract selection difficult (Reference Dixon-Woods, Bonas and Booth5). Consequently the literature search for qualitative evidence should be wide ranging and follow up on footnotes and reference lists, hand-search relevant journals and “gray literature,” and search by author name (Reference Barroso, Gollop and Sandelowski3). Evidence may also be found from Web sites of national patient organizations.
The search of journals will often include some related to the disease/condition or form of technology under investigation and other general sources of qualitative research, such as The Patient; Health Expectations; Value in Health; Medical Anthropology Quarterly; Social Science and Medicine; Culture, Medicine, and Psychiatry; Anthropology and Medicine; and Sociology of Health and Illness.
Assessment of Qualitative Research. Similar to other systematic reviews, the quality of each study should be assessed using prespecified checklists (Reference Kuper, Reeves and Levinson14;Reference Malterud18) or by criteria specifically developed by the researcher (Reference Nicholas, Globerman, Antle, McNeill and Lach19). Quality assessments should evaluate the following: purpose of the study and relevance to study question; context (population/setting/values comparable?); appropriateness of methods; transparency of data generation, analysis, and interpretation (avoidance of bias); connection between research question and conclusions (internal consistency); and the account of the knowledge generated given the methods (relevance for practice).
Synthesis of Qualitative Studies. The purpose of a synthesis of qualitative studies is quite different to that of clinical effectiveness, which may use meta-analysis to determine an estimate of the effect of a technology and associated uncertainty; seeking to hone in on the true effect. For qualitative studies, the purpose is more exploratory and so methods such as meta-ethnography (Reference Noblitt and Hare20) and narrative analyses (Reference Popay, Roberts and Sowden21) are used. Meta-ethnography seeks to interpret studies rather than merging or generalizing them—looking for a new theory or “line of argument” to explain all the studies, while narrative analyses seek to go beyond a summary of research findings to generate new insights. As these analyses are specialized, it is important that they are performed by an experienced researcher and that he/she is fully involved in the conclusions that are drawn from this work and the implications for the rest of the HTA.
Primary Research Methods
Qualitative or/and quantitative methods can be used to generate new evidence and the form of study used will depend on the research questions, purpose of analysis, and the available resources. Whatever form of primary research is used, it is important to determine whether ethical approval is required for the research and ensure that participants in the study give fully informed consent, if required.
Qualitative Methods. Primary studies using qualitative methods to understand patients’ perspectives are most relevant when the goal is to get in-depth knowledge about the value and impact of a specific technology on the life of patients; how customs, attitudes, and traditions influence patients’ preferences; and patients’ visions and requirements concerning a technology, and its economic and organizational aspects.
The most commonly used qualitative methods for generating evidence to determine patients’ perspectives are individual in-depth and focus group interviews.
Participant-observation can also be useful, as what people say they do, and what people actually do, can be in contradiction. It is also useful for gaining an understanding of the physical, social, cultural, and economic contexts in which patients live or are receiving care. For this reason, data generated through participant observation in a real life setting (field work) can complement the subjective information reported by participants.
Exploring patient issues in an HTA requires the researcher to recognize that knowledge always arises from certain methodological, theoretical, and analytical positions (Reference Malterud18). For example, this means that “open” questions are not open. Patients are asked a question from a particular position. Even if a researcher begins by saying, “Will you tell me something about your experiences with . . .”, a choice of what to include and what to omit is made. The researcher wants to hear about certain specific experiences and not all possible experiences. Consequently qualitative data must always be analyzed and interpreted using theory based methods so that the results can be appropriately generalized.
In addition to traditional methods of research, new opportunities for qualitative research are emerging with the advent of social networking, and interesting work has been done to identify community views from Weblogs (Reference Street, Braunack-Mayer and Facey23). Use of such material requires the same considerations of quality and generalizability as more formal research.
Quantitative Methods. Primary studies involving quantitative methods are most commonly used if there is limited time for input and research questions are clear, when evidence on patients’ perspectives has been found but there is a need to test findings in the specific/national regional context, to input to a cost utility analysis, and to address issues of generalizability and support triangulation of evidence.
Quantitative data on patients’ perspectives can be generated from survey questionnaires administrated to a sample of patients from the target population. Questionnaires should be as short as possible with clear and precise questions, including relevant responses patients could give. The use of validated questionnaires will improve the robustness of results.
Several internationally validated generic instruments exist for the measurement of health status in any condition (e.g., EQ-5D, SF-36) and for some diseases (such as arthritis) specific instruments to measure quality of life have been developed and validated. New bespoke questionnaires to measure health status can be developed, but these need to carefully developed and fully validated, involving patients in the development and piloting to ensure it is understandable.
It is important to have a comprehensive, complete, and accurate listing of the target population (Reference Bailey2) to determine the best sampling method. The collection of data in the clinical setting has to be well organized according to the chosen form of survey administration, providing letters to present the research and a deadline for feedback. Incentives, short questionnaires, and sponsoring by credible organizations can help to improve response rates (Reference Edwards, Roberts and Clarke6).
Evidence Submissions. Patient organizations often collect information about the reasons why patients or carers call them, or conduct surveys of their members about living with their illness. Many patient groups do not put this information into the public domain, but they may be willing to share it with researchers. Hence, it can be valuable to establish a process for requesting submissions of evidence from patient organizations to answer specific questions using qualitative or quantitative information. For example, the Scottish Medicines Consortium seeks evidence from patient organizations on overview of organization; number of patients affected; experience with currently available therapy (perceived advantages and disadvantages; preferences and needs - met and unmet); information to explain how the health problem affects patients/carers; and potential impact of new technology (how it matches up to users’ needs and preferences, advantages/disadvantages over current therapy, impact on lives of patients and carers). Similarly to all other contributors, patient organizations should be asked to provide evidence sources and declarations of interest.
ENGAGEMENT OF PATIENTS IN THE HTA PROCESS
Just as a health technology is shaped by people as it is developed, HTAs are shaped by the people who participate in them. HTAs involve value judgments and decisions about a variety of issues throughout the process, including which topics should be assessed, which research questions should be answered, what evidence should be included, what findings are important and whether and how these findings should inform recommendations.
HTA organizations vary as to who is involved in making these decisions, and how they participate. Typically, experts such as health professionals and researchers provide important insights from a wide variety of disciplines. Involving these stakeholders may help contextualize technologies and ensure that they can be used in a health service. However, if the aim of HTA is ultimately to improve patient outcomes, a critical and often ignored stakeholder group are patients; the group most directly affected by the health technology (11).
If we have moved to an era where patients work in partnership with their health professionals, rather than as the passive recipients of healthcare, it is reasonable that they participate in the HTA process. A survey of the International Network of Agencies for HTA showed there is increasing interest in involving patients and patient organizations in the deliberative processes of an HTA, but that their involvement is not systematic nor widespread (Reference Hailey and Nordwall10).
Expectations for Patient Involvement
There are two principal reasons why patients should be included in HTA, and indeed why the public in general is increasingly being involved in policy making in many countries, notably in Europe, North America, and Australia.
The first reason is because patients have unique knowledge that can enlighten and inform an HTA (11). The experience of living with a condition and/or using health services can provide an additional perspective to that of other experts. Just as clinicians can provide insights into the real world context in which technologies are used, patients can provide a real world understanding of the illness/condition, and the benefits and disbenefits of using particular technologies. This can illuminate why some technologies that appear effective in clinical trials prove not to be so in real life (for example, uptake in screening or adherence to a treatment regimen). Moreover patients have a unique insight into health care as they are the only actor that participates in all aspects of the pathway of care. This experiential knowledge can improve the judgments and decisions made within an HTA.
The second reason is concerned with improving transparency and openness in public policy. Democratic arguments would support the idea that people directly affected by policies, such as patients and carers, be involved alongside other members of society in shaping public policies. Beyond their contribution to knowledge, patients, like other members of society involved in HTA (researchers, policy makers, health professionals) bring their own perspective on what is, and is not, valuable in technologies. Therefore, their input may not only enrich the expertise, but also help to make more explicit the normative assumptions that support HTA recommendations (Reference Lehoux, Daudelin, Demers-Payette and Boivin17). As HTA is intended to be a form of policy analysis, being open to more effective patient participation in the HTA process may be important to improve the social legitimacy and implementation of HTA recommendations.
Patient Engagement Mechanisms
The way in which patients are involved in HTA varies considerably between organizations and countries (Reference Hailey and Nordwall10). This is partially a result of the impact of local norms and values, but also due to the absence of an agreed understanding of what is meant by patient engagement and how it should be managed.
Rowe and Frewer (Reference Rowe and Frewer22) offer a simple yet comprehensive typology of public engagement mechanisms, based on the flow of information, which distinguishes between communication, consultation, and participation. Such typology has recently been adapted to the context of HTA (Reference Gauvin8).
Communication refers to engagement methods in which information flows from the HTA organization to patients. An HTA publishing a plain language guide to an HTA report is an example of communication. Consultation occurs when information flows from the patients to the HTA organization. Consultation with patient organizations and individual patients might focus on the choice of topic for HTA, current experiences and potential impacts of the technology, or feedback on a draft report. In communication and consultation, the information flow is one way.
In the third type, participation, the flow of information is two way. Participation occurs when information is exchanged between the organization and patients. This process of dialogue and negotiation can reduce the risk of misinterpretation and change opinions. As such, participation mechanisms can be incorporated into existing deliberative processes used in HTA. There are a wide range of established techniques for participation (eg. deliberative dialogues, consensus conferences, nominal group techniques, Delphi methods, citizens’ juries) and the technique undertaken should be governed by the intended level of participation in decision making.
A common concern among patients is that their involvement can be tokenistic, but true participation cannot be merely reduced to “having a patient sit at the table” (11). The quality of the deliberative process relies on participants’ ability to contribute competently, and on the establishment of “fair deliberation” procedures (Reference Abelson, Forest, Eyles, Smith, Martin and Gauvin1;Reference Webler, Renn, Webler and Wiedemann25). Early involvement, training (e.g., about the process, technical language used in HTA, and topic under discussion), the choice of an appropriate participation method and support from HTA organizations, which provides impartial moderation that facilitates mutual respect and opportunity for participation between participants, may help to ensure that patients contribute meaningfully to the HTA process and output.
DISCUSSION
In recent years more has been done to focus HTA on patients’ needs, with increasing discussion of patient reported outcomes and use of quality of life measures to inform economic evaluation. We argue that this is not enough and explicit collection of evidence to determine patients’ preferences about their care and use of health technologies is necessary to inform the important value judgments that are inherent in the HTA process.
Bridges (Reference Bridges and Jones4) suggests HTAs should focus on patients’ problems, take patients’ perspectives, and accommodate patients’ preferences, while allowing patient participation in the HTA process. This empowers the patient and develops a sense of ownership in the evaluation and decision-making process. This would seem essential as the health systems that HTA is seeking to inform strive to ensure that decisions are made in partnership between clinician and patient. Indeed, without sufficient patient involvement there is a risk of HTA findings being rejected as a result of patient/political pressures.
So we argue that HTA should go beyond the consideration of clinical and cost-effectiveness, to assess robust evidence about the perspectives of the ultimate user/receptor of the technology that can enlighten the social and psychological issues related to the real-world use of health technology. To achieve this, we recommended that every HTA contains a specific section about patients’ perspectives to create a truly patient-focused HTA.
In addition to collection of robust evidence about patients’ perspectives, we believe that HTA must enable patients to participate meaningfully in the value judgments and decisions that are fundamental to its policy analysis process. As part of this, transparency about the influence of the patients’ perspectives in the deliberative process is essential. Otherwise participation and evidence collection may be seen as tokenistic
Guidance exists on provision of robust scientific evidence about patient related aspects in HTA (Reference Kristensen and Sigmund13), but as with other aspects of HTA there is always room for methodological advancement (particularly in the synthesis of qualitative evidence). We also recommend that a professional experienced in social/humanistic research should be responsible for gathering evidence about patients’ perspectives and its presentation and interpretation in the HTA. However, Hailey (Reference Hailey9) noted that HTAs have limited access to expertise in the social sciences and so we recommend greater collaboration between the HTA community and researchers.
Hailey (Reference Hailey9) also found that time and cost were barriers to “consumer” involvement in HTA. However, the time and cost of performing assessments of clinical or cost-effectiveness are rarely questioned. Often decision-making bodies are happy to commit substantial resources to ensuring that these elements are robustly assessed, but they are not willing to fund research to robustly determine patients’ perspectives. We think this needs to be challenged.
In terms of patient participation in the HTA process, guidance exists for patient organizations to engage in an HTA process that allows evidence submission (11), but there is no clear guidance for HTA agencies. The HTAi Interest Group on Patient/Citizen Involvement in HTA will continue to provide material to support this process and has produced an extensive glossary of HTA terms for patients, which will be developed with feedback from users. Important work is likely to emerge from a new study that aims to develop a framework to support patient participation in HTA activities in hospitals (Reference Gagnon, Lepage-Savary and Gagnon7). The HTAi Interest Group will work with others in the evidence-based medicine fields to develop guidance to support effective patient participation in HTA. This will take account of barriers to consumer involvement in HTA (Reference Hailey9) and learn from principles for effective patient/public participation developed by public policy and health research organizations (Reference Hewlett, DeWit and Richards12;Reference Telford, Boote and Cooper24, International Association of Public Participation).
CONTACT INFORMATION
Karen Facey, Hon MFPH, CStat, PhD, BSc (Hons) (k.facey@btinternet.com), Chair, HTAi Interest Group on Patient/Citizen Involvement in HTA, Honorary Senior Research Fellow, Department of Public Health and Health Policy, University of Glasgow, 1 Lilybank Gardens, Glasgow G12 8RZ, United Kingdom
Antoine Boivin, MD, PhD (c) (antoine.boivin@gmail.com), Doctoral candidate, Scientific Institute for Quality of Healthcare, Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands
Javier Gracia, MD, MPH, MSc (javier.gracia@salud.madrid.org), Researcher, Health Technology Assessment Unit (UETS), Agencia Laín Entralgo, 27 Gran Vía, Madris 28013, Spain
Helle Ploug Hansen, Mag.scient, PhD (hphansen@health.sdu.dk), Professor, Institute of Public Health, University of Southern Denmark, J.B.Winsløwsvej 9b, 5000 Odense Denmark
Alessandra Lo Scalzo, Mphil, MSc (loscalzo@agenas.it), Researcher, Innovation and Development – HTA Unit, Agenzia Nazionale per I Servizi Sanitari Regionali – AGENAS, Via Puglie 23, Roma 00187, Italy
Jean Mossman, BSc (jean.mossman@bfinternet.com), Member of the HTAi Interest Group, HTAi, 1200, 10405 Jarsper Avenue, Edmonton, Alberta, Canada T5J 3N4
Ann Single, BSc (singlehaworth@gmail.com), Member of the HTAi Interest Group, HTAi, 1200, 10405 Jarsper Avenue, Edmonton, Alberta, Canada T5J 3N4
CONFLICT OF INTEREST
K Facey has received funding as an employee of the MHRA (UK) at the Committee on Safety of Devices, as a Member of the Board at NHS Forth Valley (UK), and for Being Chair of Policy Forum in 2009–2010 at HTAi. She has received consultancy fees or travel funding from the National Institutes of Health Research, the EUnetHTA network, Guidelines International Network, HTA agencies, patient organisations, international pharmaceutical companies, and governmental departments. She is Chair of HTAi Interest Group on Patient/Citizen Involvement in HTA and her husband is a general practitioner in the UK.
A Boivin has received funding from Fonds de recherche en santé du Québec.
J Mossman has received consultancy fees from the Bone Cancer Research Trust and Myeloma UK, and patient organisations: She has received grants, travel funding, or honoraria from several pharmacological companies. She is policy lead for European Federation of Neurological Associations which has received grant funding from several pharmaceutical companies.
The other authors report having no potential conflicts of interest.
