Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-02-06T16:02:04.599Z Has data issue: false hasContentIssue false
  • English
  • Français

Consumer involvement in the health technology assessment program

Published online by Cambridge University Press:  01 November 2004

Jane Royle  and
Sandy Oliver
Jane Royle
Affiliation:
University of Southampton The Wessex Institute for Health Research and Development
Sandy Oliver
Affiliation:
University of London Institute of Education
Rights & Permissions [Opens in a new window]

Abstract

Objectives: This study aims to describe a cycle of development leading to sustainable methods for involving consumers in the management of a program commissioning health technology assessment.

Methods: Staff time was dedicated to developing procedures for recruiting and briefing consumers to participate in prioritizing, commissioning, and reporting research. Resources and support were developed in light of early feedback from consumers and those working with them. These were piloted and amended before being used routinely.

Results: Over 4 years, procedures and resources have been developed to support six consumers attending seven to eight prioritization meetings a year; thirty to forty-five consumers each year commenting on research need for particular topics; thirty consumers a year commenting on research proposals, and twenty a year commenting on research reports. The procedures include clear job descriptions, induction and development days, clear briefing materials, payment for substantial tasks, and regularly seeking feedback to improve procedures.

Conclusions: Explicit, inclusive, and reproducible methods for supporting consumer involvement that satisfy National Health Service policy recommendations for involving consumers in research require dedicated staff time to support a cycle of organizational development.

Keywords

Consumer involvementHealth-care technologyHealth services research
Type
GENERAL ESSAYS
Information
International Journal of Technology Assessment in Health Care , Volume 20 , Issue 4 , November 2004 , pp. 493 - 497
Copyright
© 2004 Cambridge University Press

The Health Technology Assessment (HTA) program is a needs-led commissioning research program that aims “to ensure that high quality research information on costs, effectiveness, and broader impact of health technologies is produced in the most efficient way for those who use, manage, and provide care in the NHS” (11) [emphasis added]. It is similar to other HTA agencies in needing to prioritize its research program but differs from many in that it does not conduct its own HTAs but commissions research teams to do conduct them. The HTA program is cyclical with five stages (see Figure 1), beginning and ending with delivery of services.

National Health Service Research and Development Health Technology Assessment (NHS R&D HTA) program cycle.

The HTA program's prioritization strategy group and advisory panels, supported by the National Coordinating Centre for Health Technology Assessment (NCCHTA), decide which of the many suggestions received from the National Health Service (NHS) and its users should become research priorities. The program then issues calls for proposals and commissions research by open competition to answer the questions posed by those research priorities. The results of the research are then published as reports in the HTA monograph series. Commissioning and publishing involve rigorous peer review, and all projects are closely monitored while under way. Current NHS policy (1;2;10) encourages active consumer involvement in all stages of the program: identifying and prioritizing research topics, commissioning, conducting and reporting research, and getting the findings into use.

The HTA program has adopted the definition of a consumer: “as patients, carers, long-term users of services, organisations representing consumers' interests, members of the public who are the potential recipients of health promotion programs and groups asking for research because they believe they have been exposed to potentially harmful circumstances, products or services” (2). More specifically, to maximize consumer input into the existing structures of the HTA program, consumers should be linked to consumer networks so that they are able to draw on a wide body of consumer opinion. To avoid conflicts of interest when supporting the HTA program a consumer should not normally be a health practitioner, manager, or researcher.

In 1998, the HTA program conducted a short pilot study that systematically involved health-care consumers in its work (3). The pilot study found that consumers made unique contributions to the HTA program. Consumers tended to highlight issues about patients' views, social contexts, information and support needs, long-term outcomes, and dissemination of research findings to consumers (4). Their involvement exposed processes that needed further thought and development (5).

This study describes how consumer involvement has been further advanced through a cycle of development. This process is an exercise in organizational change rather than a rigorous evaluation of the impact of involving consumers. We report on the resources and methods developed over 4 years experience with dedicated staff and active involvement of all program participants.

METHODS

The HTA program invested dedicated staff time over 4 years for developing consumer involvement that included a half-time Consumer Liaison Manager, an External Adviser devoting half a day a week, and a consumer involvement steering group whose members were the NCCHTA Director, the Executive Director, the Consumer Liaison Manager and the External Adviser.

Methods for involving consumers were developed for three major tasks in the HTA program: prioritizing, commissioning and reporting research. Organizational procedures, job descriptions, and person specifications for consumers were drawn up and methods were refined in light of feedback gathered from semi-structured interviews with consumers and others working with them in the program. Newly developed resources for consumers were evaluated for their usefulness through questionnaires to consumers. In light of this feedback, we developed and amended procedures and resources (6–9). This developmental process is summarized in Figure 2.

Cycle for developing consumer involvement.

Consumers were recruited (a) as members of the advisory panels that decide which, of the many suggestions received from the NHS and its users, should become research priorities; (b) as experts to comment on the summaries of research need (vignettes); (c) as peer reviewers of research proposals; and (d) as peer reviewers of draft final reports.

The HTA program needs help from a large number of consumers each year: six to attend regular meetings prioritizing research topics; thirty to forty-five consumers to comment on research needed for particular topics, thirty or more consumers to comment on research proposals, and twenty or more consumers to peer review draft final reports. We identified consumers from directories of consumer organizations, the World Wide Web using a search engine for particular health conditions, and personal contacts. We sought organizations that had a high consumer profile (e.g., consumer led), were a national group with a local network, and had a high research profile. Within each organization, we sought an individual who matched the person specification for the task and was willing to embark on this task as defined in the job description.

Training and procedures were developed for NCCHTA staff to enable them to take on the role of identifying and contacting consumers to comment on vignettes, research proposals, and draft final reports.

RESULTS

We successfully engaged consumers from volunteer-led organizations, major charities, campaigning groups, and self-help groups. These groups included topic-specific groups (e.g., National Asthma Campaign, Diabetes UK, Alzheimer's Society, and Cystic Fibrosis Trust), generic groups (e.g., National Association for Patient Participation and Community Health Council), population-specific groups (e.g., Age Concern, National Childbirth Trust, and Action for Sick Children), and groups that deal with a range of issues (e.g., Fibromyalgia Association, Long-Term Medical Conditions Alliance).

Consumers initially found panel meetings for prioritizing research topics difficult due to the speed at which discussions took place and their unfamiliarity with the process. However, as they attended more meetings, familiarity made the meetings easier. Consumers' requests and suggestions for support prompted the development of a mentor system to provide support for new consumer panel members, revisions to the consumer panel members' job description and person specification, a help sheet for consumer panel members, and discussions with panel chairs who play a key role in determining the atmosphere of meetings and the inclusion of consumers.

Initial efforts to consult consumers about research need met two obstacles. One was the unfamiliarity of consumers with the idea of prioritizing research needs. Another was the unfamiliarity of researchers with consumer organizations and their ways of working. Subsequently, a help sheet was developed with and for consumers to explain the scope and purpose of the program and how consumers can help at this stage. Simultaneously, researchers involving consumers were given more support from the Consumer Liaison Manager. This specialized support has led to greater consumer involvement at this stage, but it precluded full and efficient integration of consumers into routine tasks.

Consumers found the task of peer reviewing research proposals technically demanding. In addition, they were not always sure of the role in providing research focused or consumer focused comments. In light of this feedback, the forms requesting peer reviewers' comments were amended to ask questions from a consumer perspective first (before questions about scientific merit and costs), and emphasize that peer reviewers are free to ignore questions they do not feel able to answer (for a copy, see the HTA Web site www.ncchta.org). To provide further support, guidelines for consumers as peer reviewers, originally developed within the Cochrane Collaboration, were adapted for the specific needs of the HTA program (6;7). These guidelines explain the role of peer reviewing and suggest how consumers may approach the task (for a copy see the HTA Web site www.ncchta.org). Members of the panel commissioning research found many of consumers' comments helpful and thought that consumers brought insights that were not highlighted by other referees.

Consumers invited to comment on draft final reports faced similar difficulties; they found the reviewing forms inadequate and irrelevant to the main thrust of their contributions. Even so, consumers have been able to highlight issues about patients' views, social contexts, information and support needs, long-term outcomes, and dissemination of research findings to consumers. They have provided both positive, reassuring comments and suggestions for improvement, particularly about how the report might be made more accessible and informative to patient choice. However, on occasions, consumers have not been able to review a report. Sometimes this finding was because no payment is available for reports under 200 pages in length. Other difficulties have included the technical nature of the reports and the topic of the report not being in line with the policy of the consumer organization. In view of the disproportionate effort required, we have abandoned consumer peer review of draft final reports. An alternative strategy is to use a science writer to support dissemination of findings.

DISCUSSION

Summary of Achievements

During the 4-year period of the development process, consumer involvement has been integrated throughout the program and has been supported by new procedures, resources and the Consumer Liaison Manager. The HTA program informed the development of NHS policy guidelines for involving consumers in research (2) and meets policy recommendations for involving consumers in prioritizing research topics through committees in terms of having two consumers per panel; supporting new consumers with induction/development days, a job description, a help sheet and the offer of a mentor; paying consumers travel, subsistence, carer costs, and a fee for attending the meetings; and continuously seeking to improve procedures.

Challenges Faced by Consumers and the HTA Program

Consumer panel members face a difficult task. The HTA program asks them to consider a wide range of topics not restricted to their main focus of interest and the business of the panel meetings is conducted at a fast pace. A better match between consumer experience and the task they face was found when consumers were invited to comment on briefings about research need. However, these vignette topics are rarely repeated, so contact between the program and the consumer tends to be one off, leaving little opportunity to learn and benefit from the experience in a subsequent request.

Consumers were not always clear as to what was required of them. When they peer reviewed research proposals, a dilemma they faced was whether to focus on a research perspective or a consumer perspective. Vignettes are conceptually more difficult in that consumers are being asked to comment on a type of document that they may not have come across previously. Further development is needed to meet each of these challenges.

The issue of payment was frequently raised by consumers. At the time, the only payments available for consumers involved with the HTA program are for panel members who are able to claim a committee fee for attending a panel meeting or induction day and consumers who peer review draft final reports that are over 200 pages. Consumers are increasingly requesting payment for commenting on vignettes and peer reviewing research proposals.

Strengths and Weaknesses of this Development Approach

Involving consumers has raised awareness of consumer perspectives for everyone in the HTA program, as staff, panel members, and peer reviewers have been exposed to high-profile requests for consumer input and to the subsequent consumer contributions. The cycle of development provided explicit and inclusive methods (see Figure 1), which drew on the multiple perspectives of consumers, HTA program participants, and NCCHTA staff, albeit on a small scale.

Strengths and Weaknesses of the Methods Developed

The main strengths of the methods developed have been clarity of aims and procedures and development of information and resources to support ongoing consumer involvement. Professionals contributing to the program also benefited from improved resources and procedures.

We have faced difficulties in quickly identifying appropriate consumers, and consumers faced the challenge of working to a tight time table throughout the prioritization process. Success is related to the degree of enthusiasm of individual consumers and staff. An ongoing program of induction and further support for staff is required, especially after staff changes while skills for involving consumers are not yet widely held among researchers, managers, and administrators. Similarly, training for fellow panel members, especially chairmen, is valuable.

Future Developments

Effort is now being redirected toward encouraging involvement of consumers as active partners in the design and conduct of commissioned research projects. The program encourages research teams wishing to involve consumers to include in their application: the aims of active involvement in this project, a description of the consumers (to be) involved, a description of the methods of involvement, and a budget for consumer involvement.

With routine procedures and resources for supporting consumer involvement throughout the program, work is currently moving from a development phase to a phase of maintenance, management, and evaluation. How consumers have been involved, at what cost, what contributions they have made, and what influence they have had is now being formally evaluated.

The authors thank consumers for giving their time to discuss their involvement in the HTA program. John Gabbay and Lynn Kerridge helpfully commented on drafts of this article. This study was funded by the NHS R&D National Coordinating Centre for Health Technology Assessment.

References

Department of Health. 2001 Research governance framework. London: Department of Health;
Hanley B, Bradburn J, Gorin S, et al. 2000 Involving consumers in research and development in the NHS: Briefing notes for researchers. Winchester: Help for Health Trust;
Oliver S. 1998 Needs and feasibility study for developing consumer involvement in the NHS R&D Health Technology Assessment Programme. Report to the National Coordinating Centre for Health Technology Assessment. Southhampton: NCCHTA;
Oliver S, Milne R, Bradburn J, et al. 2001 Investigating consumer perspectives on evaluating health technologies. Evaluation. 7: 468– 486.Google Scholar
Oliver S, Milne R, Bradburn J, et al. 2001 Involving consumers in a needs-led research programme: A pilot project. Health Expect. 4: 18– 28.Google Scholar
Oliver S. 2000 Guidelines for consumers interested in peer-reviewing research 1. Southhampton: HTA;
Oliver S. 2000 Guidelines for consumers peer-reviewing research 2. Southhampton: HTA;
Royle Jand Oliver S. 2002 Interviews with consumers to develop the prioritisation of health technology assessments. Paper presented at the 18th ISTAHC Conference, Berlin, Germany, June 9-12,
Royle J, Oliver S. 2000 How consumers peer review research proposals for the HTA programme and how commissioners respond. Paper presented at the 16th ISTAHC Conference, The Hague, Netherlands, June 18-21,Google Scholar
Royle J, Steel R, Hanley B, Bradburn J. 2001 Getting involved in research: A guide for consumers. Winchester: Consumers in NHS Research Support Unit;
Stein K, Milne R. 1998 The NHS R&D Health Technology Assessment Programme. Research and development for the NHS. In: Baker MR, Kirk S, eds. Oxford: Radcliffe Medical Press;
Figure 0

Cycle for developing consumer involvement.