Health technology assessment (HTA) is increasingly used to provide information for making decisions and formulating policies in the health sector (Reference Kubesch, Parada, Moharra, Estrada and Cortés14). Unlike evidence-based medicine (EBM), which has a very clinical approach and is based on use of the best evidence available in clinical practice, HTA is a wide-ranging process that combines the retrieval of information available in scientific literature with the analysis of other aspects of the evaluation context (epidemiological, demographic, and economic data, or information on the use of health services and technologies) (Reference Conde4).
The HTA process itself should be based on a series of sub-processes, starting with the search for information and ending with a summary of this latter. The systematic and comprehensive search for the information required to answer these questions forms the basis of a HTA report, so the strategy of searching in various information resources is a key step.
However, the biomedical information explosion brought about by the huge increase in scientific publications and bibliographic databases has created a serious problem because it makes it difficult for users to manage and compare all sources simultaneously (Reference Güemes Careaga and Gutierrez Ibarluzea8). One possible bias lies in the accessibility of the different databases and their management.
The Internet has become the richest and most important source in the search for biomedical information. Several free sources and resources can be accessed on-line (MEDLINE, LILACS), but access to certain databases (EMBASE, CINAHL) requires making a payment to the organizations that produce and/or distribute them (Reference López15). High license costs limit access to certain databases and the full texts of some key publications (Reference Araujo1).
This leads to a situation whereby HTA research agencies and units and the Health Services have incomplete, individual subscriptions to different databases, platforms, managers and search engines (Ovid, ISI Web of Science, etc.). Differences have even been detected between the subscription policies of the different agencies and units and their ability to meet these costs due to functional dependency and reduced budgets (7).
HTA agencies and units in Spain have adopted a similar line to that of other countries because their task has been made easier by the biomedical information explosion and easy on-line access to the latter.
However, a variable has come into play in Spain that has not occurred in other contexts: the early decentralization of the health system with the de facto result that, despite standardized cohesion, Spain has seventeen health systems, with their own unique features.
This has also led to the development and implementation/proliferation of new agencies and units for assessing health technologies and health services. However, their needs differ on the basis of several variables, such as available financial resources and the cultural context where they operate (7).
Most of the organizations in Spanish national health system are dependent on subscription management macro-organizations that restrict subscription independence and the ability to conduct targeted subscriptions based on needs and actual use. The pooling of resources and a common subscriptions policy would benefit all participants, as it would facilitate access to the information sources commonly used for HTA.
Although this may appear to be basic common sense, it has not been done previously, which has led to an inability to access the sources and information deemed necessary. An investigation is, therefore, required to, first, define the sources that could constitute the core to be taken into consideration when drafting a HTA report, partially described in two previous documents (Reference Bidwell and Fabricious Jensen2;Reference Gutierrez-Ibarluzea and Egües-Olazabal10) and, second, provide data on their use to determine their usefulness.
OBJECTIVES
The aim of this study was to facilitate access to the necessary resources for substantiating HTA reports. It also attempted to optimize the information resources of HTA agencies and other health services research bodies by identifying the research needs and opportunities of agencies as a criterion for allocating information resources, and finally, to generate and maintain a common repository of resources for HTA and health services research that could be freely accessed from a shared workspace platform.
METHODOLOGY
Two main lines of action were followed to achieve the primary objective of the project: identification of the needs and opportunities of information access by HTA agencies and units as the criterion for allocating resources, and the generation and maintenance of a common repository of HTA and research resources. The following actions were taken, which are explained in detail later: (a) Survey of HTA agencies and units to analyze information resources and optimization of needs; (b) Prioritization and subscription to information resources or databases; and (c) Analysis of use of subscribed resources.
Survey of HTA Agencies and Units to Analyze Information Resources and Optimization of Needs
A basic questionnaire was drawn up and distributed among national HTA agencies and units to identify their needs (see Table 1). We asked two questions: “What are the resources your agency or unit subscribes to (databases, magazines, other resources)?” and “What are the resources your organization does not subscribe to and you consider relevant?”
Table 1. Agencies Consulted and Geographical Location
For the survey design, a search was performed in the literature for possible sources used by worldwide HTA agencies (INAHTA). In addition, HTAi IRG (interest group on information resources of the international society, HTAi) was consulted as a generator of knowledge. The questionnaire had two parts: (i) The actual resources of each agency (subscriptions to journals, databases and other information sources), and finding out which of these resources were regularly accessed. (ii) Resources to which agencies did not have access but considered relevant.
The questionnaire was e-mailed to the heads of documentation of the AUnETS group (Spanish Health Technology Assessment Agencies and Units) for fulfillment. The results were updated at two different stages.
Next, information needs were analyzed to propose the resources that best covered these needs at a later stage. The survey was designed so as to reveal any differences in access to resources and subscriptions, along with different perceptions of the importance of the resource in question. This was done by including specific questions on these extremes.
Prioritization and Subscription to Information Resources or Databases
Information resources were selected from among producers and distributors, taking into account the prioritization criteria identified in the Bibliographic Search Protocols Project (Reference Güemes Careaga and Gutierrez Ibarluzea8) (see criteria in Table 2).
Table 2. Prioritization Criteria
After defining the resources to subscribe to from those prioritized, a policy of collaboration agreements was adopted with suppliers and publishers in a bid to secure competitive prices and subscription benefits tailored to the needs of HTA and the Health Services that would stretch funds and allow access to the maximum number of resources or extend subscription for as long as possible.
Analysis of Use of the Subscribed Resources
Use of the subscribed databases was analyzed to find out how they were being used. Data were collected in accordance with the practice of the information providers, which hampered analysis of the information because it was done in different ways. Nonetheless, the authors focused on three common indicators to most databases: the number of sessions begun, the number of searches performed and the documents retrieved. The results were grouped by different variables (monthly or quarterly).
RESULTS
The results of the research are described below and organized into the sections defined in the Methodology section of this study.
Survey of HTA Agencies and Units to Analyze Information Resources and Optimization of Needs
The survey was sent in two rounds, obtaining response rates of 100 percent and 90 percent, respectively. The final results of the survey are summarized in Supplementary Table 1, which can be viewed online at www.journals.cambridge.org/thc2010031.
The information obtained from the two rounds revealed that subscriptions did not vary substantially but that they were made in different periods (1 year time difference), meaning that subscription policies were stable and differed only in the subscription to full texts of certain journal packages.
The survey found similarities in access to various resources. These were freely accessible resources like MEDLINE, LILACS, or DARE, and those to which the Spanish government subscribes for the health community, such as Cochrane Library Plus or ISI Web of Science.
A different trend was observed with subscription packages to journals and other databases, because paid access to information resources is made through individual subscriptions. In this case, individual organizations subscribe to what they consider necessary for their tasks.
Respondents also mentioned that the number of subscriptions in upcoming renewals was expected to fall due to overall budget cuts.
The survey's analysis of preferences highlighted considerable differences in the selection of resources by information specialists. The survey showed that there was no common approach to determining which databases to subscribe to, obtaining mixed results.
The databases preferred by information specialists included Hayes (11 percent), EMBASE (11 percent), UpToDate (9 percent), Scopus (9 percent), CINAHL (9 percent), and ECRI (7 percent). These resources can be split into three general categories: general databases (Scopus, EMBASE, CINAHL), databases of clinical questions (UpToDate, Clinical Evidence, InfoPoems) and managers with specific packages, such as EBSCO or Ovid.
Prioritization and Subscription to Information Resources or Databases
After identifying the information needs, a list of the resources that best met these needs was proposed. Subscriptions to various information resources were suggested based on the above prioritization criteria (Table 2). The absence of a subscription to these resources by most HTA agencies and units was also considered a criterion for subscription.
The resources suggested for subscriptions included: Scopus, Ovid EBMR, Clinical Evidence, DynaMed, ECRI, and Hayes. The contents of these information resources are based on packages of journals or collections (Scopus, Ovid EBMR), answers to clinical questions (DynaMed, Clinical Evidence) and self-produced HTA reports (ECRI and Hayes). Hence, they could help create a fairly complete reference collection for health research.
Taking into consideration the types of subscription supplied by providers (personal or institutional license), we opted for a personal one with a key/password logon for ten users, as this would guarantee access to information by all information specialists of the group in any location.
Analysis of Use of the Subscribed Resources
To evaluate the subscriptions made and the possibility of making new ones, an analysis was conducted on the use of the subscribed databases from the moment access was made available up until February 2009 (the subscriptions started in April 2008 for Scopus; in June for Ovid EBMR, Clinical Evidence, Dynamed and ECRI; and in September 2008 for Hayes).
The aim was to establish a common protocol of needs based on the use of the resources. The indicators taken into account in this evaluation were the number of sessions begun, the number of searches made, and the documents retrieved. The results were grouped by different periods (total, monthly, and quarterly).
The results of total use of the subscribed databases are shown in Table 3. The results are also grouped by type of resource in Table 4.
Table 3. Results of Use of the Subscribed Databases
Table 4. Results of Use by Type of Resource
The statistics on the use of the information resources reveal that their use is limited (e.g., Ovid EBMR, with 23 sessions, 121 searches, and 112 retrieved documents, or DynaMed, with 39, 128, and 67, respectively). Use was mainly concentrated in several well-known databases (Scopus, with 437 sessions, 2,458 searches, and 1,620 retrieved documents, or Hayes, with 107, 250, and 2,603, respectively, taking into account that subscription to this source was made at intervals of several months).
The data extracted from the monthly and quarterly variables confirmed the totals. Scopus was the most used database over the months, although the possible reason for this was that two bibliometric projects were being carried out simultaneously (Reference Parada Martínez, Millaret Senpau, Parada Martínez and Pons Ràfols17;Reference Sobrido Prieto, González Guitián and Cerdá Mota20). On the contrary, Hayes was the resource that obtained more text accesses.
DISCUSSION
Because HTA agencies supply information for making decisions on time and in the context where these must be made, some information/evidence will clearly be incorporated from different reference contexts. As a result, the definition of resources that must inevitably be taken into account cannot be absolute.
Different HTA organizations have undertaken methodological approaches (Reference Conde, Imaz, González and Alcaide3;Reference Sampson, McGowan, Lefebvre, Moher and Grimshaw19) at different stages in the process, including searches for information (Reference Douw, Vondeling, Eskildsen and Simpson5;Reference Ibargoyen-Roteta, Gutierrez-Ibarluzea, Asua, Benguria-Arrate and Galnares-Cordero13). However, there is no general consensus regarding which sources are needed for drafting a HTA document and which could be the most efficient in each case.
The increasing variety of available resources, platforms, systems, interfaces, etc., the disparity between different systems of access, and the different query languages depending on where the search is carried out, makes it difficult to locate and retrieve relevant information. To resolve this problem, we would need a unique system of information access, allowing information to be located from a single interface and within a single query session (Reference Ponsati18).
Moreover, the increase in the volume of information available, the diversification of sources, constant changes and the increasingly complex information needs and demands make studies on information consumption essential as a tool for good resource management (Reference Suárez Balseiro, Sanz Casado, Vergara González and Sotolongo Aguilar21). This data collection is an important task for the managers of subscriptions as it provides information on how they are being used and may help minimize the costs of licensing for access to resources (Reference Hyland and Wright12).
Information specialists of AUnETS (Spanish Health Technology Assessment Agencies and Units) worked on a document to formalize the initial information needs and the sources that could be used to meet this needs. This work also involved information specialists of the international HTA society interest group on information resources (HTAi-IRG). The results of this research were the subject of two works (Reference Güemes Careaga and Gutierrez Ibarluzea8;Reference Güemes-Careaga and Gutiérrez-Ibarluzea9), which, together with other publications in the field (7), determined the need for reflection on HTA agency subscriptions in the context of Spanish national health system and for an attempt to extrapolate this situation to that of international HTA agencies.
LIMITATIONS OF THIS STUDY
One limitation to this research concerned the management of resources in different environments. The possibilities offered by new information and communication technologies have not led to increased facilities for the interaction of resources or subscription purchases. In fact, geographical distance has had a negative impact on the understanding of providers of information sources and resources. This lack of understanding has led to extended deadlines, difficult subscription formulas and, in some ways, increased the cost of the process in terms of time and staff occupation.
HTA organizations also face the problem that the mechanisms used by publishers for access to their resources are very diverse and sometimes technologically complex (validation by username and password, URL, IP, or a combination of these).
All the added difficulties have meant that the processing of the use of the subscribed databases and information resources refers to a shorter period than the authors would have liked for establishing conclusions on their use. Hence, their use will be monitored over a longer period than scheduled and, thus, the final conclusions of this section should be taken with caution.
Last, to understand some of the results, we must take into account the fact that two projects were being carried out in the same field at the same time (Reference Parada Martínez, Millaret Senpau, Parada Martínez and Pons Ràfols17;Reference Sobrido Prieto, González Guitián and Cerdá Mota20). The concurrence of these projects may have influenced database use at certain times. Thus, a continuous follow-up of their use since the end of these projects would be necessary to confirm the results and come to conclusions in the light of these.
Spanish agencies are quite homogeneous in their mission and visions, they are mainly related to regional governments (only one exception AETS-ISCIII) and their recommendations are directed to regional decision makers. Five of seven agencies have established horizon scanning experiences and the health technologies covered are also similar. In this sense, our results confirm that even in similar environments or contexts that do not warrant diverse subscription policies, there is a heterogeneity in subscriptions that certainly—and in the light of subsequent analysis—does not meet real needs. Although these differences cannot be easily overcome, and because there is a policy of exchange, coordination and joint action between the latter that stems from the philosophy of HTA and is set out by the organizations themselves, one of the common denominators that we need to promote is that of ensuring access to the information resources deemed necessary by HTA organizations.
The adoption of a common subscriptions policy facilitates the constitution and maintenance of access to information resources suited to the needs of such users. The aim, therefore, is to meet the need for sharing documentary resources (negotiating better contracts, securing access to documentation packages that would be impossible individually, etc.) (Reference Suárez Balseiro, Sanz Casado, Vergara González and Sotolongo Aguilar21). This study should be extendable to other contexts and be concurrent with the activities of the HTAi interest group and its developments (6;11). In this regard, the creation of a common repository accessible by all professionals of these organizations would be very beneficial as it would grant them access to actions in course and methodological support tools, reports assessing health technologies, emerging technologies, consensus documents and clinical practice guidelines. In addition, a platform would facilitate access to the sources of information that shape the map of necessary resources, together with links to national and international assessment networks and related projects (16).
CONCLUSIONS
This study provides data on the information managed by Spanish HTA agencies and units. The information presented may help to establish a management system common to all organizations.
There are differences in the way that HTA agencies and units access the different sources and resources of biomedical information. Combined subscription to several resources for documentation services was suggested as a way of resolving these differences. Despite the above, subscribed resources are scarce, which raises the need to make more information resources available to information specialists to improve healthcare policies and decision making at national, regional, and local level.
The authors also consider it necessary to establish a system between the diverse agencies and units for sharing resources and developing collective catalogues that will facilitate access to reference materials (exchange of information and experiences). This would help develop better coordination in the HTA community.
After analyzing the data, three general ideas were obtained: (i) The least used databases were those related to clinical questions; (ii) In general, database use is fairly limited, with the exception of the more specialized resources, in which most searches are made; and (iii) Resources are not used continuously and appear to be subject to peaks of activity when documents are being drawn up.
As a final point, data collection and the analysis of statistics on the use of databases, platforms, etc. are now of vital importance, because they show how people use information resources. Knowing how information professionals use them is of great value when it comes to decision making. From the data obtained, policies could be adopted whereby some resources are cancelled and organizations subscribing only to what they actually use, thereby gaining efficiency from the outset.
CONTACT INFORMATION
Lorea Galnares-Cordero, MJ (l-galnares@ej-gv.es), Health Technology Assessment Researcher, Osteba-Basque Office for HTA, Department of Health and Consumer Affairs – Basque Country, Olagibel, 38, 01004 Vitoria-Gasteiz (Basque Country), Spain
Iñaki Gutiérrez-Ibarluzea, MS, PhD (osteba7-san@ej-gv.es), Associate Professor, Department of Biochemistry, Nursing University School Vitoria-Gasteiz, José Atxotegi z.g, 01009 Vitoria-Gasteiz (Basque Country); Health Technology Assessment Technician, Osteba-Basque Office for HTA, Department of Health and Consumer Affairs, Donostia-San Sebastian, 1, 01010 Vitoria-Gasteiz (Basque Country), Spain
CONFLICT OF INTEREST
The authors' institute received funding for this work from the Spanish Quality Agency of the Spanish Ministry of Health and Social Policy.
