Hospitals use a wide range of health technologies, mainly drugs, medical devices and equipment, but also diagnostic or therapeutic procedures and organizational or support systems. Traditional decision making for acquisition or investment on these technologies was based on many issues including medical trends, institutional prestige, expert advice, professional pressure, or budget viability. Rapid technologic advance and rising healthcare costs have created the need for a comprehensive scientific and multidisciplinary approach to add rationality to these decisions, increase efficiency and minimize opportunity costs. HTA provides this broad approach, filling the “know-do gap” between research and practice and creating a bridge between the world of science and that of real decision making (Reference Batista and Hodge1). The skills of many different disciplines are needed for this task, including methods from clinical epidemiology, evidence-based medicine (EBM), biostatistics, management, quality improvement, health services research (HSR), economic evaluation, medical decision models, engineering, law, ethics, and social sciences (Reference McDonald2–Reference Goodman and Ahn4). Both administrators and professional users need valid evidence on safety, effectiveness, efficiency, and other short or long-term implications of the use of health technologies (Reference Sheldon5). This key information is synthesized in HTA reports for managers and financers to sustain strategic decisions regarding resource allocation and policy making, or in clinical practice guidelines (CPG) for healthcare professionals to aid in direct patient care and use of technology.
HTA programs started as centralized systems to assist governments make policy decisions regarding acquisition or coverage of medical technology. In Latin America, despite budget restrictions and high variability in the use of health technologies, HTA developed later than in the rest of the world: among 103 HTA institutions in twenty-four countries detected in 1996, only one belonged to a Latin-American country (Reference Infante6). In Argentina, the national coordinating HTA unit (Unidad Coordinadora de Evaluación y Ejecución de Tecnologías Sanitarias, UCEETS) was formally created at the Ministry of Health in 2009, and its members have participated ever since in regional and international collaborative networks like MERCOSUR, RedETSA, and INAHTA.
Though central macro-level HTA has thrived in Latin America during the last decade, meso-level hospital-based experiences are still scarce in our region. The largest experience is the Brazilian NATS (Núcleos de Avaliação de Tecnologias em Saúde), a network of twenty-four public university hospitals created in 2009 to promote technology assessment, practice guidelines and capacity building (7). At the time of the initiation of our program in 2001, a few American hospital-based HTA experiences were found in the literature, mainly in Canada (Reference Juzwishin, Olmstead and Menon8) and the United States (Reference Steinberg and Graziano9), but none in our region. There is some evidence that these programs have positive consequences in healthcare facilities, improving quality of care and at the same time generating significant financial savings (Reference McGregor and Brophy10).
The aim of this study is to describe the first hospital-based HTA program in a public hospital in Argentina, and to report some of its clinical, educational, economic, and organizational effects after 10 years of its implementation.
METHODS
Structure
The HTA program started in March 2001 at the Hospital de Pediatría “Juan P. Garrahan”, a teaching, public-setting, national pediatric referral center, with 500 tertiary care beds and a self-managed budget. Its main goal is to promote a rational and evidence-based technologic development and improve the use of existing technology at hospital level (Table 1). The program has four main pillars: HTA reports to aid executive decision making on acquisition of hospital technology, pediatric guideline elaboration and implementation, professional capacity building in research and management, and technical support for HSR projects.
Table 1. General Goals and Main Activities of the HTA Program
The author of this study serves as a full-time coordinator for the program; her background training includes a Master degree in Clinical Effectiveness (certified by Universidad de Buenos Aires and Institute for Clinical Effectiveness and Health Policy, IECS), and more than 20 years of continuous education and teaching of health professionals in disciplines relevant to HTA (research methodology, EBM, clinical epidemiology, and biostatistics). Given the broad approach of the program, the HTA coordinator also interacts with various relevant hospital groups and actively participates in the committees for the surveillance of drugs and other technologies (equipment and devices), quality improvement, research review, and hospital career development.
A group of twelve multidisciplinary hospital staff professionals constitute the HTA Committee and participate part-time alongside with their pediatric care tasks. Their selection criteria included technical skills, will to participate, and different professional backgrounds representing main hospital specialties. The current composition comprises eight physicians (two pediatric clinicians, two neonatologists, one intensive care specialist, one infectologist, one general surgeon, and one radiologist), two pharmacists, one biochemist, and one biomedical engineer. Their main function is to provide a connection with key patient care areas and an interdisciplinary look at complex issues (detecting problems, establishing priorities, selecting expert reviewers or optimal implementation methods).
METHODOLOGY
The HTA program produces basically two types of documents: CPGs (elaborated by the HTA coordinator or experts in the corresponding field) and HTA reports (brief technical or full reports and economic evaluations, carried out almost exclusively by the program coordinator). For HTA reports or CPGs, our evaluation cycle consists of four main phases: prioritization, evidence synthesis, communication and monitoring.
Priority for evaluation comes from two main sources: administration request for decision making regarding the acquisition of a new technology applied for by hospital staff, or utilization data of existing technologies showing unexpectedly high variability in use or rising associated costs. Professionals applying for incorporation of new technologies must fill in and submit a standardized formulary, locally adapted from the Danish mini-HTA (Reference Vestergaard, Ehlers and Kidholm11) and the Spanish GANT (Reference Briones, Loscertales, Pérez Lozano and en nombre del Grupo12), which includes vital input information for prioritization, assessment perspective and final decision making. For technologies already in use, patient and pharmacy records and other hospital databases like diagnosis-related groups serve as input for continuous utilization monitoring and analysis. These data provide useful information for priority setting and impact measurement, and also reveal common indications, main consumer services, costs, and variations over time.
Evidence analysis is based on a systematic review of the relevant literature. According to the specific problem and the urgency of the matter, this may include searching for existing HTA reports from international agencies, published guidelines, meta-analyses, utilization reports or economic evaluations, and sometimes other gray literature sources like epidemiological data or industry information. The evidence from these sources is submitted to a comprehensive analysis and adaptation to the local context, taking into consideration hospital needs and budget impact. This evidence synthesis is communicated to hospital administration either as a full HTA or brief technical report or cost-effectiveness analysis, and sometimes also disseminated in a more user-friendly CPG format targeted at pediatric professionals.
The HTA report includes a synthesis of the available evidence with minimal raw technical data to allow easy reading and comprehension by hospital administrators, a brief conclusion summarizing the status of available information and a recommendation to accept, reject or incorporate in a restricted manner. To ensure transparency, HTA reports are accessible to staff through the hospital Intranet; they are also shared with other HTA centers at a national level through a restricted-access library.
Practice guidelines are also elaborated on pediatric subjects considered relevant to our hospital patient spectrum: patient management guidelines provide a multidisciplinary approach to the diagnosis and therapy of complex pediatric diseases, and utilization guidelines convey recommendations for adequate use of health technologies. CPG elaboration methodology consists either on adaptation to the local context of published high quality guidelines (appraised by the AGREE instrument) when available, or on a systematic review of the best existing scientific evidence (meta-analysis, randomized trials, cohort studies). Draft versions are submitted to peer-review and expert consensus before dissemination of the final version to facilitate adherence. All CPGs have an executive summary of recommendations for quick consultation, and a glossary with MeSH terms linked to PubMed to enable easy searching on related subjects; disease management guidelines also include clinical pathways or algorithms to orientate patient care. Implementation strategies are usually tailored to potential users, and may involve one or more multi-mode strategies like academic or focus-group meetings, publication of a brief pharmacy bulletin or a full-text guideline with internal diffusion by means of Intranet or external dissemination through the hospital Web page (www.garrahan.gov.ar) and virtual campus to make it accessible for other pediatric care professionals.
The final phase is to assess the impact of guideline implementation or the clinical results of technology incorporation. Again, this may involve various data sources including the mentioned hospital databases or ad hoc measurements and data collection and analysis by hospital specialists or the HTA team itself. This information not only serves as a means to document the clinical or economic effects of HTA, but also to prompt the need of reinterventions, new assessments or future disinvestment.
The remaining two components of the program are aimed at multiplying the impact by incorporating more hospital professionals to the task. Capacity-building consists mainly of an annual course in research and management for pediatric professionals initiated in 2006, directed by the HTA coordinator and dictated in collaboration with IECS. Course contents cover fields related to HTA: epidemiology, biostatistics, EBM, HSR, strategic planning and healthcare programs, quality improvement, and economic evaluations. These disciplines are virtually absent during professional grade formation in our setting, and constitute fundamental tools for hospital staff to carry out patient care or management tasks in a satisfactory manner. Certification includes the design of a research or management project, and this has led to other HSR initiatives for which the HTA coordinator provides technical support, the fourth component of our HTA program.
RESULTS
HTA Reports
During the first 10 years of the HTA program (2001–2011) eighteen HTA reports were produced (Table 2). Technologies assessed covered a broad range, from drugs (6 = 33 percent) and other therapeutic (6 = 33 percent), preventive (2 = 11 percent), or diagnostic (2 = 11 percent) procedures or devices, to institutional programs (3 = 17 percent). The main focus of the assessment was effectiveness (12 = 67 percent) and/or safety (10 = 56 percent); other dimensions included cost and budget impact (6 = 33 percent), cost-effectiveness (2 = 11 percent), and organizational impact (3 = 17 percent). Six (33 percent) of the technologies assessed were already in use; in most of these cases the assessment included a utilization analysis, and the final recommendations were also disseminated to professional users in a CPG format. Among the remaining twelve new technologies, only two (11 percent) were not recommended for incorporation (optical coherence tomography and Reiki use in children), and other two (11 percent) were recommended for restricted use in specific patient subgroups (palivizumab and polymerase chain reaction, PCR, for detection of E. coli O157). Mean time from request to final report was 10.3 months, ranging from 7 days (rapid technical reports) to 39 months (full HTA reports with utilization review and final CPG for users, or institutional program assessments). Most of the reports (15 = 83 percent) were completed by the program coordinator; only three were produced in collaboration with members of the HTA committee.
Table 2. HTA Reports and Guidelines of the PETS Program (2001–2011)
CPG Implementation
In 10 years, eleven evidence-based CPGs were submitted to expert consensus and peer-review and then disseminated through various implementation modes (Table 2). CPG focus was patient management in six guidelines (55 percent) and adequate use of interventions (drugs, devices or procedures) in the remaining five (45 percent). All CPGs were published in both printed and electronic formats and disseminated through the hospital Intranet and external Web page for free full-text access by pediatric professionals. Two CPGs (albumin and immunoglobulin use) were also disseminated in a brief one-page pharmacy bulletin version for easy everyday consultation. The implementation strategy for the guidelines on albumin and infusion pumps included face-to-face meetings with the professional users, and for infusion pumps a poster with highlights of the recommendations for nurses was also distributed to all nursing stations throughout the clinical wards. In these last three CPGs, an evaluation of the clinical and economic impact of CPG implementation was undertaken at some point during the postintervention period to document before–after variation in usage, adherence to the CPG, patient effects, and financial expenses or savings attributable to the CPG. During the first year after guideline implementation, albumin consumption and associated annual costs were reduced by 50 percent (savings worth $50.000) and immunoglobulin by 10 percent ($40,000 initial annual savings, $300,000 in the following 2 years given the rise in drug price). The continuous monitoring of these measurements also served as input to prompt the need of reinterventions, guideline revision, or contact with users.
An additional measurement of both indoors and outdoors impact is the surveillance of the number of hits each CPG has on the hospital Web site: annual figures round between 1,000 and 10,000 hits per guideline (mean: 3,790 annual hits, maximum number of hits attained by the infusion pump CPG with more than 40,000 total hits in 4 years), turning this into the second most viewed section of the hospital Web site after the pediatric drug formulary.
Capacity Building
From 2006 to 2011, a total of 225 hospital professionals took the annual course on research and management, including pediatric clinicians and specialists, pharmacists, biochemists, physical and respiratory therapists, psychologists, social workers, nurses, and administrators. In 2008, the course opened to outdoor professionals working in the pediatric field to promote collaboration with other institutions. Certification was attained by more than 95 percent of the alumni; most of the remaining 5 percent finished the course but decided not to take the examination. A total of sixty-three research or management projects were produced under professor supervision and submitted for final course evaluation, covering a broad range of pediatric issues and methodological designs including clinical research, HSR, quality improvement, hospital programs and cost-effectiveness analyses. As these were group projects, most of them were multidisciplinary in nature, and a few were even inter-institutional or multicenter in collaboration with other pediatric hospitals. The members of the HTA Committee also participated in a 1-year applied course on guideline adaptation dictated by the National Academy of Medicine, during which they produced the basis of a CPG on preoperative pediatric management still under prepublication review process.
Technical Support
Besides course project supervision and active participation in the institutional research review committee, the HTA coordinator has coached twenty-two HSR projects for the implementation and impact evaluation of hospital programs or quality improvement interventions. These research ideas came mainly from professionals who had taken the research and management course, but also from staff members applying for certification of other research or specialty post-grade courses or doctoral thesis.
Organizational Effects
We observed along the years some additional qualitative benefits which could at least partially be attributed to institutional effects of the HTA program. These include: encouragement of critical reading and EBM, multidisciplinary interaction and consensus, promotion of continuous post-grade education in HTA-related disciplines, and user involvement in the decision-making process of new technology incorporation. Sustained managerial commitment and support, together with in-house professional recognition and outdoors visibility were crucial factors to allow the expansion of our HTA program.
DISCUSSION
Initial Program Context
This is to our knowledge the first and to-date only formally existing hospital-based HTA program in the public setting in Argentina. The conditions for its initiation in March 2001 were almost ideal. The author had just finished a post-grade master course in clinical effectiveness, and the hospital needed to sum all efforts to cope with an unexpected escalation of healthcare costs due to the economic crisis the country faced at that time. As a public facility caring for children most of whose families have no health coverage or insurance, and (unlike most hospitals in Argentina) a fixed budget to manage, our hospital presented the perfect setting to install a local HTA experience, and luckily our executive administrators understood so.
Barriers during Implementation
At the time our program started, EBM was creating great controversy and many experts were suspicious about it. We needed to convince all hospital stakeholders of the usefulness of evidence-based decision making. Policy makers often have urgent decisions to make and a different framework basis, including financial, political or even prestige issues. HTA must be conducted from a comprehensive perspective and in a timely manner if its findings are to be satisfactorily implemented (Reference Drummond and Weatherly13). The first assessment undertaken by this program is a good example of how HTA served administration: during the first year after guideline implementation, albumin consumption and associated annual costs were reduced by 50 percent, generating substantial savings to the institution, a valuable effect of much relevance to hospital administrators considering the critical economic situation at the time. Another breakthrough for hospital administrators was the HTA report on comparative effectiveness (failure rate) of different brands and models of cochlear implants, gaining ministry acceptance to change purchasing practices and substitute the failing model of the device (shortly after withdrawn from the market by the FDA).
HTA also needs to break down other barriers to implementation, like those between researchers and clinical practitioners. This “know-do gap,” which creates variability in practice and promotes inefficiency, requires a common language to facilitate communication between these two worlds. In our case, staff capacity-building and consensed evidence-based CPGs provide the means to fill this gap (we actually named our guidelines with the acronym GAP, which stands for “Guias de Atencion Pediatrica”). Changing prescription patterns and overcoming professional inertia is not easy. Working with opinion leaders and promoting expert consensus was a vital strategy to support recommendations and enhance guideline adherence (Reference Goodman14). Healthcare professional involvement in HTA activities and supervised guideline development also promotes future compliance and enhances cost-consciousness, helping the staff to recognize the economic implications of their prescriptions and incorporate the concepts of efficiency and cost-effectiveness. Both pharmacy bulletins on albumin and immunoglobulin contained actual cost information besides recommendations for hospital consumers, and serial utilization monitoring of these drugs allows constant reevaluation of the process. Moreover, as a teaching third-level pediatric referral hospital, we have gained a national and regional recognition status generating a quite large outdoors influence area; we believe this achievement comes with great responsibility, and so level of evidence and strength of recommendations are explicitly provided in CPGs to allow other pediatricians to make decisions adapted to their own context.
We also needed to disseminate epidemiologic skills and management tools among hospital staff. Healthcare professionals do not always receive systematic grade training in research methodology or management in our setting, so gathering those who had the required abilities and starting capacity-building of others was essential to the development of the program (15). Short EBM workshops for staff members and applied research design courses for residents and fellows were initiated at the institution under the direction of the HTA coordinator. These brief courses, though initially useful to create an evidence-based culture, proved insufficient along the years to produce a sustainable change among physicians and other healthcare professionals; the need for a more profound approach was the driving force to set off our annual course in 2006, dictated in an uninterrupted way till today. Continuous education and participation of hospital professionals in HTA activities helped maintaining the EBM culture and reinforcing the need for efficiency and cost-awareness at hospital level, an ongoing task to multiply the future impact of our HTA program and warrant its sustainability (Reference Hailey16).
Multidisciplinary interaction is also an essential factor for potential success of HTA initiatives (Reference Umscheid, Williams and Brennan17). Proper detection of priorities for assessment requires active participation of the HTA coordinator in the regular meetings of hospital committees for the surveillance of drugs, devices and equipment, and permanent interaction with the Pharmacy and Medical Technology departments who are responsible for the purchase and management of a variety of hospital health technologies. Today, the chief biomedical engineer of the technology department and two pharmacists with pharmacoeconomics skills are members of the HTA Committee; these specialists are not regular constituents of all HTA units (Reference Rogers18;Reference Martelli, Lelong, Prognon and Pineau19).
Hospital HTA
Hospital based-HTA is growing. A worldwide survey published in 2008 by the Hospital-Based HTA Sub-Interest Group of the Health Technology Assessment International (HTAi) Society identified only three (9 percent of responders) hospital-based HTA experiences located in South America (Reference Cicchetti, Marchetti, Dibidino and Corio20). The typical profile reported was that of a teaching hospital with an HTA unit or multidisciplinary committee dedicated to inform both hospital administrators and clinical practitioners on the safety, effectiveness and organizational impact of a wide range of health technologies. Our model is consistent with this general pattern, and our aims and methodology are similar to other reported hospital-based HTA experiences (Reference Catananti, Cicchetti and Marchetti21;Reference Sampietro-Colom, Morilla-Bachs, Gutierrez-Moreno and Gallo22). Our program involves some additional HTA-related activities like systematic and continuous staff capacity-building and mentoring of research projects to document the impact of health services initiatives or fill pediatric evidence gaps, both in collaboration with the research and teaching department where our program was initially located (Reference Battista, Déry and Jacob23).
Hospital-based HTA programs can be a valuable strategy to reduce inappropriate variability in utilization, control hospital costs, deal with financial restraints reasonably and minimize opportunity costs of unnecessary technology use without compromising quality of care. However, HTA impact evaluation is not an easy task, given the broad range of assessed technologies, involved stakeholders and expected effects (Reference Bodeau-Livinec, Simon, Montagnier-Petrissans, Joël and Féry-Lemonnier24). A proposed framework to assess the performance of an HTA organization includes a series of dimensions grouped in four main functions: goal attainment, production, adaptation to the environment, and maintenance of culture and values (Reference Lafortune, Farand, Mondou, Sicotte and Battista25). We have reported some quantitative results reflecting mainly goal attainment and productivity. Along these years we have also observed some qualitative organizational changes related to other less-measurable dimensions of program performance: a sustained EBM culture, the encouragement of multidisciplinary interaction and consensus, a growing number of healthcare professionals motivated to undertake post-grade courses in HTA-related disciplines, active involvement of hospital users and managers in the decision-making process regarding incorporation of new technology and utilization review of existing ones, and both internal and external visibility, credibility, and recognition.
LIMITATIONS AND FUTURE CHALLENGES
We have some limitations and yet unresolved issues. First, the HTA unit has only one full-time member (the HTA coordinator, and just since 2004) and all members of the HTA Committee are assigned only part-time to HTA activities. This lack of full-time human resources somehow hindered the potential growth of the program: as HTA reports depend on one person, they take a longer response time, and CPG production could have been larger than the actual rate of two to four documents per year. Interaction with other committees was a crucial coping strategy: most new drug assessments were made in collaboration with the Drug Committee, and some device issues were addressed by the Techno-Surveillance Committee.
Second, specific evidence for complex conditions in the pediatric age is usually scarce and there is often demand for urgent decision making in conditions of insufficient evidence or extrapolation from adult studies. Particularly for rare pediatric conditions, generating local evidence through primary clinical research is time-consuming and often impractical; so all that we are left with is evidence from small case series and expert opinion. In these situations, we often have to make decisions with great uncertainty, and usually recommend that the incorporation be reassessed after a period of specific data collection to document effectiveness in our patients.
Third, as a teaching hospital with over 400 residents and fellows and twice as many visiting trainees involved in patient care, professional turnover is constant, and maintaining CPG adherence over time is harder than usually expected. Despite a comprehensive and participative approach to HTA, there is still some level of resistance to change. Certain experts may believe that guideline recommendations can get in the way of their expertise. The old habit of basing junior professional training on “this is how we do things here” is hard to eradicate. It is also hard to get rid of the generalized myth that generic drugs or cheaper devices are not always worse quality than expensive famous brand products. Finally, senior professionals may resent being audited or receiving a rejection to incorporate a new technology.
Our future challenges include the need of further end-user involvement in the HTA process, horizon scanning to detect emerging technologies in the pediatric field minimizing the need for urgent decisions, and the design and permanent update of a hospital equipment database to facilitate preventive maintenance and obsolescence judgment and allow for strategic planning of hospital equipment acquisition.
Today, our institutional HTA program is ongoing and evolving, and has gained acknowledgement both at hospital and central levels. Our experience both feeds and nurtures through collaboration at the national HTA unit (UCEETS) and the national (RedARETS) and regional (RedETSA) HTA networks. We also joined the HTAi Hospital-Based HTA Interest Subgroup, and hope to thrive and further improve through these alliances and collaborative channels.
CONCLUSIONS
Our experience shows that an HTA program is both feasible and useful in a public hospital of a developing country. Promotion of hospital-based HTA, professional integration in HTA activities and network collaboration to discuss unresolved issues with colleagues can multiply the benefits and optimize the use of hospital budgets.
CONFLICTS OF INTEREST
The author has no conflict of interest to report.
