Hostname: page-component-7b9c58cd5d-6tpvb Total loading time: 0 Render date: 2025-03-15T20:20:21.061Z Has data issue: false hasContentIssue false

USING THE HEALTH TECHNOLOGY ASSESSMENT TOOLBOX TO FACILITATE PROCUREMENT: THE CASE OF SMART PUMPS IN A CANADIAN HOSPITAL

Published online by Cambridge University Press:  05 June 2017

Thomas G. Poder*
Affiliation:
Centre de Recherche du CHUS and UETMIS, CIUSSS de l'Estrie - CHUS Département d'Économique, École de Gestion, Université de Sherbrooketpoder.chus@ssss.gouv.qc.ca
Rights & Permissions [Opens in a new window]

Abstract

Objectives: The aim of this study was to present the experience of a Canadian hospital-based health technology assessment (HTA) unit that performed the traditional functions of the HTA process along with many other activities to facilitate the choice of smart pumps.

Methods: A rapid literature review was initiated, but little evidence was found. Moreover, the evidence provided was too far from our hospital context. To help our decision makers, we offered them a list of various services based on the skills of our HTA unit staff.

Results: To involve our HTA unit in the choice of the new smart pumps led to a strong collaboration between hospital services. After a rapid review on smart pumps, we proceeded to establish the clinical needs, followed by an evaluation of technical features. To ascertain clinical needs, we participated in the establishment of a conformity list for the tender, a failure and mode-effect analysis, an audit on the use of actual smart pumps, and simulation exercises with nurses and doctors to evaluate the ease of use and ergonomics. With regard to technical tests, these were mainly conducted to identify potential dysfunction and to assess the efficiency of the pump. This experience with smart pumps was useful for evidence-based procurement and led to the formulation of a nine-step process to guide future work.

Conclusions: HTA units and agencies are faced with rapid development of new technologies that may not be supported by sufficient amount of pertinent published evidence. Under these circumstances, approaches other than evidence-based selection might provide useful information. Because these activities may be different from those related to classic HTA, this widens the scope of what can be done in HTA to support decision making.

Type
Assessments
Copyright
Copyright © Cambridge University Press 2017 

The development of health technology is crucial to promote patient safety and a better organization of care. During the past 4 decades, technological innovation has yielded remarkable advances in health care. However, these advances have also contributed to burgeoning healthcare costs (Reference Bodenheimer1). Considering the current context of scarce resources, ensuring healthcare quality and safety for all has become a major challenge for healthcare systems around the world. To cope with this huge task and to support managers and clinicians when taking decisions about health policy, technology acquisition as well as the organization and management of services and clinical practices, health technology assessment (HTA) has been developed.

HTA has been defined by Henshall et al. (Reference Henshall, Oortwijn, Stevens, Granados and Banta2) as “a form of policy research that systematically examines the short- and long-term consequences, in terms of health and resource use, of the application of a health technology, a set of related technologies or a technology-related issue.” Because this definition may have been considered too extensive, a more stringent one was adopted by the International Network of Agencies for Health Technology Assessment: “HTA is the systematic evaluation of the properties and effects of a health technology, addressing the direct and intended effects of this technology, as well as its indirect and unintended consequences, and aimed mainly at informing decision making regarding health technologies” (3).

By its nature, HTA is thus a multidisciplinary field involving many specialists who gather high-quality scientific information on clinical effectiveness, cost-effectiveness, and social, ethical, and organizational impacts of technologies and interventions used in the areas of health. These multidisciplinary groups use explicit analytical frameworks, drawing from a variety of methods, to meet the needs of decision makers (Reference Lampe, Makela and Garrido4). Most of these methodological developments have been made at a national level, for HTA agencies. However, we also need to consider the specific needs in a hospital setting, because decisions on investment and implementation are frequently made at local hospitals. In this setting, a recently established European initiative led to adoption of the AdHopHTA handbook, a handbook of hospital-based HTA (Reference Sampietro-Colom, Lach and Pasternack5;Reference Sampietro-Colom, Lach and Cicchetti6).

All these developments led to new products and services different from those related to the production of systematic literature reviews combined with a study of the context, which still remains the core business of HTA (Reference Merlin, Tamblyn and Ellery7). The mini-HTA is a well-known product developed by the Danish Centre for Evaluation and Health Technology Assessment (Reference Ehlers, Vestergaard and Kidholm8). This product was mainly developed to reduce time production. However, because it originally included a systematic literature review (Reference Merlin, Tamblyn and Ellery7), time production could sometimes still be regarded as too long by decision makers.

To overcome this limitation attributed to the Danish mini-HTA, a new template was recently developed with a new section that acquires other types of data used in the assessments besides (or instead of) the literature review. In parallel, other HTA producers also developed rapid reviews (Reference Merlin, Tamblyn and Ellery7). Beside this need for rapid publication, other needs are expressed by a variety of stakeholders in the HTA production process (Reference Kidholm, Ølholm and Birk-Olsen9), leading to a large variety of products and services. Examples of additional products are horizon scanning report (Reference Merlin, Tamblyn and Ellery7), publication in scientific journal (Reference Lehoux, Tailliez, Denis and Hivon10), clinical practice guideline (Reference Demirdjian11), health economic evaluation (Reference Poder and Fisette12), and mathematical decision model (Reference Stephens and Doshi13). Examples of services are field evaluation (14;Reference Poder, Bellemare, Bédard, He and Lemieux15), simulation exercise (Reference Chilcott, Tappenden and Rawdin16), technical and methodological support (hospital-based HTA), capacity building in research and management (Reference Demirdjian11), survey and questionnaire validation (Reference Bedard, Poder and Larivière17), pharmacy management (Reference Martelli, Billaux, Borget, Pineau, Prognon and van den Brink18), and laboratory experiment (Reference Poder, Pruneau and Dorval19).

Considering this proliferation in new products and services provided by HTA agencies and hospital-based units, one can ask if a hospital-based HTA unit has all the necessary skills and resources to meet the needs of decision makers and be useful for them and for the society. In this article, we present a case of the choice of the new smart pumps for our center, as facilitated by our hospital-based HTA unit. This corresponds to the new application of HTA methods at a local university hospital (i.e., information for procurement) and is in line with the new imperative of the Conference Board of Canada for value-based procurement (Reference Arshoff, Henshall, Juzwishin and Racette20;Reference Prada21). In addition to the case presentation, we discuss the difficulties encountered and provide a list of nine steps that hospital-based HTA units can follow for future work.

CONTEXT OF THE APPRAISAL

In 2010, to address the various technical problems that resulted from the implementation of Hospira company Symbiq smart metering pumps at the Centre Hospitalier Universitaire de Sherbrooke (CHUS), a working group was set up. This group comprised clinical staff, managers, biomedical engineers, and members of the HTA unit of the CHUS. Our university hospital is located on two sites (Hôpital Fleurimont and Hôtel-Dieu), but with only a single HTA unit that provides services for both sites. The objective of the working group was to identify the frequency and causes of the observed dysfunctions (e.g., white screen, false alarm, and wireless communication) and to take corrective action in cooperation with the Hospira company. By the end of 2013, general management decided to repeat this process for the preparation of the tender to replace the metering pumps in place at CHUS. Hospira company Symbiq smart metering pumps should, in effect, be withdrawn from the market no later than the end of June 2015, given that some technical problems could not be corrected by the company.

The objective of setting up such a working group before the tender was to conduct a comprehensive assessment of the available technologies on the market as well as the needs of our clinical staff to avoid the mistakes made in the previous tender. Preparatory work for the tender was initiated at the beginning of 2014, and the budget for the acquisition of new metered pumps was 3 million Canadian dollars.

Metered pumps are generally used to administer, either continuously or intermittently, intravenous medical substances or solutes to the patient. Unlike traditional metered pumps, smart pumps allow the integration of predefined drug protocols to be integrated into it by using a drug-dose management software. This software uses a library of medicines comprising maximum and minimum doses, with some being passable and others being impassable. If these limits are exceeded, an audible and/or visual alarm will be triggered. The computerized management of programmed parameters also validates the calculation of doses and flow rates, thereby precluding certain calculation errors. All programming and alarms data are stored in the pump and are used to generate reports.

OBJECTIVES

The main objective of this article is to use the case study of smart pumps’ renewal as an example to the additional activities conducted in collaboration with the HTA unit of the CHUS and to provide guidance for procurement (tenders) based on HTA.

METHODS

The works in connection with the procurement of the smart pumps covered the following five areas:

Scientific Literature

A rapid review was conducted. The research has focused on studies comparing the various benefits of different models of pumps available on the Canadian market. A search in the ScienceDirect and PubMed databases, as well as the Canadian Agency for Drugs and Technologies in Health (CADTH), Institut national d'excellence en santé et en services sociaux (INESSS), Emergency Care Research Institute (ECRI), Health Canada, and the Food and Drug Administration (FDA) databases was conducted. This search was completed by the identification of studies included in the references of the articles retrieved and by the documentation provided by other University Hospitals in the province. This search was conducted in February 2014 and updated in December 2014.

Compliance Criteria

The identification of the compliance criteria for the tender (i.e., features that the pump should have) was performed in four steps: (i) extraction of criteria from the literature (Reference Morgan and Siv-Lee2225), (ii) consultation with other Quebec University Hospitals (CHU de Québec, CHU Sainte-Justine), (iii) consultation with clinical staff, and (iv) validation by the biomedical engineering department (BED).

During the consultation with the clinical staff (i.e., focus group with representatives from prioritized units), they were asked to identify the criteria that had been overlooked during the development of the first list of compliance criteria. After adjusting this list, the same people scored each criterion in a grid according to whether they were considered to be mandatory, important, desirable, and optional. The resulting criteria were used to establish a final list including only the mandatory criteria for the tender. During this last step, the BED and the procurement service ensured that the list produced was in compliance with provincial rules to conduct a tender.

The care units prioritized when identifying compliance criteria and clinical needs were neonatology, pediatrics, intensive care, operating theaters, emergencies, and oncology. The needs of the pharmacy, of the user transport service, and the BED were also taken into consideration.

Needs of Care Staff

The assessment of care staff needs in terms of pump usage and the number of channels was carried out in three distinct stages. First, a discussion was held regarding the choice of continuing with a smart pump or returning to a traditional pump model. This debate was supplemented by the existing literature on the pros and cons of smart pumps and through consultations with care staff representatives. Individuals who participated in the debate were from nursing care, the care staff in the operating theaters, the medical profession, pharmacy, biomedical engineering, procurement service, and the HTA unit.

Then, a consultation with the care staff, followed by a Failure Modes and Effects Analysis (FMEA), was conducted to determine whether metered pumps or syringe pumps should be used for the intermittent administration of a drug (26). To document this FMEA, a map of the administration process for intermittent medication was created for both a metered pump and a syringe pump. Then, the FMEA consisted of identifying types of error, ranking them in order of importance, and identifying their causes to better assess the process to be implemented to prevent their occurrence. Notably, during the classification process, we changed the ISMP Canada's detectability scale (26) from four to three levels (low, medium, and high) to facilitate this categorization according to the knowledge of the participants. A final choice between the advantages and disadvantages of each technology, including the corrective measures put in place, enabled the choice of the technology to be retained.

Finally, the needs of the care staff were evaluated in terms of the number of channels currently on use at the CHUS and the nature of their use. To do this, an audit was conducted over 2 days in March 2014 in all care units (i.e., 48 units for a total of 713 beds) and in different sites where we could find metered pumps or syringe pumps (i.e., two BED workshops, the clinical research center, and the simulation laboratory). The audit was performed by four pairs of nurse clinicians in clinical development using a computerized questionnaire on InfoPath. The main data collected were regarding the following parameters: the model and the pump number, its location, whether or not the pump was in use, the name of the drug library used, the type of product infused, the programming features, as well as the tubing mounting.

Technical Features

The technical feature tests to be performed were selected on the basis of the difficulties experienced with smart pumps in different University Hospitals in Quebec (CHUS, CHU de Québec, CHU Sainte-Justine). Technical tests identified in the scientific literature were also taken into consideration. A focus group of representatives of care and BED staff, as well as a consultation with the head of risk management at the CHUS, was used to select the tests to be performed. The chosen tests were as follows: mechanical functionality, screen lock (steps, duration, and difficulty), air emptying under gravity and pump purge, free flow, pump starting and restarting time (compared with the actual pump), opening time of the door (compared with the actual pump), rate of appearance of air bubbles over 48 hours of use (one at 10 ml/hr and the other at 80 ml/hr), Taxol at 200 ml/hr to study if the drug foams, anti-reflux valve with secondary methylene blue tinted infusion, alarm in dB at 1 meter for different stages and programming, number of steps and programming times with and without the drug library, intravenous pole stability with several pumps, and proximal and distal occlusion.

Clinical Difficulties and Facilities

The evaluation of the difficulties and facilities encountered by the care staff with the pumps was carried out during the clinical simulation tests. For this, several scenarios responding to frequent real situations, but also corresponding with critical situations, were examined. These scenarios were constructed according to the experiences of the care staff, as well as the examples provided by different University Hospitals in Quebec (i.e., scenarios used at the CHUS in 2010, scenarios of the CHU de Quebec and those of the CHU Sainte-Justine). In total, ten scenarios were constructed and validated in discussion groups. These scenarios were then integrated into questionnaires using 4-level Likert scales (unsatisfactory, less than satisfactory, satisfactory, and highly satisfactory).

Each participant had three general care scenarios to evaluate and then one or several scenarios specific to his/her care sector specialty. The six targeted sectors were as follows: general care (3 scenarios), critical care (2 scenarios), operating room (1 scenario), oncology (1 scenario), pediatrics (1 scenario), and user transport service (2 scenarios). For each scenario, between 2 and 16 criteria were evaluated (e.g., ease of handling of the pump, ease and speed of various programming parameters, clarity of the display, alarm management). Brief training regarding the programming of the pump was provided to the participants before the testing. The choice of providing brief training was also allowed for the ease of learning and intuitive character of the programming in clinical tests to be measured. The participants were instructed to carry out the scenarios as if they were in a real situation. They would subsequently give their feedback and comments by means of the online survey platform SurveyMonkey.

Final Evaluation

After all the activities were carried out, a final evaluation of the pumps was performed according to six criteria with different weighting: safety (25 percent), clinical efficacy (30 percent), ergonomics (15 percent), maintenance (5 percent), information technology (10 percent), and investment perennity (15 percent). These criteria are those used by the BED for a tender. The weights were discussed during a meeting with representatives of the BED, the nursing and medical staff, the pharmacy, the procurement service, and the HTA unit. Each weight was discussed until a consensus was reached. To be selected, a pump had to achieve a minimum final score of 70 percent and a minimum score of 70 percent in both safety and clinical efficacy criteria. Three representatives were in charge to score each criterion: one from the BED, one from the procurement service, and the last one was the person in charge of the smart pumps’ renewal (i.e., a manager from the nursing department). The final score was the mean of the score provided by all three representatives.

RESULTS

Scientific Literature

The literature surveyed was mainly limited to implementation experiences of metered pumps and very few articles showed a direct comparison (Reference Wetterneck, Skibinski and Roberts2730). One study compared the configuration of different smart pumps to determine the best one that would reduce the amount of medication errors (Reference Fan, Pinkney and Easty31). This study was conducted under laboratory conditions and concluded that no pump is perfect. Therefore, they indicated that each organization must evaluate the available pumps according to their own needs. According to these authors, several characteristics, however, would make it possible to increase the efficiency and security of smart pumps: (i) Users should be encouraged to use the dose-error reduction system; (ii) The default programmed settings must match the type of information included in the prescription and be presented in the same order; (iii) Alert messages must be informative and prominent; and (iv) The secondary infusion mode must be easily accessible and information regarding the infusion method should be clearly visible.

Other studies compared the safety of using different generations of pump (i.e., traditional, smart, and smart with bar code) (Reference Rothschild, Keohane and Cook32Reference Trbovich, Pinkney, Cafazzo and Easty35). As indicated by Pinkney et al. (Reference Pinkney, Trbovich, Fan, Rothwell, Cafazzo and Easty36), some of these studies indicate a significant reduction in some errors while others report a minimal impact. In particular, the study of Trbovich et al. (Reference Trbovich, Pinkney, Cafazzo and Easty35) reports that the “hard limit” (i.e., impassable) of the smart pump will reduce dosing errors compared with a traditional pump (38 percent versus 75 percent), while the “soft limit” (i.e., passable) will not (i.e., staff ignore the alarm). This study also indicates that the smart pump neither reduces “incorrect drug” errors, nor errors related to secondary infusions. Finally, this study reports that the conversion program for unit calculations in smart pumps can significantly reduce conversion errors (58 percent versus 93 percent).

A recent review of the literature indicates that smart pumps generally reduce wrong rate and wrong dose errors as well as pump setting errors, which reduces the number of undesirable events and improves practices (from a zero error reduction to a 79 percent reduction and from no adverse events reduction to a 22 percent reduction) (Reference Ohashi, Dalleur, Dykes and Bates37). However, these improvements depend strongly on the pump's design, configuration (e.g., hard limit, default library, library quality), its deployment (e.g., type of training and presence or absence of barcodes), and care staff compliance (Reference Pinkney, Trbovich, Fan, Rothwell, Cafazzo and Easty36). Ohashi et al. (Reference Ohashi, Dalleur, Dykes and Bates37) also indicate that smart pumps introduce the possibility of the wrong drug library being used and do not allow for the detection of some errors such as omissions, preparation, prescription, or programming errors. In addition, it is possible that there could be errors or omission of drug(s) from the drug library (Reference Husch, Sullivan and Rooney38). Ohashi et al. (Reference Ohashi, Dalleur, Dykes and Bates37), however, indicate that the errors induced by the configuration of smart pumps are extremely rare.

It is also noteworthy that none of these studies indicate the occurrence of major dysfunctions with the use of smart pumps, whereas from the experience at our hospital, we know that such dysfunctions may occur. In addition, the pumps mentioned in the literature do not correspond to those available in the Canadian market. Accordingly, the information provided by this rapid review of the literature, while useful, was not sufficient for a decision on the acquisition of smart pumps.

Compliance Criteria

A comprehensive list of 278 criteria in seventeen categories was rated by five groups of two to three representatives of the clinic (i.e., nurses, clinical officers, a biomedical engineer, a pharmacist, respiratory therapists, and an anesthetist). The scores obtained and the modifications carried out led to the establishment of a final list of 108 mandatory criteria in eleven categories for the tender.

Needs of Care Staff

Following the analysis of the scientific literature (see above) and group discussions on whether to issue a tender for smart pumps or for traditional pumps, a collective decision to go ahead with smart pumps was taken. The choice of the smart pump was made because it prevents a large number of errors related to wrong dose and unit conversion, the care staff are used to using the drug library, a return to nonsmart pumps would be considered as going against the tide of technological change and, the transition to smart pumps has been strongly recommended by the ECRI since 2007 (39;40).

The mapping processes for primary and secondary infusions with smart pumps and syringe pumps were completed in March 2014. These maps were used to conduct a FMEA aimed at determining whether a smart pump or a syringe pump, should be used for the administration of intermittent medication. The main risks identified during the FMEA for each technology were as follows: infectious or particle contamination, lack of bacterial and chemical stability, incompatibility between the current infusion and medication to be administered, wrong dose, wrong administration duration, administration delay, longer preparation time, preparation method, multiple steps for administration, rinse volume, and administration of an unwanted bolus.

In the case of seven of the eleven risk categories, the smart pump presented a greater risk than that of the syringe pump (Table 1). Some risks associated with the use of the smart pump may, however, be avoided through the ongoing training of the care staff. On the other hand, the changing of practices associated with the substitution of smart pumps for syringe pumps for the secondary infusion is a major risk in a context of pump replacement. In particular, this could lead to increased pressure on staff in terms of changing practices and potentially pose additional risk of errors, in addition to making staff management more difficult.

Table 1. Risks Identified by the FMEA for the Administration of Intermittent Medication

Note. ☺ = no or low risk; ▲ = high risk.

It was also considered that the purchase of syringe pumps would generate additional costs because it is still necessary to buy smart pumps. On the other hand, the syringe pump tubing would be cheaper than that for the smart pump, but the higher cost of preparing the syringe in the pharmacy would reduce this advantage. Thus, in light of these elements, general management has concluded that the benefits of the syringe pump were not high enough to justify an immediate change of practice for secondary infusions.

The audit conducted to determine the number of channels helped identify 692 pumps (880 channels) in the inventory and 20 other pumps without an inventory number (total of 900 channels). The results obtained indicate that 42.7 percent of pumps were found to be in operation at the time of the audit, and only 7 percent were being used for secondary infusion. Among the programmed smart pumps, the correct clinical care unit (CCU) was selected in 98 percent of cases. Solute was infused into 62.4 percent of channels A and 43.1 percent of channels B. The concordance between the infused product and the product selected in the CCU was consistent with 83 percent for channel A, 89.3 percent for channel B, and 84.1 percent for secondary infusions.

The patient's name and the date were inscribed on the primary bag in 17.3 percent and 36.3 percent of cases, respectively. With regards to secondary infusion, the patient name and the date were inscribed in 76 percent and 81.3 percent of cases, respectively. For ongoing secondary infusions, over 89 percent of mini-bags were placed higher than the primary bag and 92.3 percent had correct tube fitting. When a mini-bag was present, but no secondary infusion was in progress, only 10 percent of primary infusions were higher and only 16 percent were clamped shut (Table 2).

Table 2. Main Results of the Audit

Note. PCA = patient-controlled analgesia; CCU = clinical care unit; A-B = channels.

Technical Features

Only the Infusomat pump from the B. Braun company met all 108 compliance criteria listed in the tender. Technical tests were, therefore, undertaken on this pump. Tests were repeated no more than five times in each case.

Barring some tests in which a liquid flow was found to be oozing from the periphery of the Y tubing, none of the technical tests indicated specific problems. For example, we observed no problems related to air bubbles, free flowing, drug foaming, anti-reflux valve, false occlusion problems, or excessively long programming times. The sound levels of the pumps ranged on average between 38 and 70 dB during operation (starting, opening, closing, purge, and various flow rates) and 71 dB during the alarm, with a peak of up to 95 dB. In comparison, the Canadian standard in healthcare settings is 50 dB at all times with a maximum peak of 90 dB, except in pediatric settings where the maximum is 65–79 dB.

Clinical Difficulties and Improvements

Representatives from each care sector were nominated by their managers to participate in the clinical simulation tests. A total of 55 individuals participated in the tests in August 2014. Given that each participant was asked to complete the three general care scenarios followed by scenario(s) specific to their care sector, it was possible to divide the participants as described in Table 3.

Table 3. Participants’ Breakdown

With the exception of the “ease of identifying the volume infused following zeroing,” for which only 80.5 percent of respondents reported as being satisfied or very satisfied, the evaluation results were all significantly above 80 percent of satisfied or very satisfied, most often between 90 and 100 percent. Notably, the positive response for “ability to bypass the alarm and the ease with which this was done” was very high (92 percent), which could be problematic in the future. Many comments were also made subsequent to the clinical simulation tests. These have helped to adapt the training plans provided during the massive deployment of pumps to care units.

Some comments were also helpful with regard to the default settings of the selected pump (e.g., duration and intensity of the alarm) and for the building of drug libraries. Finally, some instances of flow at the anti-siphon valve of the tubing were observed during the clinical simulation tests. This was also observed later in the staff training phase during the massive deployment of pumps. It was, therefore, decided to use the second tube model offered by the company.

DISCUSSION

What We Did

The results of the work done at the CHUS showed great similarity with those reported in other studies (Reference Namshirin, Ibey and Lamsdale28;41) in that the analysis of the human factor was conducted in conjunction with technical evaluations. This approach enabled us to best determine the needs of users and consider the implementation of adaptation strategies.

Our work has identified several limitations associated with the use of smart pumps. Nevertheless, it appears that the training programs in place, as well as preventive measures, will reduce them. First, it was clearly identified that smart pumps can reduce, but not eliminate completely, programming-related medication errors. Indeed, some alarms from the pumps can be overridden by the caregiver, and several strategies can be used to circumvent the dose-error reduction system, which can sometimes appear as an additional constraint (Reference Ohashi, Dalleur, Dykes and Bates37). Therefore, it is suggested that drug libraries be further standardized, to adapt them to changes in practice, reduce the number of unnecessary alarms, and configure the pump to make it more difficult to bypass the dose-error reduction system (Reference Pinkney, Trbovich, Fan, Rothwell, Cafazzo and Easty36;Reference Ohashi, Dalleur, Dykes and Bates37).

Furthermore, coupling the smart pumps with a barcode system connecting the patient to their prescription would significantly reduce the errors that are currently not addressed by smart pumps, such as errors of patient name and drug name. Although this addition is not within the immediate plans of the CHUS, this is an interesting option for the future. These improvements provided by the smart pump technology, however, are not sufficient to reduce errors alone, and a strong compliance of care staff with regard to the safe use of the pump remains fundamental (Reference Rothschild, Keohane and Cook32). Therefore, a training and deployment plan with a strong emphasis on the human dimension of pump use remains essential.

Regarding the needs of the healthcare staff, a collegial approach to setting compliance criteria and conducting FMEA and clinical simulation tests has allowed most care staff to be included in this stage of the tender process. This approach has the advantage of allowing greater staff adhesion to the change and thereby to ensure its successful implementation. Similarly, the involvement of healthcare staff in certain technical tests allowed the nullification of some concerns related to previous experiences.

Other activities, such as assessment of the compatibility of the wireless network, the management of the training plan, were also conducted during the preparation of the tender. These have, however, been carried out without the cooperation of the HTA unit. A situation that varies according to the HTA unit, as shown by the work of the CHU Sainte-Justine which mainly considered the issues raised by the renewal of their metering pumps (41). In addition, other tests that were not addressed as part of the assessment for the tender are to be realized and will allow us to further adjust our practices. These include tests of hemolysis generated by the smart pump during blood transfusion, which were conducted in 2016 (Reference Poder, Boileau and Lafrenière42).

What Information We Bring

From this experience, we can identify nine generic steps in an enlarged HTA process to comply with the needs of decision makers: know the technology: provide an overview of current technologies and characteristics; document advantages and disadvantages (e.g., literature review, interviews with users); identify needs (e.g., clinical and managerial); check if technical features comply with needs; document process (e.g., FMEA) and identify potential pitfalls and risks (e.g., costs); perform technical tests; conduct clinical tests with end-users (e.g., simulation and clinical trial) and identify solution (e.g., training); final assessment (e.g., MCDA); provide recommendations.

Of note here is that a key factor in the success of the project was the collegial approach that we adopted during the preparation of the procurement along with a permanent consideration of human factors.

HTA Toolbox

In this study, we showed that HTA activities are useful for value-based procurement. This is markedly different from the “traditional” procurement that focuses on technical issues rather than benefit analysis and usability. In this setting, a better integration of activities between the BED, the procurement service, and the HTA unit is highly desirable. However, even if this were done, an HTA manager may still legitimately ask the question of whether HTA specialists or biomedical engineers can offer all the services described in this study and follow all steps.

Indeed, skills required to perform these activities are very diverse and include methodological support, design, and supervision of technical tests or clinical trials, statistical analysis, computer programming, conducting surveys, and FMEA along with more traditional activities as the systematic literature review. All these new activities rely much more on specific individual skills than on the training provided to HTA specialists and biomedical engineers. As a consequence, conducting such activities to meet the needs of decision makers may well prove to be a double-edged sword.

On one hand, traditional activities would not provide enough information to allow for a sufficiently informed decision making. Therefore, complementing these traditional activities with other more varied ones would be very useful for decision makers (Reference Dupouy and Gagnon43;Reference Hailey44). On the other hand, the needs of hospitals can sometimes be so specific that the skills of HTA specialist or biomedical engineers may not be sufficient to produce reliable information. There is, therefore, a risk of producing biased information that lacks in rigor and expertise. In such cases, it would be necessary to delegate some unconventional activities to other specialists.

However, this delegation is not always possible, especially in hospitals with limited employees having sophisticated methodological assessment knowledge, and when the HTA unit and BED are often considered the only resources available internally for the evaluation of medical technologies. An external delegation would still be possible, but it has additional costs and, above all, without adequate knowledge of the needs and context of the hospital, it could not be carried out within the decision window. In this setting, to survive and prove their utility, it is necessary for HTA producers to combine these activities with new products and services. Interestingly, as noted by Lehoux et al. (Reference Lehoux, Tailliez, Denis and Hivon10) and Merlin et al. (Reference Merlin, Tamblyn and Ellery7), even after years of efforts provided by several agencies to standardize or streamline the content of HTA reports, there continues to be increasing diversification of their products and services. This paradox is due not only because of the need for a quick response to requests from policy makers but also for identifying products and services more tailored to their needs and context (Reference Sampietro-Colom, Lach and Cicchetti6;Reference Kidholm, Ølholm and Birk-Olsen9).

CONCLUSION

Through this case study pertaining to the renewal of smart pumps, we showed that it is possible to widen the scope of HTA activities to comply with the needs of decision makers. To achieve this, a high degree of flexibility as well as multidisciplinary skills are necessary. The success of this approach is promising considering that the needs of policy makers will continue to evolve with the never-ending transformations in the healthcare system. However, this will increase the challenges faced by stakeholders in HTA. To meet these needs and to prove their usefulness, HTA units will have to find a balance between traditional activities related to HTA and the new ones described in this study. This could necessitate a diversification of skills of the team members of the HTA unit.

CONFLICTS OF INTEREST

The author has no conflict of interest to declare.

References

REFERENCES

1. Bodenheimer, T. High and rising health care costs. Part 2: Technologic innovation. Ann Intern Med. 2005;142:932957.Google Scholar
2. Henshall, C, Oortwijn, W, Stevens, A, Granados, A, Banta, D. Priority setting for health technology assessment. Theoretical considerations and practical approaches. Priority setting Subgroup of the EUR-ASSESS Project. Int J Technol Assess Health Care. 1997;13:144185.Google Scholar
3. INAHTA. What is Health Technology Assessment (HTA)? International Network of Agencies for Health Technology Assessment. 2016. http://www.inahta.org (accessed January 29, 2016).Google Scholar
4. Lampe, K, Makela, M, Garrido, MV, et al. The HTA Core Model: A novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. 2009;25:920.Google Scholar
5. Sampietro-Colom, L, Lach, K, Pasternack, I, et al. Guiding principles for good practices in hospital-based HTA units. Int J Technol Assess Health Care. 2015;31:457465.Google Scholar
6. Sampietro-Colom, L, Lach, K, Cicchetti, A, et al. The AdHopHTA handbook: A handbook of hospital based health technology assessment (HB_HTA). Public deliverable. The AdHopHTA Project (FP7/2007-13 grant agreement nr 305018) 2015. http://www.adhophta.eu/sites/files/adhophta/media/adhophta_handbook_website.pdf (accessed December 25, 2016).Google Scholar
7. Merlin, T, Tamblyn, D, Ellery, B. What's in a name? Developing definitions for common health technology assessment product types of the International Network of Agencies for Health Technology Assessment (inahta). Int J Technol Assess Health Care. 2014;30:430437.Google Scholar
8. Ehlers, L, Vestergaard, M, Kidholm, K, et al. Doing mini-health technology assessments in hospitals: A new concept of decision support in health care? Int J Technol Assess Health Care. 2006;22:295301.Google Scholar
9. Kidholm, K, Ølholm, AM, Birk-Olsen, M, et al. Hospital managers’ need for information in decision-making: An interview study in nine European countries. Health Policy. 2015;119:14241432.Google Scholar
10. Lehoux, P, Tailliez, S, Denis, JL, Hivon, M. Redefining health technology assessment in Canada: Diversification of products and contextualization of findings. Int J Technol Assess Health Care. 2004;20:325336.Google Scholar
11. Demirdjian, G. A 10-year hospital-based health technology assessment program in a public hospital in Argentina. Int J Technol Assess Health Care. 2015;31:18.Google Scholar
12. Poder, TG, Fisette, J-F. Are drug-coated balloons cost-effective for femoropopliteal occlusive disease? A comparison of bare metal stents and uncoated balloons. J Comp Eff Res. 2016;5:335344.CrossRefGoogle ScholarPubMed
13. Stephens, J, Doshi, J. International survey of methods used in health technology assessment (HTA): Does practice meet the principles proposed for good research? Comp Eff Res. 2012;2:29.Google Scholar
14. INESSS. Réflexion sur l’évaluation terrain en évaluation des technologies et des modes d'intervention en santé. Montréal, QC. 2012. https://www.inesss.qc.ca/publications/publications/publication/reflexion-sur-levaluation-terrain-en-evaluation-des-technologies-et-des-modes-dintervention-en.html (accessed January 29, 2016).Google Scholar
15. Poder, TG, Bellemare, C.K. Bédard, S, He, J, Lemieux, R. New design of care: Assessment of an interdisciplinary orthopedic clinic with a pivot nurse in the province of Quebec. Orthop Nurs. 2010;29:381389.Google Scholar
16. Chilcott, J, Tappenden, P, Rawdin, A, et al. Avoiding and identifying errors in health technology assessment models: Qualitative study and methodological review. Health Technol Assess. 2010;14:1107.Google Scholar
17. Bedard, SK, Poder, TG, Larivière, C. Validation d'un outil de mesure de l'interdisciplinarité en pratique clinique: le questionnaire IPC65. Sante Publique. 2013;25:763773.Google Scholar
18. Martelli, N, Billaux, M, Borget, I, Pineau, J, Prognon, P, van den Brink, H. Introduction of innovative medical devices at French university hospitals: An overview of hospital-based health technology assessment initiatives. Int J Technol Assess Health Care. 2015;31: 1218.Google Scholar
19. Poder, TG, Pruneau, D, Dorval, J, et al. Effect of warming and flow rate conditions of blood warmers on red blood cell integrity. Vox Sang. 2016;111:341349.Google Scholar
20. Arshoff, L, Henshall, C, Juzwishin, D, Racette, R. Procurement change in Canada: An opportunity for improving system performance. Healthc Manage Forum. 2012;25:6669.Google Scholar
21. Prada, G. Value-based procurement: The new imperative for Canada. Online presentation, May 10, 2016. http://hscn.org/Data/Sites/8/media/1050-gabriela-prada.pdf (accessed December 25, 2016).Google Scholar
22. Morgan, L, Siv-Lee, L. Intelligent infusion. Nurs Manage. 2009;40: 2024.Google Scholar
23. Phelps, P, Kleinke, P. Choosing a pump: The request for proposal and value analysis. In: Phelps, P, ed. Smart Infusion pumps implementation, management, and drug libraries. Bethesda, MD: American Society of Health-System Pharmacists; 2011:926.Google Scholar
24. Rossi, PG, Camilloni, L, Todini, AR, et al. Health technology assessment of the negative pressure wound therapy for the treatment of acute and chronic wounds: Efficacy, safety, cost effectiveness, organizational and ethical impact. Ital J Public Health. 2012;9:4666.Google Scholar
25. ECRI Institute. Best and worst infusion pump. Ratings for six products. Health Devices. 2012;41:378391.Google Scholar
26. ISMP-Canada. Modèle canadien d'analyse des modes de défaillances et de leurs effets (AMDE). Identification proactive des risques dans le milieu de la santé. Montréal, QC: ISMP-Canada; 2007.Google Scholar
27. Wetterneck, TB, Skibinski, KA, Roberts, TL, et al. Using failure mode and effects analysis to plan implementation of smart i.v. pump technology. Am J Health Syst Pharm. 2006;63:15281538.Google Scholar
28. Namshirin, P, Ibey, A, Lamsdale, A. Applying a multidisciplinary approach to the selection, evaluation, and acquisition of smart infusion pumps. J Med Biol Eng. 2011;31:9398.Google Scholar
29. Elias, BL, Moss, JA. Smart pump technology. What we have learned. Comput Inform Nurs. 2011;29:TC61TC67.Google Scholar
30. ECRI Institute. Infusion pump integration. Health Devices. 2013;42:210221.Google Scholar
31. Fan, M, Pinkney, S, Easty, A. Smart medication delivery systems: Infusion pumps. Supplementary report: Evaluation of effective smart pump design and education strategies. Toronto, Ontario: University Health Network; 2010.Google Scholar
32. Rothschild, JM, Keohane, CA, Cook, EF, et al. A controlled trial of smart infusion pumps to improve medication safety in critically ill patients. Crit Care Med. 2005;33:533540.Google Scholar
33. Murdoch, LJ, Cameron, VL. Smart infusion technology: A minimum safety standard for intensive care? Br J Nurs. 2008;17:630636.CrossRefGoogle ScholarPubMed
34. Trbovich, P, Jeon, J, Easty, A. Smart medication delivery systems: Infusion pumps. Toronto, Ontario: University Health Network; 2009.Google Scholar
35. Trbovich, PL, Pinkney, S, Cafazzo, JA, Easty, AC. The impact of traditional and smart pump infusion technology on nurse medication administration performance in a simulated inpatient unit. Qual Saf Health Care. 2010;19:430434.Google Scholar
36. Pinkney, S, Trbovich, P, Fan, M, Rothwell, S, Cafazzo, JA, Easty, A. Do smart pumps actually reduce medication errors? Biomed Instrum Technol. 2010;44:6469.Google Scholar
37. Ohashi, K, Dalleur, O, Dykes, PC, Bates, DW. Benefits and risks of using smart pumps to reduce medication error rates: A systematic review. Drug Saf. 2014;37:10111020.Google Scholar
38. Husch, M, Sullivan, C, Rooney, D, et al. Insights from the sharp end of intravenous medication errors: Implications for infusion pump technology. Qual Saf Health Care. 2005;14:8086.CrossRefGoogle ScholarPubMed
39. ECRI Institute. Evaluation: General-purpose infusion pumps. Health Devices. 2007;36:309336.Google Scholar
40. ECRI Institute. Top 10 health technology hazards for 2013. Health Devices. 2012;41:123.Google Scholar
41. UETMIS-Ste-Justine. Constats et enjeux relies à l'acquisition et à l'implantation d'un nouveau parc de pompes volumétriques et de pousse-seringues au CHUS Sainte-Justine. Montréal, QC: UETMIS-Ste-Justine; 2009.Google Scholar
42. Poder, TG, Boileau, J-C, Lafrenière, R, et al. Quantitative assessment of haemolysis secondary to modern infusion pumps. Vox Sang. 2017;112:201209 Google Scholar
43. Dupouy, C, Gagnon, M-P. The influence of hospital-based HTA on technology acquisition decision. Int J Hospital-Based Health Technol Assess. 2016;1:1928.Google Scholar
44. Hailey, D. Commentary on “The influence of hospital-based HTA on technology acquisition decision”. Int J Hospital-Based Health Technol Assess. 2016;1:2930.Google Scholar
Figure 0

Table 1. Risks Identified by the FMEA for the Administration of Intermittent Medication

Figure 1

Table 2. Main Results of the Audit

Figure 2

Table 3. Participants’ Breakdown