Innovative medical devices can improve the delivery and outcome of patient care. In a context of increasing healthcare expenditure, the introduction of innovative and costly medical devices has become a major issue for health policy makers. According to the International Information Network on New and Emerging Health Technologies (EuroScan International Network), “Emerging technologies are technologies that are not yet adopted by the health care system. [. . .] Medical devices will be before marketing, or within 6 months of marketing or marketed but less than 10 percent diffused or localized to a few centers.” Health technology assessment (HTA) practices have been successfully applied by many healthcare systems worldwide (Reference Drummond, Schwartz and Jönsson1;Reference O’Donnell, Pham, Pashos, Miller and Smith2). In France, HTA constitutes a major health policy tool for the French government (Reference Orvain, Xerri and Matillon3). In 2004, an independent health agency, the French National Authority for Health (Haute Autorité de Santé), was established to concentrate HTA activities and to provide decision support concerning the reimbursement, nationwide, of medical devices for individual use (Reference Weill and Banta4).
However, HTA agencies in France and elsewhere assess only a small proportion of the new health technologies coming onto the market each year (Reference Kidholm, Ehlers, Korsbek, Kjaerby and Beck5). Meanwhile, hospitals, and university/teaching hospitals (UHs) in particular, are faced with ever-growing demands for innovative and often costly medical devices that may not yet have been evaluated by national/regional HTA agencies (Reference Boudard, Martelli, Prognon and Pineau6). In addition to providing training for future physicians and other health professionals, UHs are also the frontrunners in medical research and technology. In a context of limited resources, this situation has forced hospitals around the world to develop local HTA systems, to guide the selection of innovative medical devices and investment in such equipment (Reference Mitchell, Williams, Brennan and Umscheid7). Hospital-based HTA models from Australia, Canada, Denmark, Spain, and Italy have now been reported (Reference Mitchell, Williams, Brennan and Umscheid7–Reference Cicchetti, Francesconi, Guizzetti, Lago and Maccarini11). In a 2008 worldwide survey, the Hospital-Based HTA Sub-Interest Group of the Health Technology Assessment International (HTAi) Society agreed on four models for HTA within hospitals: the ambassador model, mini-HTA, internal committees and HTA units (Reference Cicchetti, Marchetti, Dibidino and Corio12). These models were conceptualized on the basis of their focus of action (the production of evidence for managerial decision-making and/or to support effective clinical practice) and the level of complexity of the organizational solution implemented (individual or team/group/unit).
The concept of hospital-based HTA is not a recent phenomenon in France. Indeed, the Committee for the Assessment and Dissemination of Technological Innovations (CEDIT), which was established in 1982 to support Parisian university hospitals in decisions relating to technological innovations, is frequently cited as one of the first examples of hospital-based HTA initiatives in Europe (Reference Cicchetti, Marchetti, Dibidino and Corio12;Reference Bodeau-Livinec, Simon, Montagnier-Petrissans, Joël and Féry-Lemonnier13). Furthermore, until 2010, the French Public Health Code insisted that a drug and medical device committee (Commission du Médicaments et des Dispositifs Médicaux Stériles) within hospitals should be responsible for deciding whether drugs and sterile medical devices should be adopted and reviewing the available evidence to issue recommendations on their use. The functioning of these committees was often optimized by splitting them into two committees: a drug committee and a medical device committee. These responsibilities have since been transferred to the institutional medical committee (Commission Médicale d’Etablissement), which may include a drug and medical device committee (and/or their subcommittees) or be based on a new organizational model. Finally, since a 2006 French Ministry of Health circular, new organizational structures called innovation units (Cellule Innovation) have been set up in UHs. Innovation units keep an eye on emerging health technologies and help their institutions to put forward proposals for research on innovative medical devices. These units use full-time and/or part-time health professionals, such as health economists, pharmacists, biomedical engineers, and research assistants.
However, with the exception of the CEDIT, little is known about hospital-based HTA initiatives in France and, more generally, about the processes governing the adoption of innovative medical devices, which are mostly introduced into UHs. The objective of this study was to explore hospital-based HTA activities in French UHs for the introduction of innovative medical devices for individual use and to compare these systems with the conceptual models proposed by the Hospital-Based HTA Sub-Interest Group. Thus, the main purpose of this survey was to provide a picture of the organizational approaches to the assessment of new and innovative medical devices followed in French UHs.
METHODS
Setting
There are officially twenty-seven UHs in mainland France. The UHs of the three largest French cities—Paris, Lyon, and Marseille (PLM)—have specific features because they each include several self-governing hospitals (for convenience, we will refer to these organizations as “local PLMs”). According to this differentiation, the total number of French UHs reaches seventy-six: twenty-four UHs outside PLM and fifty-two “local PLMs” making up the three UHs of PLM. Another specific feature of PLM-UHs is the existence of a central administrative structure for the assessment/purchase of drugs and devices. However, “local PLMs” can also purchase drugs and devices independently. For convenience, we will refer to the central structures as “central PLMs”.
Literature Review
We first carried out a literature review, to identify relevant contributions dealing with hospital-based HTA activities in French university and/or teaching hospitals, and with medical devices in particular. The purpose of this literature review was to retrieve studies reporting the organizational approaches used in French UHs. This review was complementary to a larger literature review carried out in a previous study on a related topic (Reference Martelli, Lelong, Prognon and Pineau14). The scope of technology was limited to medical devices for individual use, with the exclusion of medicines and large medical equipment (like MRI machines, etc.). We searched for articles, letters, and reports published in English or French in Medline, Embase, the HTA database and Google Scholar. Databases were screened for publications from January 1990 to September 2012. The electronic search was conducted by two researchers (N.M. and M.B.) separately, with various combinations of terms, including “hospital based health technology assessment”, “mini-health technology assessment”, “medical device”, “health technology assessment”, “internal committee”, “health technology assessment unit”, “medical device committee”, “local health technology assessment”. We did not use MeSH terms, because they were not suitable for our search and were either too broad or too specific. We retrieved 649 publications, including four articles dealing with hospital-based HTA activities for assessing medical devices in a French UH, but only one detailing the organizational approach used within the institution. Given the lack of information available from previous studies, we decided to adopt a qualitative approach to data collection.
Data Collection: Semi-structured Interviews
Semi-structured interviews appeared to be the most suitable solution for collecting original data on hospital-based HTA in the French UHs. We designed the framework of the interview on the basis of relevant topics concerning hospital-based HTA identified from previous publications, the few reports of French experience identified and our own personal knowledge of this topic. A preliminary interview guide was developed and tested in three interviews. Open-ended questions and closed questions covering six topics were included in the final interview guide (see Supplementary File 1). The six topics were as follows: (i) the nature of the institution, (ii) activities relating to innovative medical devices, (iii) the technology assessment and decision-making process, (iv) the methodology for technology assessment, (v) factors likely to influence decisions, and (vi) suggested improvements to the current process. In general, fifteen to twenty interviews are sufficient to achieve data saturation and to identify trends in the collected material (Reference DiCicco-Bloom and Crabtree15). We, therefore, decided to perform eighteen interviews on a stratified sample of UHs.
Selection of UHs and Interviewees
The scope of the study was limited to medical devices for individual use. Thus, hospitals with no acute care (no surgery or no interventional medicine, for instance) were removed. We also eliminated our own UH, which is a “local PLM,” from the analysis, to avoid bias and to ensure impartiality. The final analysis, therefore, included fifty-five acute-care hospitals. The representativeness of UHs was ensured by the use of a stratified sampling method based on the number of hospital stays for acute care per year per UH. In 2011, the total number of hospital stays for acute care in UHs was 4,906,272 in France, with PLM-UHs accounting for 36 percent of the total, and the other UHs accounting for 64 percent. Interviews were thus carried out to give a similar distribution between PLM-UHs and non–PLM-UHs (36 percent and 64 percent): seven of the eighteen interviews were carried out at PLM-UHs and eleven were carried out UHs outside PLM. We decided to include all three “central PLMs”, because they had an original organizational model, and the other four PLM-UHs were selected at random from the thirty-one remaining “local PLM”. Eleven UHs were selected at random from the twenty-four non–PLM-UHs. Thus, in total, three central PLMs, four local PLMs and eleven UHs not from PLM were included in the survey.
According to the French Public Health Code, hospital pharmacists are responsible for the purchase, management and dispensing of sterile medical devices. As a result, hospital pharmacists are inevitably involved in HTA processes within French UHs and are likely to have a comprehensive knowledge of these processes. Each pharmacist in charge of medical devices at a selected UH was contacted by e-mail or telephone and asked to participate in a survey on innovative medical devices. If a pharmacist declined to participate in the survey, another UH from the same category was selected at random, during a second draw.
The Survey
All interviews were conducted from October 2012 to April 2013 by the same interviewer (N.M.). Given time and budget constraints, the interviews took place face-to-face at the participants’ workplace or by telephone. Published studies provide little evidence of data loss or distortion in interviews conducted by telephone (Reference Novick16). Furthermore, this survey aimed to collect data on processes or facts and was not a sociological survey, in which visual cues might provide decisive information. The anonymity and confidentiality of the data exchanged were guaranteed to all participants. Finally, each participant was asked to give consent and to allow the audio recording of the interview on a tablet computer. None of the interviewees refused to be recorded.
Data Analysis
All interviews were transcribed verbatim and coded by two researchers (N.M. and M.B.) separately. The coding of segments (words and sentences) enables researchers to assign frequency values more easily, to determine the presence/absence of a subtheme or links to other codes (Reference Green and Thorogood17). Segments were coded according to a list of twenty-six codes covering the six topics covered in the interview guide. In cases of discordant coding, the two researchers discussed the discrepancy until a consensus was reached and the decision taken was then entered as the final code. We checked intercoding and intracoding reliabilities, by calculating Cohen's kappa coefficient (Κ). Intercoding reliability compares the coding given by the two researchers, whereas intracoding reliability compares the coding given by the same researcher on two occasions, 1 month apart. Three of the eighteen UHs were selected at random for the measurement of intercoding reliability and three of the remaining fifteen UHs were selected at random for the measurement of intracoding reliability. According to J.R. Landis and G.G. Koch (Reference Landis and Koch18), Κ values between 0.61 and 0.80 indicate “strong agreement” and Κ values between 0.81 and 1 indicate an “almost perfect agreement”. We considered Κ values greater than 0.61 to be acceptable. Κ reached 0.62 for intercoding reliability, 0.88 for intracoding reliability for NM and 0.81 for intracoding reliability for MB. Descriptive statistics for the coded material are reported as proportions for categorical data.
RESULTS
Description of the Technology Assessment and Decision-making Processes
From the hospital pharmacists’ statements, we were able to determine that there were three major types of HTA processes for adopting new medical devices for individual use (Table 1): medical device committees, innovation committees and “pharmacy & management” processes. Each of these processes could be the main HTA process or a secondary HTA process, depending on the institution. “Secondary” suggests that more than one process was operating within the institution concerned. The criteria determining the process to be followed were identified. The main process is followed for expensive medical devices or if expenditure could exceed a financial threshold during the year. Secondary processes, when present, were used for low volumes of medical devices and/or expenditures below the threshold or for emergencies. Medical device and innovation committees appeared to be almost exclusively main processes, whereas “pharmacy & management” processes tended to be secondary (Table 1). Regardless of the UH category, all processes began with a medical request. Based on the statements of the interviewees, we were then able to highlight the key steps in the different processes and their specific features.
Table 1. Decision-Making Processes in the Eighteen UHs Surveyed (n = 18)
aItems for which there may be more than one answer; for this reason, the sum does not equal the total of participants.
1. Medical Device Committees
Medical device committees were mentioned in each UH category. These committees are those described in the French Public Health Code until 2010. These committees were described as multidisciplinary committees issuing recommendations concerning the sterile medical devices used within the institution and reviewing requests for new medical devices. Standardized HTA forms are frequently used to collect such requests from physicians or surgeons. In most cases, the participants indicated that physicians asking for new devices presented a report of the medical significance of the product during a plenary session. The economic aspects of technology assessment were performed by a pharmacist or a public health specialist. Finally, the committee adopted an opinion on the request: positive, negative, or awaiting additional information. For almost three quarters of the UHs with a medical device committee, financial managers were permanent members of that committee. Consequently, the final decision could be made by the committee itself. Almost two-third of committees also implemented patient follow-up for those treated with the technology. Finally, two of the fourteen medical device committees described had recruited full-time and part-time professionals to set up an HTA unit (one was an innovation unit and the other was a scientific secretariat).
2. Innovation Committees
Innovation committees were mentioned by hospital pharmacists at three non–PLM-UHs. Such committees are not officially recognized in French legislation, but they all bore the same name and had virtually the same organization and functioning in the three UHs surveyed. Pharmacists described them as multidisciplinary committees dedicated to the assessment of new health technologies, including medical devices in particular. As for medical device committees, standardized HTA forms were frequently used to requests from physicians or surgeons. In the three UHs concerned, experts within the institution were designated by the committee to assess the requests. As for medical device committees, the physician asking for the new device presented a report and the potential value of the technology was debated during a plenary session, but innovation committees did not seem to have the authority to approve the adoption of a new technology. All pharmacists pointed out that requests were ranked in order of priority, according to the prior technology assessment. The ranking was submitted to the hospital manager, who made the final decision according to available innovation credits, which were often provided by regional health authorities. One of the three innovation committees described had an innovation unit providing it with support for technology assessment.
3. “Pharmacy & Management” Process
Participants from eight UHs described another process that we named “pharmacy & management”. Again, the interviewees gave descriptions of organization that were very similar. The hospital pharmacists of the UHs at which this was the main process indicated that a standardized HTA form was used. Technology assessment was then based on local data input from physicians or surgeons and a literature review and economic assessment carried out by the pharmacist. In most cases, the pharmacist summarized all the relevant information, to produce a report for the decision-makers. The final decision was made by a financial or hospital manager in concertation with the pharmacist.
Limitations of the Technology Assessment and Decision-making Processes
For medical device and innovation committees, the two limitations most frequently cited were delays in decision-making and a quality of assessment too closely linked to the reimbursement available. This second aspect led interviewees to believe that the devices funded by French National Health Insurance were less carefully evaluated than those for which no reimbursement was available. A lack of involvement of physicians in medical device committees was also highlighted. For “pharmacy & management” processes, hospital pharmacists reported a lack of multidisciplinary collaboration and a lack of expertise concerning the medical devices as key limitations.
Methodology of Technology Assessment
As illustrated in Table 2, budget impact was a key part of the assessment mentioned by all non–PLM-UHs and local PLMs. Impacts on organization and strategy were reported only by the interviewees from the non–PLM-UHs and central PLMs. We also asked the participants about the frequency of economic data use. Twelve (66.7 percent) interviewees stated that economic data were always used for the HTA, whereas three (16.7 percent) stated that such data were often used and three (16.7 percent) said that economic data were sometimes used. The frequency of economic data use appeared to be higher at non–PLM-UHs than at the other UHs.
Table 2. Methodology for Technology Assessment (n = 18)
aItems for which there may be more than one answer; for this reason, the sum does not equal the total of participants.
Factors Likely to Influence Decisions
The most frequently cited factor likely to influence decision-making was the cost of the new technology (Table 3). One third of the participants also stated that efficacy and the motivation of the physician requesting the new technology were also likely to influence decisions. Finally, two thirds of interviewees stated that the evaluation strongly influenced the final decision, the remaining third indicating that it had a moderate influence.
Table 3. Factors Likely to Influence Decisions (n = 18)
aItems for which there may be more than one answer; for this reason, the sum does not equal the total of participants.
Suggestions for Improving the Current Process
The interviewees were asked to make suggestions for the improvement of the current technology assessment and decision-making process at their own institution. Five (27.8 percent) participants suggested that national health agencies should deliver an expert opinion on the technology earlier, four (22.2 percent) suggested that specific budgets should be allocated to new technologies and four (22.2 percent) suggested that the management of conflicts of interest should be improved. We then asked the interviewees to suggest the best ways to coordinate medical device assessment activities between French UHs. Half interviewees suggested publishing national guidelines for hospital-based HTA in UHs and promoting a national association of medical device committees. One-third of the participants also highlighted the need to set up a national network of hospital-based HTA professionals.
DISCUSSION
On the basis of our interviews with hospital pharmacists, we can now outline the principal trends for hospital-based HTA initiatives within French UHs.
First, many UHs, wherever they are located, have maintained a medical device committee, despite new French legislation theoretically abolishing these committees. However, we noted disparities among the medical device committee processes studied. The UHs of large metropolitan areas seemed to have made the greatest strides toward the development of hospital-based HTA activities by, for instance, setting up HTA units. Their approach to technology assessment focused more on the technology itself and less on budgetary impact than that of other UHs. This may be because large centers are at the cutting edge of innovation and have to review many new health technologies, very frequently, to remain nationally, or even internationally competitive. In non–PLM-UHs, medical device committee processes have been developed around budgetary and hospital strategy impacts. Due to their more limited financial resources and growing competition with surrounding UHs, the adoption of new technologies must promote the hospital's image and, thus, the recruitment of patients in a regional context. It is, therefore, no coincidence that innovation committees have emerged in this environment. Indeed, innovation committees were designed precisely to provide assistance for managerial decision-making concerning the adoption of innovative medical devices for strategic purposes, whereas medical device committees also have other functions, such as issuing recommendations for the institution. In this respect, French medical device committees are closer to the conceptual model of the internal committee than innovation committees.
In both types of committee, extensive delays in decision-making appeared to be a frequent problem, as reported elsewhere (Reference Bodeau-Livinec, Simon, Montagnier-Petrissans, Joël and Féry-Lemonnier13;Reference Umscheid, Williams and Brennan19). We think that the “pharmacy & management” process has emerged as an alternative to multidisciplinary committees, to decrease the time required for assessment and decision-making, whilst maintaining a formal process in a short time frame, through the use of standardized HTA forms and/or the production of a sufficiently comprehensive report for decision-makers, for example. Can we consider this system to be a proper hospital-based HTA model? This system is heavily dependent on a single professional and lack of expertise was cited as one of its major limitations. This model is far from perfect, but it may be a relevant way to tackle “back door adoption” without formal assessment. We have previously described the process at our own institution, which is not unlike the “pharmacy & management” processes reported here (Reference Martelli, Lelong, Prognon and Pineau14). The use of local data inputs and standardized HTA forms suggests that “pharmacy & management” processes in French UHs may be associated with hospital-based HTA activities and the mini-HTA, through the predominant role of a single professional. Nevertheless, there may be a lack of multidisciplinary collaboration and assessments carried out by a single person cannot really be considered to be HTA. This model is still far from the Danish mini-HTA or other mini-HTA-like models described in previous studies (Reference Kidholm, Ehlers, Korsbek, Kjaerby and Beck5;Reference Ehlers, Vestergaard and Kidholm9;Reference Sampietro-Colom, Morilla-Bachs, Gutierrez-Moreno and Gallo10). We also noted the use of standardized HTA forms in the context of multidisciplinary committees. This shows that the models defined by the HTAi have some limitations. Indeed, they are not mutually exclusive and can be combined within hospitals (Reference Gagnon20). We believe that the question of HTA form use in French hospitals requires further study, with a view to designing a national decision support tool, like the mini-HTA form released by the Danish Center for Health Technology Assessment (DACEHTA) (Reference Ehlers, Vestergaard and Kidholm9). For this reason, we collected all the HTA forms used by the UHs surveyed, for subsequent content analysis and comparison.
We also noted the growing role played by HTA units in French UHs, particularly those with innovation units. The CEDIT, with its HTA unit, pioneered the development of this highest level of hospital-based HTA in France and in Europe. By following this pathway, innovation units are creating new conditions. However, the two innovation units described by the participants displayed a very different type of organization, providing support to medical device and innovation committees. In addition, the composition of HTA teams, the scope of the health technologies studied or even the aspects assessed were not completely similar. It is, therefore, difficult to compare the CEDIT with these innovation units, which appear to be rather heterogeneous. Nevertheless, the first French national meeting on hospital-based HTA activities (4ème Journée Nationale d’Innovation Hospitalière, Nantes, June 7, 2013) showed that the two models had much in common and could progress by learning from each other.
This study has several limitations, due to the qualitative approach used. First, two of the eleven non–PLM-UH pharmacists initially approached (18 percent) refused to participate in the survey. Consequently, another two UHs were selected at random during a second draw. There would be a potential bias if refusal rates were greater than 20 percent (Reference Ghiglione and Matalon21). As refusal rates were below this threshold, the risk of selection bias may be considered to be minimal. The sample studied was relatively small. However, qualitative studies are not generally designed for detailed statistical purposes, including the generalization of data to a population, in particular (Reference Polit and Beck22). We are aware that the results presented here attempt only to describe hospital-based HTA activities in French UHs, rather than to provide a precise assessment of their distribution. Finally, the interviews carried out may have failed to reveal some interesting features because of potential omissions from participants. A more diverse participant sample might have increased the reliability of the findings.
CONCLUSIONS
Despite the limitations of this study, these findings increase our knowledge of hospital-based HTA initiatives in France. The French national health authorities are considering a more comprehensive framework for improving and coordinating medical device assessment activities across the country, but a knowledge of actual practices appears to be essential to ensure that relevant solutions are proposed. National guidelines on hospital-based HTA, the development of a French mini-HTA tool and of national database of innovative medical device assessments would be an interesting approach to improve the current situation. These conclusions strongly echo those of a European Project on hospital-based HTA called the AdHopHTA (Adopting hospital-based HTA) project, which aims to promote collaboration and the coordination of hospital-based HTA initiatives across Europe.
SUPPLEMENTARY MATERIAL
Supplementary File 1 http://dx.doi.org/10.1017/S0266462315000057
CONFLICTS OF INTEREST
All authors report they have no potential conflicts of interest.
