In Finland, the MUMM program (Managed Uptake of Medical Methods) was started 10 years ago as a joint venture of the Finnish Office for Health Technology Assessment (Finohta) at THL (National Institute for Health and Welfare) and the twenty hospital districts providing specialized health care. Its aim is to offer critically appraised information for decisions concerning uptake of new medical methods and to encourage health care decision-makers to commit themselves to evidence-based practices.
The Finnish Health Care Act from 2010 stipulates that the hospital districts are to agree on the principles for adopting new medical methods, and on coordinating the publicly-funded provision of highly specialized medical care within their catchment area. As a result of this legislation, the university hospitals have established joint HTA procedures. This includes collaboration with national authorities, Finohta and the Finnish Medicines Agency. However, Finnish legislation does not require that all new technologies applied in private health care should be assessed before acceptance for reimbursement.
Systematic reviews and recommendations based on the reviews are the tools used in the MUMM program. Information on effectiveness, safety and costs of new, emerging health technologies is presented in MUMM reviews. When relevant, also ethical or organizational issues are discussed. The form of MUMM review is closer to rapid assessment than full systematic review, and the review should be finished within six months. Proposals for review topics usually come from physicians in the hospital districts. They are presented in a structured mini-HTA format adapted from the Danish version (1). All medical methods, procedures and devices can be assessed except medicines which are evaluated by Finnish Medicines Agency. The MUMM Board, consisting of the chief medical officers of the hospital districts, selects the most relevant topics for reviews. For each topic selected, a review group of 1–3 clinicians and 3 methodological experts from Finohta is appointed, including an information specialist. The reviews go through a peer-review process and are published in the Finnish Medical Journal.
After publication, the MUMM Board gives a recommendation based on the MUMM review. This recommendation is structured and traffic lights are used to illustrate the result: when a method receives a green light it should be used. A yellow light indicates that a method can be used selectively and more information of its effects should be accumulated through research, registries and literature. A method that receives a red light should not be used.
The need to assess impact of HTA has risen in many countries. A thorough systematic review shows that there is still little evidence on HTA influence on decision making (Reference Hailey, Werkö and Rosén2). In Finland, during 10 years of the MUMM program, the need to evaluate how the program has been received and its impact on decision making has increased. In 2012, an anonymous survey for physicians was followed by a qualitative interview study (Reference Sihvo3). Next, an attempt to study implementation of the recommendations using national registers and hospital databases was conducted (Reference Ikonen, Rautiainen, Räsänen, Sihvo and Roine4). That proved to be difficult because procedure codes for new methods were lacking and appropriate codes were inadequately used. The study showed that a green light was typically associated with increased use, whereas the methods receiving a yellow light had either positive or negative trends. The very few red lights seemed to inhibit use. The 2012 survey showed that the MUMM program was considered useful by chief physicians of the hospitals but otherwise not well known and commitment to follow the recommendations was poor (Reference Sihvo3). Therefore, in 2014, a systematic follow-up was started to evaluate the success of the implementation of the new recommendations.
OBJECTIVES
To evaluate the awareness and implementation of MUMM recommendations by physicians in secondary care.
METHODS
A targeted web-based survey was sent during November 2014 to November 2015. The eleven questions covered awareness of the MUMM program, use of recommendations, and barriers for implementing these (Table 1). Taking the brief survey required less than five minutes. The survey covered five different recommendations. The target group was chief physicians of the departments relevant to the topic of the recommendation because they are in key roles when new practices and investments are considered. Each survey was sent six months after a MUMM recommendation was published. The link to the survey was distributed by the chief medical officers (members of the MUMM Board). The response rate was counted from the number of physicians the link was sent to. This was arranged by having a member of MUMM staff from THL included in all email correspondence.
Table 1. Questions in the Follow-up Online Survey on Recommendations
RESULTS
Altogether, ninety-four responses were received for the five recommendations (Table 2). The response rate varied between 50 and 70 percent for each recommendation, counted from responses received from each hospital district.
Table 2. Topics of the Recommendations Surveyed
Familiarity with the recommendations varied from 79 percent to 100 percent. By far the most important information channel was the Finnish Medical Journal, followed by superiors at the hospital and the MUMM website. In less than 10 percent of cases the recommendation had been discussed in the clinic meeting.
At least 80 percent of respondents thought that topics were relevant and that recommendations were considered clear, understandable and well prepared (Figure 1). Slightly fewer (71 percent) thought that the recommendations were useful in practice. The opinions about the recommendation on sutureless valve replacement were most critical and approximately a third of these physicians (cardiologist and heart surgeons) did not consider the topic relevant or recommendation useful. Otherwise, no significant variation according to physician specialty, age or position in the organization was observed.
Figure 1. Perceptions about MUMM recommendations (%), n = 94.
The mean of overall assessments of the recommendations, given as a grade ranging from 4 (poor) to 10 (excellent), was 8.4; lowest (8.0) for sutureless valve replacement and highest (8.8) for the gene profiling assay recommendation.
When asked in a separate question about the use of the recommendation in their unit, only approximately a third had actually used it for decision-making. Even fewer (25 percent) had used the recommendation on neuromonitoring of recurrent laryngeal nerve during surgery, and none had applied the sutureless valve replacement recommendation (Table 3). However, many stated that although the recommendation had not been used yet it could possibly be useful later.
Table 3. Usefulness of Recommendations for Decision Making in the Units (%)
a “Cannot say” answers not shown.
b FeNO = fractional exhaled nitric oxide.
In open responses, physicians stated that recommendations were useful when considering whether a method or device should be used. Chief physicians could also use these recommendations as an argument for negative purchase decisions.
DISCUSSION
The MUMM recommendations were well known among potential users, and familiarity with them had somewhat improved from 2012 (Reference Sihvo3). There was not much variation in the perceptions about recommendations among different specialties.
From the point of view of efficient change of practice, this kind of “semi-mandatory” structural implementation of recommendations would be desirable. However, though the recommendations were considered relevant, they were not systematically used. This could be partly due to the fact that the survey was conducted only 6 months after the recommendation was given and change may not be visible so quickly. Respondents commented that recommendations might be applied later. The right timing of the recommendation is important as it can influence uptake (Reference Dobbins, Hanna and Ciliska5).
In optimal practice, MUMM recommendations should be introduced in clinic meetings at the departments but this was seldom done. This suggests that their position is not as strong as the role of national clinical practice guidelines which are generally well followed in secondary care (Reference Karma, Roine, Simonen and Isolahti6).
In the earlier interview study (Reference Sihvo3) on MUMM implementation, suggestions to increase use of recommendations were in two categories: changes in the hospital organization and better dissemination of information. Many interviewees stated that the application of new knowledge is related to the culture in the organization. As chief physicians are responsible for organizational work processes, it is up to them to ensure the recommendations are acted upon. Furthermore, leaders should consistently monitor that recommendations are acknowledged and used. Physicians would appreciate targeted, personalized messages as also found in other studies (Reference Dobbins, Hanna and Ciliska5;Reference Perrier, Mrklas, Lavis and Straus7). However, due to lack of resources this is not always possible. Closer collaboration with medical specialists’ associations should be considered.
New means of knowledge transfer and exchange need to be applied in practice. It is difficult to improve adherence to recommendations if the management and organizational culture do not support the use of HTA knowledge as an essential part of decision processes. If the HTA process and information are totally separated from budgeting and purchasing, there is a risk that HTA fails to exert any impact on real life decision making. Therefore in countries like Finland, where HTA information is not mandatory for reimbursement decisions, the use of HTA information should nevertheless have a defined role that cannot be overlooked intentionally or unintentionally in the decision-making process. In the simplest model, a basic literature search and analysis on the effectiveness and cost impacts should be included the in the investment or procurement procedures before tendering and purchase decisions whenever equipment, instruments or implantable devices are considered.
The weakness of using brief surveys to study implementation is the superficial nature of results and reasons for low uptake remain uncertain. Understanding the mechanisms linked to practical implementation are important and in-depth studies should include interviews and/or detailed surveys. Our experience suggests that using both quantitative and qualitative methods to assess impact of an HTA program is beneficial, giving a more reliable picture of the impact of the program. A variety of methods to assess the impact of HTA has also been used in other countries, such as Austria (Reference Zechmeister and Schumacher8). Short-form surveys can be used also as part of the awareness process to reach key professionals to increase the implementation of recommendations.
When finalizing this article, the national HTA unit in Finland (Finohta) was closed. It is unknown at this time how HTA activities will continue, including the MUMM program.
CONCLUSIONS
MUMM recommendations were considered relevant, although they seldom changed practices during short follow-up period. Stronger implementation efforts in target hospitals would be useful. In Finland, the role of HTA supporting decision making needs to be strengthened by combining HTA information with budgeting and procurement procedures.
Brief web-based surveys are an easy way to gather evidence on uptake of HTA recommendations and to increase awareness. To understand impact mechanisms fully, more detailed surveys or interviews would be needed.
CONFLICTS OF INTEREST
The authors have nothing to disclose.
