The study by Hutton, Trueman, and Henshall provides a thoughtful and helpful set of observations about the potential benefits of linking reimbursement to requirements for further clinical research (coverage with evidence development—CED), as well as the likely challenges and obstacles to implementation. In this commentary, we will expand upon several of the key points made in their study and offer some additional suggestions for moving this policy discussion forward.
CED AS AN EVIDENCE-BASED MEDICINE TOOL
An important motivation for CED, in addition to the desire to make promising technologies rapidly available, is that the traditional hierarchy of evidence-based medicine (EBM), which has been widely adopted in health technology assessment (HTA) and coverage decision making, can impose expensive, lengthy evidence requirements to demonstrate clinical effectiveness. These rules of evidence, while fully defensible from a methodological perspective, may in some cases be inconsistent with the pace at which technologies are developed, modified, and abandoned. For example, a prospective validation of the clinical utility of cancer biomarkers might require years of follow-up, and such studies are unlikely to be affordable or feasible for small venture-backed companies developing these diagnostic tests. This temporal mismatch between the pace of technological evolution and the pace of evidence development creates a potential niche for CED.
In effect, CED can be seen as a means of implementing EBM in a real-world setting. Policy makers are often expected to make coverage decisions based on the “best available” evidence, which can, at times, be inadequate. By having a “yes” or “no” decision as the only options, promising technologies may be rejected or ineffective (or unsafe) ones adopted, depending more on political and other pressures than evidence. This finding can perpetuate the problems of scientific uncertainty, underuse and overuse of services, and failure to resolve uncertainty through further evidence generation.
POSITIONING CED IN A TECHNOLOGY'S LIFE CYCLE
CED might best be viewed as a policy mechanism that can help in those circumstances where generating reliable evidence faces various types of predictable challenges. For emerging and newly approved technologies, the challenge is that the standards of conventional EBM may require substantial additional time and expense to meet, even in circumstances where the initial evidence suggests potentially important advantages over existing technologies. The most efficient approach to evaluating such technologies without undesirable delay, from the patients and health system perspective, may be through CED.
The CED approach may also be useful for existing technologies, in at least two different contexts. First, in the case of those technologies that have been adopted with some enthusiasm, but for which reliable evidence on risks and benefits has never been generated, particularly when evidence begins to accumulate that raises questions about their net health impact. High-dose chemotherapy with bone marrow transplant for metastatic breast cancer is an example of this use of CED. Randomized trials of this technology were conducted through a CED approach as a result of increasing doubts about the benefits and risks of the procedure. Second, certain technologies may be available but underused, because there is limited market incentive to conduct the necessary studies. This may be true for preventive, health maintenance, or public health interventions that have no natural commercial sponsor, and these technologies may also be good candidates for CED. We believe CED can be a powerful tool for reducing uncertainty and better targeting new and established technologies and clinical practices at different stages throughout their life cycle. Restricting it to experimental and new technologies may not be taking full advantage of this policy's potential.
INFORMATION GENERATED THROUGH CED
The study by Hutton et al. notes that regulatory trials will often provide information limited to intermediate outcomes and with short follow-up periods. There are other important gaps in evidence that commonly remain after the completion of regulatory trials, which might also be addressed by studies supported through CED. Such gaps include information about risks, benefits, and costs in real-world setting (in the hands of typical clinicians as applied to the broad range of patients encountered outside the usual investigational context). Furthermore, questions of comparative effectiveness and value are generally not addressed in regulatory trials, nor are the risks and benefits of combination therapy with existing technologies. CED studies may provide valuable information on risks in large populations, offering substantially more information on product safety, particularly in those circumstances when a safety signal of uncertain significance has been identified. Finally, CED may provide an opportunity to explore subgroups of patients for whom benefits and risks are larger or smaller than the average effect identified in regulatory studies.
The study by Hutton et al. seems to assume that CED will generally be used in conjunction with decision making that involves cost-effectiveness models. However, the CED approach may also be applied in decision-making processes that do not formally incorporate costs or formal decision analysis. Judgments about the value of information and the implications of making a coverage decision with insufficient evidence could be made in a qualitative manner.
TIMELY EVIDENCE DEVELOPMENT
The study by Hutton et al. suggests that CED may only be appropriate for studies that can be completed in 3 years or less, and it may be true that this policy approach will have the greatest initial value in those circumstances described in the study (i.e., experimental technologies). However, some of the earliest US experience with CED involved trials of considerably longer duration. The National Emphysema Treatment Trial in the United States was completed with funding from the National Institutes of Health under a CED arrangement with Medicare, and this study took approximately 7 years to complete. Once the results were published showing significant harms and limited benefits for most patients, the procedure was almost completely abandoned, even though the Medicare coverage policy would have allowed coverage for all patients shown to have any functional improvement from lung volume reduction surgery. As mentioned above, the first major US-based CED application was for studies of high-dose chemotherapy and bone marrow transplant in patients with metastatic breast cancer, again requiring a multiyear randomized clinical trial (RCT). This study clearly showed that the procedure was harmful to patients, and the practice, has since, been abandoned. When patient access is not restricted, for example, when quality of life or safety information is collected in the context of a registry or a prospective cohort study of all eligible patients, longer time lines may not necessarily be an obstacle to implementing CED.
DECISION-BASED EVIDENCE MAKING
A critically important effect of CED mentioned in the study by Hutton et al. is that it provides the decision makers—payers, clinicians, and patients—with an opportunity to determine clinical research priorities, and to ensure that trials are designed to answer their questions. Traditional clinical research is often not designed explicitly to inform a decision, which increases the chance that the results will not be helpful in decision making. CED allows payers to use their reimbursement authority to determine what questions are studied, and critical elements of study design, such as the nature of the study population, comparison groups, study setting, outcomes measures, and so on. Such studies are generally referred to as pragmatic research, which has the distinguishing features of being designed to inform a decision. It is contrasted with explanatory research, which is intended to provide a deeper understanding of a condition or treatment, but not necessarily to assist in learning how it would best be managed.
STUDY TYPES UNDER CED
Deciding on appropriate and adequate study methods and designs under CED will be challenging, as the policy is trying to balance rapid access to technology with creation of evidence, and different stakeholders have different views of how this balance should be handled for each technology. Observational methods, perhaps using data from claims or electronic health records, have the appeal of providing broader access more quickly, but are inherently less analytically valid. The Medicare registry of implantable cardioverter-defibrillators (ICD) still contains only baseline data, without any firing data, and, therefore, is only able to look at questions such as whether patients are appropriate ICD candidates, how similar they are to trial patients, and rates of clinical complications that can be assessed by linkage to claims data. Funding has recently been secured to include firing data in this registry, which should allow analysis of patient characteristics that might predict which patients are most in need of the device.
When the uncertainty is around the effectiveness of a technology, an RCT or a pragmatic (practical) clinical trial (PCT) may be the most appropriate design. PCTs in particular, can provide reliable evidence more relevant to real-world effectiveness and may be completed more quickly and cheaply than traditional RCTs. However, these studies can also raise more complex policy and operational issues. Medicare has approved coverage for positron emission tomography scanning in patients with suspected dementia in the context of a pragmatic PCT, but such a study has not yet been initiated after more than 3 years of effort, primarily because a funding source for the research costs has not yet been identified. Furthermore, when an RCT/PCT is required, patient access may be restricted by the payer, depending on the degree of reversibility of wider technology adoption and the feasibility of a trial in such circumstances. However, the deciding factor here should not be accessibility but suitability of a study design to answering the decision makers’ question.
The practical challenge, maybe more marked with the RCT/PCT model, then becomes creating the operational infrastructure to support such studies being designed, approved by research ethics boards, funded, implemented, and analyzed in a more timely and efficient manner than is now true for traditional RCTs. Participation in clinical research has to become a routine, rather than exceptional, management option for patients and clinicians under conditions of uncertainty.
Arriving at the right study question and methodology will require a process and analytic methods with comparable sophistication to the current work of the appraisal committees that make judgments about coverage and guidance. As noted in the study by Hutton et al., this will require effective mechanisms to improve communication about priority setting, methods, and protocol development among experts and stakeholders, including patients, clinicians, payers, product developers, and researchers.
In the United States, the Center for Medical Technology Policy (www.cmtpnet.org) has convened several workgroups that include the full range of experts and stakeholders and is pilot testing a collaborative approach to developing CED study protocols on selected emerging technologies (cardiac computed tomography angiography, radiation therapy for prostate cancer, and gene expression profiling tests for breast cancer). In Europe, EUnetHTA has a dedicated work stream on the assessment of new emerging technologies with the French partners actively exploring the concept of CED. Furthermore, in the United Kingdom, the recently published Cooksey Review of health research recommended the funding of “only in research” (the CED equivalent) recommendations made by the National Institute for Health and Clinical Excellence by the National Health Service R&D. Such recommendations have already been implemented in the UK context, including the MRC-funded CLASICC trial of laparoscopic surgery for colorectal cancer and the prospective cohort study of photodynamic therapy for age-related macular degeneration.
CRITERIA FOR CED
The stated criteria suggest that CED might best be used for technologies with potential significant improvements in health outcomes, and uncertainty about effectiveness or cost-effectiveness. This is similar to the criteria proposed in the CMS guidance documents of April 2005 and July 2006, which noted that the CED approach would generally be applied to potentially important technologies for which evidence is currently not adequate to determine impact on health outcomes.
Of interest, in a recent meeting, the NICE Citizens’ Council noted the potential of “only in research” as the ‘norm’ for providing new technologies until most uncertainty surrounding their effects is resolved. This strategy may be impractical in the current setting, however, one could imagine CED as being routinely applied to products once they have achieved regulatory approval, and show some evidence of clinical advantages to current technology. Perhaps all such technologies could be subjected to CED, once there was sufficient infrastructure in place to conduct additional studies affordably and efficiently. This highlights that the potential role of CED in coverage policy depends on building the conceptual and operational infrastructure for rapid and efficient practice-based research.
CRITICAL CHALLENGES
CED may in fact provide a mechanism to expedite access to promising technologies, but the conditional limitation to those patients enrolled in a study may impose serious restrictions, depending on the size of the study, how quickly it can be launched, and how soon the study can provide data that will inform a decision. Adequate resources to support high-priority CED studies will be essential so that necessary studies can be designed and implemented without delay. The policy option has no meaningful impact if it takes a year or more to design, identify funding for, and implement the study. Therefore, there would ideally be a funding pool maintained to support these studies, as suggested by Cooksey in the United Kingdom and in recent legislative proposals in the United States to provide funding for comparative effectiveness research. Rules to determine the contribution of product developers to clinical and research costs will need to be developed, as it does not seem reasonable that all costs for these studies should be borne by the healthcare system or public and private payers.
The study notes that CED studies should be managed by, or on behalf of, decision-making bodies. It would seem that the details of study design and management involve technical and operational expertise that may not always be sufficiently available within decision-making organizations. Therefore, one could imagine that much of the work of CED studies might be assigned to new or existing research organizations that would work collaboratively with the decision makers.
We strongly agree with Hutton et al. about the value of product developers and decision makers having greater dialogue and consensus on evidence standards for specific types of technologies. The need for CED could be substantially reduced if there were clear guidance documents that articulate the evidence requirements for coverage, and decision analysis can help formulate such rules.
An important implication of the study by Hutton et al. is the need for a concrete plan of action to address some of the critical scientific, policy, methodological, financing, operational infrastructure. We appreciate the excellent work of the HTAi policy forum in framing the critical issues and questions raised by CED. They have provided an extremely valuable framework for continuing discussions on this subject. With that in hand, we believe an important element to successful implementation will be moving quickly from theory to practice by working collaborative with all key stakeholders, including patients and the public, to design and implement CED studies. Launching pilots, ideally with multinational participation, will identify and help address key obstacles to implementing this option in different settings. Such initiatives can then serve as the platform for refining and improving the applicability and acceptability of CED as a viable policy option.
CONTACT INFORMATION
Sean R. Tunis, MD, MSc (sean.tunis@netzero.net), Director, Center for Medical Technology Policy, 524 2nd Street, San Francisco, California 94107
Kalipso Chalkidou, MD, PhD (Kalipso.Chalkidou@nice.org.uk), Associate Director, Research and Development, National Institute for Health and Clinical Excellence (NICE), 71 High Holborn, London WC1V 6NA, UK; Harkness Fellow in Health Policy, Department of Health Policy and Management, Johns Hopkins School of Public Health, 624 N. Broadway, Baltimore, Maryland 21205–1999; Center for Medical Technology Policy, 524 2nd Street, San Francisco, California 94107
