Published online by Cambridge University Press: 28 May 2004
Objectives: The aim of the study reported here was to investigate whether the use of magnetic resonance imaging (MRI) impacts on the clinical management of patients presenting with chronic knee problems, reduces costs, and improves patient outcome.
Methods: A single-center randomized controlled trial was conducted. Patients attending with knee problems in whom surgery was being considered were randomized either to investigation using an MRI scan or to investigation using arthroscopy. The study investigated benefits in terms of avoidance of surgery and patient health-related quality of life (using SF-36 and EQ-5D). Costs were assessed from the perspectives of the National Health Service and patients. All analyses were by intention to treat.
Results: The trial recruited 118 patients. No statistically significant differences were found between groups in terms of health outcome. However, the use of MRI was associated with a positive diagnostic/therapeutic impact: a significantly smaller proportion of patients in the MRI group underwent surgery (MRI=0.41, No-MRI=0.71; p value=.001). There was a similar mean overall cost for both groups.
Conclusions: The use of MRI in patients with chronic knee problems, in whom surgery was being considered, did not increase costs overall, was not associated with worse outcomes, and avoided surgery in a significant proportion of patients.
Clinical history and findings at physical examination in patients with abnormalities of the knee are known to be nonspecific in the determination of the cause of the abnormality (29). In patients in whom the diagnosis is uncertain, physicians must, therefore, turn to other diagnostic modalities to inform the selection of appropriate treatments. Although diagnostic arthroscopy is an invasive and relatively high-cost procedure, proponents point to its accuracy and to the surgeon's ability to diagnose and treat abnormalities with a single intervention. However, diagnostic arthroscopy sometimes reveals no abnormality or only minor lesions. Thus, orthopedic surgeons are increasingly turning to magnetic resonance imaging (MRI) as a noninvasive means of diagnosing knee problems. Whereas MRI appears sensitive for the diagnosis of medial meniscal tears and injuries to the anterior cruciate ligament (12;14), it has lower sensitivity for injuries to the lateral meniscus, medial collateral ligament, patella retinaculum, and articular cartilage (23). Furthermore, the specificity and sensitivity of MRI decreases as the number of injured structures within the knee increases (28) and in the presence of a hemarthrosis (23).
The use of MRI in orthopedics has not been extensively evaluated, and the work that has been done has tended to focus on the narrow issue of accuracy and not explore costs and benefits more widely (3;9;10;14;16;20,22;26;27). Warwick et al. (31) report data on the diagnostic accuracy of MRI for patients presenting with knee injuries from an observational study. Patients on the waiting list for diagnostic knee surgery were offered an MRI scan, and on the basis of the results of the scan, 32% were removed from the list because their injuries did not require surgical repair. MacKenzie et al. (24;25) investigated the effectiveness of MRI of the knee. They observed a single cohort of patients and measured the diagnostic impact of MRI in terms of changes in diagnosis and diagnostic confidence, as judged by clinicians before and after the imaging examination. MRI was found to have a “diagnostic impact,” in some cases, by refuting certain clinical diagnoses and in others by improving clinician confidence in diagnosis. Additionally, the use of MRI helped to bring about new, previously unsuspected diagnoses, in 21% of patients and was associated with a marked shift away from the use of surgery. Hollingworth et al. (18) reported an investigation of changes in patient quality of life after magnetic resonance imaging of the knee. They showed statistically significant improvements in quality of life at six months, in general, although the patients remained at levels below the general population norms. The effect of MRI availability on the management process and the final functional outcome after a chronic knee injury has rarely been evaluated (17). In addition, no study has used data from a randomized controlled trial; the majority of studies are based on case-series; therefore, the results have to be interpreted cautiously. It, therefore, remains an open question whether the use of MRI for knee investigations represents a cost-effective diagnostic procedure.
This study describes an empirical investigation of the use of a diagnostic imaging technology, MRI, in the diagnosis of knee abnormalities and injuries in a district general hospital (DGH) setting. The principal purpose of this research was to determine whether, for patients presenting in a DGH with a persisting knee problem, and in whom surgery is being considered, MRI has a major impact on clinical management, health sector and patient costs, and patient outcome.
The research was based on a single center randomized controlled trial conducted at Kent & Canterbury Hospital (K&C). Research Ethics Committee approval was obtained before the commencement of the study. All study patients were recruited from the routine orthopaedic clinics, which they attended after referral either from their general practitioner (GP) or from the accident and emergency department. The aim was for trial patients to be representative of the range of knee problems seen in orthopedic outpatient clinics at a DGH. Eligibility for inclusion in the trial was assessed for all patients with a persisting knee problem who attended the clinics run by participating general orthopedic surgeons.
Patients were defined as suitable for the trial if:
The sample size calculation was based on rate of surgery and assumed that without MRI the vast majority (that is, 90%) of patients presenting with a knee injury would be investigated (or directly treated) using arthroscopy. On the basis of published observational data (4), there was reason to believe that arthroscopy might be avoided in up to 28% of cases by the use of a preliminary MRI investigation. For such a difference between groups (that is, 90% without MRI against 64.8% with MRI) to be established as statistically significant (power=80%; p value <.05), the trial required a total sample of approximately 100 patients allocated evenly between arms.
Once written consent was obtained, study patients were allocated to one of two trial arms:
The on-site study researcher, by using randomly ordered opaque sealed envelopes, undertook the patient allocation process. Patients allocated to the MRI trial arm were booked for an MRI scan (median wait for scan: 29 days) and placed immediately on the arthroscopy waiting list, even though they may not have required surgical treatment. This management pathway was adopted on the advice of the Hospital Research Ethics Committee to ensure that patients were not “disadvantaged” as a result of participating in the trial. Patients were reviewed in an outpatient clinic after their scan and a decision on appropriate management was made. Routine clinical follow-up then continued until their knee recovered. Patients allocated to the No-MRI trial arm were immediately listed for arthroscopy, reviewed in clinic, both before and after surgery, and followed-up according to routine clinical practice until the knee problem resolved. It was clearly neither feasible nor sensible to blind the study patients, researchers, or those involved in providing care to the outcome of the allocation process and so an “open-label” policy was adopted.
By using the categorization of Fineberg et al.; this study investigated the benefits of knee MRI at two levels: diagnostic/therapeutic impact and patient outcome (13). The use of arthroscopy as a form of diagnosis as well as a therapeutic intervention complicates the picture for knee investigations and prevents a complete separation in this study of diagnostic and therapeutic impacts. The observed data on whether or not surgery was undertaken, over the twelve-month follow-up period, allow an assessment of the diagnostic and therapeutic impact of MRI. From a patient's perspective, diagnostic or therapeutic arthroscopies are virtually identical: both are typically day-case procedures involving similar recovery periods.
Benefits in terms of patient outcome were measured by using two generic health status measurement instruments: the SF-36 (5;15;21;30) and the EuroQol EQ-5D (7;8;19). Assessments of quality of life were taken at baseline (at the point of recruitment) and at six and twelve months after recruitment into the trial. The baseline questionnaire and the two follow-up questionnaires (at six and twelve months) were all distributed by post- with pre-paid return envelopes. Nonresponders were sent two reminder letters with a further copy of the questionnaire.
The perspective for the cost analysis reported here is the Health Service and patients, such that the main objective was to identify all important National Health Service (NHS) and patient resource use and cost differences between trial arms. The first step in the costing process was to collect data on the NHS resources devoted to trial patients. Data were collected on the following resource parameters: the MRI scan; surgery on the knee; other relevant diagnostic procedures; other relevant therapeutic procedures; relevant drugs prescribed; associated outpatient attendances; associated inpatient episodes; and relevant contacts with other health-care professionals, such as physiotherapists and GPs. Most of these data were extracted from the patient medical records up to twelve months after recruitment. Data on the use of community health-care resources were collected by using the patient follow-up questionnaires distributed at six and twelve months. The health-care resource use items were costed using unit cost information from three sources (see Table 1).
The costs incurred by patients in attending for an MRI scan, outpatient visit, or inpatient/day-case procedure were also investigated through a survey of a subsample of all trial patients. Total patient costs were calculated by summing costs associated with travel, time, and other expenses. Unit costs were obtained from a variety of published sources.
All analyses were conducted on the basis of “intention to treat.” For the between-group comparisons of baseline patient characteristics and baseline quality of life, for continuous data, the analyses involved either t-tests or Mann-Whitney Wilcoxon tests, and for the comparisons of proportions, a π2 test was used (1).
At each follow-up time point (that is, at both six and twelve months), between-group comparisons of quality of life scores were made using the same methods as adopted in the baseline comparison. However, because this approach takes no account of the longitudinal nature of the data, mean changes in quality of life scores were estimated separately for the zero- to six-month change, the zero- to twelve-month change, and the six- to twelve-month change, using a “complete case analysis.” To allow for potential differences between groups at baseline, quality of life scores were analyzed using regression analyses. A two-limit tobit model with random effects and sample selectivity was developed (technical details of this new technique are available from the authors). Models were estimated for the six SF-36 dimensions for which data can be treated as being continuous (that is, physical functioning, bodily pain, general health, vitality, social functioning, mental health) and the EQ-5D tariff and VAS scores (2). The independent variables included in the models were trial arm, hospital site, referral source, patient sex, and patient age at baseline.
The analysis of cost data initially involved exploring the distributional form of the data separately for the two arms of the trial, to identify the extent of any skew that might present problems for standard parametric statistical tests (6). Second, because skewed distributions for cost data are commonly found, and were expected a priori in this study, a nonparametric approach to the cost analysis was likely to be required. However, estimates of the mean cost are appropriate for use in economic evaluation, even where the data are skewed, because we are interested in both the average per patient cost of a particular treatment and (because there exists a budget constraint) the total cost of care for a patient group. Thus, the nonparametric approach of bootstrapping was used to estimate confidence limits around the estimates of mean cost (6;11).
Figure 1 shows the trial profile, indicating the number of patients recruited into the study, the randomization assignment, the numbers receiving an MRI scan, and the number of measurements for each randomized group.
Flow chart describing progress of patients through the trial. MRI, magnetic resonance imaging.
A total of 118 patients consented to take part in the trial (59 allocated to each arm). At baseline, the two groups were well matched in terms of the following:
The overall response rate to the baseline quality of life questionnaire was 91.5%. For the baseline comparison of quality of life scores, no statistically significant differences between groups were found for any of the dimensions.
A total of seventy-one operative procedures were undertaken over the course of the twelve-month follow-up period, on sixty-six study patients (55.9% of all trial patients). There was a statistically significant difference between groups in terms of the proportion of patients who received surgery (No-MRI arm: 0.71; MRI arm: 0.41; pvalue=.001). The majority of operative procedures were arthroscopies (90.1% of all procedures), most of which were undertaken on a day-case basis. One patient in each trial arm had two arthroscopies in the twelve-month follow-up period. The other operative procedure undertaken in the twelve-month follow-up period on some study patients was acute repair of the anterior cruciate ligament (ACL). The use of this procedure was evenly distributed between trial arms: in the MRI arm, two patients each had a single ACL repair procedure; and in the No-MRI arm, one patient underwent two ACL procedures in the twelve-month follow-up period.
For those patients who received surgery, the time interval from randomization to surgery was not significantly different between the two groups (No-MRI: mean=152 days; SD=106; MRI group: mean=173 days; SD=119; p value=.47). This result is an artefact resulting from the requirement of the Hospital's Ethics Committee that patients allocated to the MRI arm were put on the arthroscopy waiting list at randomization in order for them not to be disadvantaged as a result of participating in the trial. Thus, it would be inappropriate to assume that this specific result would be seen in another setting, unless a similar policy of listing for surgery was in operation. The mean (SD) time from randomization to receiving an MRI scan was 42 (35) days, with a range from 7 to 141 days. A total of nine patients in the MRI arm did not attend for their MRIk scan.
The overall response rates to the follow-up quality of life questionnaires were 67.8% at six months (overall: 80/118; MRI group: 44/59; No-MRI group: 36/59), and 58.5% at twelve months (overall: 69/118; MRI group: 40/59; No-MRI group: 29/59). For the twelve-month round, the higher response from the MRI group was statistically significant (difference between proportions: 0.186; 95% confidence interval: 0.012 to 0.361).
The extent to which bias has been introduced through nonresponse was investigated for the follow-up data by comparing the baseline characteristics of responders and nonresponders. At six months, the two groups were generally well matched, but were less well matched at twelve months where nonresponders were significantly younger (p value=.03) and had their knee problem for a significantly shorter period of time (p value=.03).
The comparison of quality of life scores at six months revealed no statistically significant differences between groups for any of the SF-36 dimensions at six months after recruitment (t-test minimum pvalue=.09 for all dimensions). These data suggest that, for many patients, in both groups, problems continued in terms of bodily pain, performing daily work, or other activities (role-physical) and vitality. Similarly, both the EQ-5D tariff and VAS scores were not significantly different between groups at six months (t-test pvalue=.13 and pvalue=.88, respectively).
The pattern in the quality of life data seen at six months was repeated in the follow-up data at twelve months. Data on means and confidence intervals indicate a trend of between-group differences: the mean scores for the No-MRI group are higher across all parameters.
Figure 2 shows the change in quality of life scores (both SF-36 and EQ-5D) from zero to twelve months. By definition, this figure only reports data for study patients who returned both the baseline and the relevant follow-up questionnaire. A similar picture emerged of a trend at twelve months follow-up in favor of the No-MRI group (see Figure 2). In the No-MRI trial arm, a larger positive change (or smaller negative change) in quality of life scores was seen for all SF-36 dimensions and for the two EuroQol dimensions.
Change from zero to twelve months, SF-36 and EuroQol data (mean scores, 95% confidence intervals). MRI, magnetic resonance imaging; PF, physical functioning; RP, role-physical; BP, bodily pain; GH, general health; VT, vitality; SF, social functioning; RE, role-emotional; MH, mental health; EQ-5D therm, EuroQol EQ-5D thermometer.
The results of the two-limit tobit models, with random effects and sample selectivity, confirm the success of randomization in that, at baseline, there were no statistically significant differences between trial arms in any of the quality of life dimensions. The sample selection models identify characteristics that predict participation in both the six- and twelve-month follow-up surveys. In general, response appears to have been associated with patient age (that is, younger patients were less likely to respond) and trial arm (that is, patients allocated to “No-MRI” were less likely to respond). The main models estimated using data from the six- and twelve-month surveys, and adjusted for sample selectivity, show no strong evidence of differences between trial arms. The dummy variable for trial arm failed to reach conventional levels of statistical significance in all models. However, the trial arm variable had a negative coefficient in virtually all follow-up models (at both six and twelve months). This finding is consistent with a trend toward higher quality of life scores for patients in the No-MRI group, as discovered in the earlier analyses.
Summary information on health service resource use and costs, broken down by trial arm, is presented in Table 2. A similar pattern of resource use was found in the two groups in terms of outpatient attendances, drug costs (excluding drugs used in surgery), physiotherapy sessions, and GP visits. Other than MRI, very few further investigations were undertaken on study patients. As indicated above, the difference between trial arms in terms of the proportion of patients who received surgery was statistically significant.
Table 2 also reports estimates of the NHS cost per patient. The frequency distributions for Health Service costs by trial arm revealed very skewed distributions: in both groups, some patients are associated with a relatively low cost and some are associated with a relatively high cost, in part depending on whether they received surgery. No statistically significant difference was found between mean costs (that is, the bootstrap 95% confidence interval for the difference between groups crossed zero, as shown in Table 2).
The results for patient costs and for both NHS and patient costs combined are given in Table 2. The mean patient cost was a little higher in the MRI group, given that virtually all patients in that group had the additional travel and time costs associated with attending for the MRI scan. When all costs are considered (both NHS and patient costs) the results mirror those for NHS costs only: no statistically significant difference between means overall.
The principal finding of this research is that the use of MRI in patients presenting with chronic knee problems has a positive diagnostic/therapeutic impact in terms of reducing the risk of surgery. This research, therefore, provides some support for the findings of previous research where observational research designs have been used (14–16). The reduced need for surgery represents an important benefit to patients, given that surgery involves inconvenience (that is, 1 day, or sometimes longer, in hospital and typically several further days until full recovery), and risks associated both with the use of anesthetics and the surgery itself. Whereas such risks are low in frequency and, hence, were not observed in the trial cohort, they can be very serious in terms of both morbidity and mortality. The additional cost associated with providing MRI to all patients is offset in full through the avoided costs of surgery in some patients, making the policy cost-neutral. Other benefits associated with MRI in this patient population, such as improved outcomes or enhanced health-related quality of life, were not observed.
One of the disappointing features of the data collected as part of this trial was the response rate to the follow-up questionnaires at six and, especially, twelve months. One possible explanation for this relates to the age group of the sample. First, it is well known that questionnaire response is related to age, with younger people generally being poorer responders, and second, younger people are more likely to be mobile; therefore, some study patients may have changed residence during the course of the study. Of interest, the nonresponse at twelve months was higher in the No-MRI group. One interpretation of this finding is disappointment at not having been allocated to the MRI group. At the time of recruitment, all patients were told about the potential benefits of MRI (although it was stressed that the benefits were uncertain and might not be realized for all patients), but those allocated to the No-MRI group were then denied the MRI alternative and as a result may have been less inclined to respond to the questionnaire. The extent to which the poor response introduced bias was investigated: in general, nonresponders at twelve months tended to be those with lower (that is, poorer) quality of life scores at baseline. Therefore, because responders represented a group with better quality of life, the lower response in the No-MRI group would suggest a better quality of life picture, on average, for those that did respond. This is the finding that emerged from the trial and whilst one of the analytical methods used (that is, the two-limit censored regression models with random effects and sample selectivity) adjusted for nonresponse, some caution is required in the interpretation of the quality of life results.
Another limitation of the study concerns the follow-up period of twelve months. The consequence is that, for all patients, data are censored at that point, if they have not been censored earlier. This finding may have had a particularly important impact on some of the key resource use parameters, in particular the use of arthroscopy. The time from randomization to surgery among all study patients ranged from 6 to 352 days. That some study patients underwent surgery toward the end of the twelve-month follow-up period suggests that a longer follow-up period would almost certainly have seen a larger number of patients receiving surgery. However, given the necessity of trial design that all patients were listed for surgery, regardless of the trial arm to which they were allocated, it is not certain that a longer follow-up would necessarily have seen a narrowing of the gap between arms in the proportions receiving surgery.
The evidence presented in this study lends support to the conclusion that the use of MRI in patients presenting at DGHs with chronic knee problems, in whom arthroscopy is being considered, (a) does not increase NHS costs, (b) is not associated with significantly worse outcomes, and (c) avoids surgery in a significant proportion of patients. However, the link between diagnostic processes and changes in health outcome is indirect; therefore, the finding of no MRI-related effect on health outcome may be a consequence of the limited power available in this trial. Further research to confirm (or contradict) the findings of the trial data would be valuable.
We thank all those who advised and supported the research team at various stages throughout the project, particularly Martin Buxton, Ian Russell, Cindy Billingham, and Christian Salas. We are also indebted to all those who worked so hard on the ground and to Sue Cashmore who prepared the final manuscript. Finally, thanks go to all the patients and hospital staff who generously gave their time to help with this research. This project was funded by a grant from the UK NHS Health Technology Assessment R&D Programme.
Unit Cost Estimates
Flow chart describing progress of patients through the trial. MRI, magnetic resonance imaging.
Change from zero to twelve months, SF-36 and EuroQol data (mean scores, 95% confidence intervals). MRI, magnetic resonance imaging; PF, physical functioning; RP, role-physical; BP, bodily pain; GH, general health; VT, vitality; SF, social functioning; RE, role-emotional; MH, mental health; EQ-5D therm, EuroQol EQ-5D thermometer.
Health Service Resource Use and Costs (£) over 12 Months (Base Case)