Published online by Cambridge University Press: 25 October 2005
Objectives: Interventions to support patient self-care of their condition aim to improve patient health and reduce health service costs. Consequently, they have attracted considerable policy interest. There is some evidence of clinical effectiveness but less attention has been paid to whether these interventions are cost-effective. This study examines the quality and quantity of existing evidence of the cost-effectiveness.
Methods: A systematic review was carried out to assess the extent and quality of economic evaluations of self-care support interventions. Thirty-nine economic evaluations were assessed against a quality checklist developed to reflect the special features of these interventions.
Results: The majority of the studies claimed that self-care support interventions were cost-effective or cost saving. The overall quality of economic evaluations was poor because of flaws in study designs, especially a narrow definition of relevant costs and short follow-up periods.
Conclusions: The current evidence base does not support any general conclusion that self-care support interventions are cost-effective, but ongoing trials may provide clearer evidence.
Self-care support interventions are defined as those interventions that enhance patients' ability to make decisions intended to alter the effect of their conditions on their health, by means of their responses to symptoms, or monitoring their condition, or self-treatment. As a result of this broad definition, the term “self-care support interventions” covers a wide range of other commonly used terms, including self-management and self-help.
There is considerable policy interest in interventions to support patients' self-care, to improve patient health, to empower patients, and to release the time of health professionals for other activities (11). In the United Kingdom, there is a national movement toward supporting patients' self-care of chronic conditions (10). The Expert Patients Programme, based on the Chronic Disease Self-management Program intervention developed in the United States (33) aims to introduce lay-led self-management training for patients with chronic conditions. This intervention currently is being rolled out across the National Health Service (NHS), and there is increasing interest in self-care interventions for chronic conditions elsewhere (55).
Most completed evaluations of these interventions have been studies of effectiveness (changes in outcomes) rather than cost-effectiveness. There is no logical reason for privileging either cost or effectiveness data in a choice between interventions. Both are required. Interventions with favorable effects may have high costs. Interventions with worse effects may be worthwhile if they reduce costs and release resources that can be used to produce higher valued outputs elsewhere. Assessing the cost-effectiveness of interventions requires a full economic evaluation: the intervention must be compared with an appropriate alternative, and all the relevant costs and effects of both interventions should be considered (11). In this study, we assess the published evidence on cost-effectiveness of self-care support interventions and discuss some of the issues arising in economic evaluations of such interventions.
Defining self-care support interventions is problematic (2). We adopted a working definition of self-care support as patients making decisions intended to alter the effect of their conditions on their health, by means of their responses to symptoms, or monitoring their condition, or self-treatment. The definition is broad, and the range of interventions is correspondingly wide, ranging from use of oral anticoagulants, by means of education programs for diabetes patients, to making it easier for patients to self-medicate by relaxing restrictions on the sale of certain types of drugs.
A systematic literature search and review of published economic evaluations of self-care support interventions was carried based on the above definition. Full details are provided elsewhere (20). Eleven specialist databases were searched, including NHS Economic Evaluation Database, Health Economic Evaluations Database, Database of Abstracts of reviews of effects, Health Technology Assessment database, and the National Research Register. Most of these databases were started in 1995, but some contained studies dating from 1993. They were searched without date restrictions to April 2003. In addition, MEDLINE was searched from 1966 to 1994. We also had an initial purposive sample of 27 studies, provided by an expert in the field. The studies identified as economic evaluations are listed in Table 1, whereas the paper selection method is shown in Figure 1. Studies were included if they were considered to be full economic evaluations, that is, the intervention was compared with an appropriate alternative, and all the relevant costs and effects of both interventions were considered (12). Hence, study type was not restricted to randomized controlled trials, although this study design was the most frequently encountered.
A checklist was developed to quality assess the identified economic evaluations. Chiou et al. (6) identified nineteen published guidelines, checklists, and criteria lists for economic evaluations after a systematic search of the English language literature since 1990. We obtained copies of the nineteen lists and adapted them in the light of the special features of self-care support interventions. Our checklist is available elsewhere (20) and draws mainly on Drummond et al. (12), NHS Centre for Reviews and Dissemination (40), and Forbes et al. (15). Quality criteria were included in the checklist if they were either important in assessing the quality of economic evaluations in general (for example, the perspective of the study) or particularly relevant to the evaluation of interventions to support self-care (for instance, the measurement of costs to include patients' out-of-pocket expenditure).
As with the Drummond checklist, there are several subquestions under some of the main questions. For these questions, an assessment (subjective in nature) was made to assess whether the paper met the quality criteria. Such subjective assessments are a feature of most quality assessment checklists in the economic evaluation literature.
Economic evaluations of self-care support interventions tend to be more difficult than evaluations of more conventional interventions.
Assessment of interventions requires a comparator. Unfortunately, the comparator is often less well-defined in self-care support interventions than in other interventions for which the comparator is a placebo or another intervention. The usual comparator in self-care support interventions is no active intervention (54). This strategy has several drawbacks. First, it makes it more difficult to compare interventions against each other and, hence, to choose the best one. Second, patients will in most cases already be practicing some form of self-care that may take a variety of forms with differing resource implications. For example, some individuals may respond to their condition by self-medication or self-treatment, others may take time off work, and others may soldier on with reduced productivity. Evaluators need to consider all such possible resource effects.
When outcomes are heavily affected by patient expectations and beliefs, care must be taken to avoid Hawthorne effects: at least part of any change in outcomes from an intervention would arise for any intervention compared with an alternative of no active intervention. For example, in one study, the control group (which had been called for an interview to explain the trial and to have baseline measurement taken but received no active intervention) had improved health and self-efficacy (22). In more-conventional interventions, it is possible to allow for these effects by administration of placebo therapies to patients who are blinded to their allocation to the control and intervention groups. But, although it is feasible to blind those assessing the effects of an intervention (35), neither placebo controls nor blinding subjects to their allocation are possible in the case of self-care support interventions. Hence, because Hawthorne effects are potentially present, it is better to compare active interventions against each other rather than against a passive usual-care control group.
In addition, studies of group-based self-care support interventions do not allow for the potentially beneficial effect of being a member of a group of individuals with similar conditions.
There is a higher risk of control group contamination. Access to conventional drug or other clinical interventions typically requires the consent of the health professionals running the intervention. Such controlled access is more difficult to ensure with self-care support interventions that are designed to reduce the role of the professionals in the care of the individual patient. For example, with patient education materials, patients in the control group may get access to the materials if they are published or from members of the intervention group. Randomization by center rather than by individual can reduce the risk of contamination.
Trials of interventions to support self-care are more likely to be pragmatic i.e. to take place in a normal health service setting. Given the complexities of health-care systems, more care is required in specifying the likely consequences of the intervention to ensure that they are measured. For example, a self-care intervention for a chronic condition may reduce the demand for general practitioner (GP) consultations by patients with the condition. It is necessary to measure the number of consultations. But a reduced demand for consultations for a set of conditions will have knock-on effects. For example, in a health service such as the English NHS with no copayments, the demand for GP consultations is rationed by waiting time for appointments. A reduced demand for one type of consultation will reduce the waiting time for all types of consultation if the total number is held fixed. Hence, the reduced waiting time is an effect of the intervention that should be measured (4). The specification of such system effects can often be subtle and require the construction of formal theoretical models to guide the collection of relevant data.
Analysis of the effects of the intervention for subsets of the control and intervention populations can be an ex post exercise in data mining. However, if there are subgroups for whom the intervention is particularly beneficial or harmful, then this should be taken into account in the implementation and targeting of policy. There are a number of reasons why there are likely to be policy-relevant differences within the populations exposed to interventions to support self-care. First, the baseline of no active intervention control may differ for different groups in terms of how actively they are already self—managing their conditions and how successful they are. Second, the effect of the intervention may vary with observable characteristics of patients, such as education or age. Third, some self-care support interventions, such as the Expert Patients Programme (10) in the United Kingdom, are designed to assist patients to manage a wide range of chronic conditions, and it is possible that their effect may vary across conditions. Thus, evaluations of self-care support interventions may require fuller socioeconomic data on individuals than is usual in trials of conventional interventions.
Self-care support interventions are likely to have a wider range of outcomes, because they are often intended both to improve patient health and to empower patients by giving them greater control of health-affecting decisions. Hence, evaluations need to include a wider range of outcome measures. For example, the evaluation of the Expert Patients Programme includes measures of self-efficacy to exercise regularly, to manage disease, to manage symptoms, and to manage depression, a measure of communication with health professionals, and subjective well being in several domains, in addition to a battery of more conventional physical and mental health measures (43).
Evaluations should adopt a societal perspective and take account of costs wherever they fall, including on patients. Patient costs, which include time off work, out-of-pocket expenses, and travel are often difficult to measure. Given that self-care support interventions are designed to alter the way patients manage their conditions, patient costs are likely to be more important than with more-conventional interventions.
There is some evidence that beneficial effects of interventions last for quite short periods (41). One study we reviewed reported that a beneficial effect of intensive self-management education for asthma patients apparent at 1 year was not apparent at 2 and 5 years (27). Hence, results from studies with short follow-up periods may not be reliable guides to long-term effects. Many self-care support interventions involve patients with chronic conditions so that long-term consequences are important.
There are fundamental tradeoffs in choosing the length of follow-up in trials of self-care support interventions. Longer follow-up periods may provide information on the time path of effects, but it increases the risk of control group contamination in the absence of any barriers to control group patients adapting the same self-care techniques as the intervention group. Thus, a reduction in the difference in outcomes between control and intervention groups over time may reflect a genuine reduction in the effect of the intervention on the intervention group, or it may reflect an improvement in the condition of members of the control group who adopt the same self-care practices as the intervention group. It may be possible to distinguish these explanations by examining the trend in the levels of outcome of the two groups, in addition to the difference between their trends.
The transferability of results to other settings is crucial (13). In studies of conventional interventions, issues of transferability are most usually raised for cost estimates. Differences in unit costs of resources between countries or over time mean that disaggregated data (volume and unit costs for different types of resource) are more likely to be useful in other settings. Differences in unit costs across settings may suggest that cost minimizing input mixes may differ; therefore, a simple recalculation of costs using the original study volume data but local unit costs can be misleading.
Transferability of outcome effects may be more of an issue for self-care support interventions than other types of intervention. Cultural factors that affect patients' receptiveness to self-care may influence both the no intervention baseline and the effect of an intervention. Thus, it is important that the context of the intervention is clearly specified.
The systematic literature search produced 573 papers. The abstracts were read independently by two members of the team. Sixty-seven full papers were obtained if both reviewers considered them likely to be economic evaluations of self-care support interventions. These papers were then read independently by two members of the team to determine whether they were economic evaluations of self-care support interventions. Thirty-six of these papers were identified as being suitable for review. A similar procedure applied to the twenty-seven papers in the initial purposive sample identified eight economic evaluations, two of which were not found from the literature search. A further study (39) was found from the citations of one of the papers from the literature review, bringing the total number of papers to thirty-nine.
The total of thirty-nine papers considered to be full economic evaluations were separately assessed against the criteria list by two reviewers, with six being assessed by two reviewers. For two (21;27) of the six studies, there was complete agreement between both reviewers. For two other studies (42;46), mean agreement was 95 percent. For the Lorig et al. (34) study, mean agreement was 86 percent, and for the study by Kruger et al. (28), it was 77 percent.
Nineteen of the thirty-nine evaluations of self-care support interventions were US-based. Scandinavian countries (Norway, Finland, Sweden, and Denmark) provided data for seven studies, whereas six of the studies were UK-based.
All the studies were published after 1993, and we think it unlikely that the search missed many earlier studies, because MEDLINE was also searched back to 1966. Older studies are also likely to be less relevant.
The most common form of clinical study used was the randomized controlled trial (n=22).
Of the thirty-nine studies identified, thirteen could be considered cost-effectiveness analyses or cost utility analyses, where costs and consequences of at least two alternatives were formally compared. The remainder were cost consequence analyses where the incremental costs and outcomes were not formally calculated and/or compared.
Self-care support interventions were applied across a range of (mainly chronic) conditions. Asthma (n=10), diabetes (n=6), arthritis (n=3), and heart disease (n=2) were the most commonly specified conditions, whereas “chronic disease” was evaluated in a further four studies. Self-care support interventions in acute conditions (including patient self-testing) were examined in four studies.
Four studies did not report the length of the follow-up period. Of the remainder, 12 months was the most common follow-up period (n=16), with ten studies having a follow-up period in excess of 1 year.
Most of the studies conclude that the self-care support intervention was either cost saving or cost-effective (or both) (n=29). Only two studies showed that the intervention was not cost-effective, and the remainder (n=8) were inconclusive. Although this may appear to broadly promote the concept of the self-care support interventions, the majority of the studies had significant flaws that limit both the internal and external validity. The most common flaws were: (i) Poor costing methodology. Only 26 percent of studies had a societal perspective. The rest defined costs narrowly, for instance only considering the impact on primary care or ignoring patient expenditure. Several had a limited assessment of costs with, for example, unit cost data not presented so that replication of results is not possible. (ii) Inappropriate comparison group. Several studies used a simplistic design comparing costs/outcomes in one period with costs/outcomes in the following period without any adjustment for external factors that may have influenced changes. For example, regression to the mean may be an important factor where costs are high in one period and lower in the next.
Inadequate handling of uncertainty. Few performed sensitivity analyses or presented confidence intervals around mean estimates of cost or effect.
Missing data were either ignored or dealt with inappropriately. None of the studies reviewed handled missing data using recommended techniques (such as multiple imputation).
Short period of follow-up. The majority of studies (n=25) had a follow-up (or time horizon) of 1 year or less.
As the quality of papers was generally poor, and the interventions, the conditions, the setting, and geographical locations were so heterogenous, it was not appropriate to synthesize the data. It is difficult to make any robust inferences from these studies.
The literature around self-care support interventions is wide ranging. Different terminology was used between countries, settings, and over time. We have used the term self-care support interventions but are aware that other terminology has been used previously and that the terminology is likely to change in the future.
Interventions to support patient self-care are very diverse and take place in many types of settings, so drawing general conclusions about the cost-effectiveness of such interventions is problematic. There were clear differences in the results from studies in different countries. The two studies that showed that these interventions were not cost-effective were both UK-based (14;30). Only one UK-based study claimed the self-care support intervention was cost-effective (42), whereas the remaining three UK studies were inconclusive. Despite this finding, the implementation of a self-management program has recently been rolled out across the United Kingdom.
Of the nineteen US-based studies, fifteen claimed the self-care support intervention was cost-effective, only one claimed that the intervention was not cost-effective (44), with the remaining three studies showing inconclusive results (1;19;51). Of the seven studies based in Scandinavia, four demonstrated the self-care support intervention to be cost-effective (7;16;17;29), with the remaining studies being inconclusive (19–21;25–27).
The effectiveness of interventions may also differ between conditions. For example, there were four evaluations of chronic illness (22;32–34), all claiming that the interventions were cost-effective. However, for osteoarthritis of the knee, there were two studies (30,37): one of which showed equivocal results (37), the other demonstrating that the self-care intervention was not cost-effective (30). Similarly, the type of intervention is likely to impact on effectiveness and cost-effectiveness. For example, groups providing training in self-care (32–34) may be very different in their effectiveness and cost-effectiveness to pharmacists providing information for individuals (47).
Resource and unit cost data may not be readily transferable between systems. It is also likely that the outcome effects of these interventions may be culturally dependent. Some interventions concentrate on a very narrow patient population others consider all chronic conditions. It is not surprising that the conclusions of studies based on these diverse populations are different. There may also be more control group contamination in some studies than in others (for example where the intervention is based on a published guidebook, it may be possible for control group patients to access that information).
Most of the studies reviewed were of poor quality, most having one or more major drawbacks. A previous review of evaluations of self-management interventions (2) was also critical of the methodology and of the lack of consideration of the cost-effectiveness of self-management interventions. Bower et al. (5) noted that there were no data on long-term clinical or cost-effectiveness for these interventions and “available evidence is limited in quantity and quality and more rigorous trials are required to provide more reliable estimates of the clinical and cost-effectiveness of these treatments.”
Although the studies reported results that are largely in favor of interventions to support self-care, their generally poor quality limits their usefulness for policy. Costs were often measured from a narrow perspective, follow-up periods were short, and little allowance was made for uncertainty in important cost and outcome parameters.
While the current evidence base is limited, there is much research being conducted in this field in the United Kingdom and in the United States. These ongoing evaluations are likely to improve the knowledge base and provide better data for decision-makers to determine whether self-care support interventions are cost-effective.
Current evidence does not support the hypothesis that these interventions are cost-effective. Thus, there is little evidence from the economic evaluation literature supporting the increased uptake of these interventions. Trials that are ongoing may provide better estimates of the cost-effectiveness of certain interventions across a range of conditions. Cost-effectiveness may vary between condition, setting, and geographical location and results may lack generalizability.
Gerry Richardson, BSc (Econ) (gar2@york.ac.uk), Centre for Health Economics, University of York, Heslington, York YO10 5DD, UK, Hugh Gravelle, PhD (hg8@york.ac.uk), Professor, Centre for Health Economics and National Primary Care Centre, University of York, Heslington, York YO10 5DD, and University of Manchester, Manchester M13 9PL, UK, Helen Weatherly, MSc (gar2@york.ac.uk), Research Fellow, Centre for Health Economics, University of York, Heslington, York YO10 5DD, UK, Gill Ritchie, MSc (gritchie@rcgp.org.uk), National Collaborating Centre for Primary Care, Royal College of General Practitioners, Frazer House, 32-38 Leman Street, London E18EW, UK
We are grateful for help, advice, and suggestions from Peter Bower, Ayesha Dost, and Anne Rogers. We are also grateful to Liz Ingham for her technical assistance. Gerry Richardson has a Researcher Development Award funded by the NHS Research and Development Programme. The study on which the paper is derived was funded by a grant from the Department of Health. The views expressed are not necessarily those of the funders. In addition, the National Primary Care Research and Development Centre receives support from the Department of Health.