Aphasia occurs in one-third of people who survive a stroke (1) and affects all aspects of communication. It is estimated that fifty people per 100,000 of the general population will have a stroke and still suffer from aphasia after 6 months (2). Evidence suggests that, although the majority of recovery may occur in the first 6 months after stroke, improvements in language skills can continue to be made for several years (Reference Kurland, Baldwin and Tauer3–Reference Raymer, Beeson and Holland5). However, continued treatment places high demands on limited health care resources, particularly when therapy requires face-to-face contact with speech and language therapists; hence, in practice such treatment is often restricted. The objective of this study is to investigate the potential cost-effectiveness of self-managed computer therapy combined with usual stimulation (such as participation in normal language stimulation activities and support groups) compared with usual stimulation alone in people with aphasia.
Computer programs developed for the treatment of aphasia provide targeted therapy exercises based on individual needs, focusing on personally relevant vocabulary and patients’ conversational and word finding needs. Word finding is associated with the ability to retrieve the correct word from memory when it is needed; hence, this skill is central to effective communication. Computer therapy provides opportunities to self-manage continued aphasia treatment, and there is evidence to suggest that the software can help to improve communication outcomes in reading, spelling, and expressive language (Reference Katz and Wertz6–Reference van de Sandt-Koenderman8). However, until recently studies of self-administered word finding therapy have been limited to descriptive case series and no cost-effectiveness analyses have been undertaken. In this study, we report a cost-utility analysis undertaken alongside the Cost-effectiveness of Aphasia Computer Treatment Compared to Usual Stimulation (CACTUS) pilot randomized controlled trial.
The clinical results of the CACTUS trial are reported elsewhere (Reference Palmer, Enderby and Cooper9). In brief, the CACTUS trial was a single-blinded, parallel-group, stratified, pilot randomized controlled trial in which thirty-four participants with aphasia were randomized to computer treatment or usual stimulation, in a UK setting. A 5-month intervention period was followed by a 3-month period without intervention to explore whether the treatment effect was maintained. Participants were included in the study if they had a diagnosis of stroke and aphasia with word finding difficulties as one of the predominant features as assessed by the Comprehensive Aphasia Test (CAT) (Reference Swinburn, Porter and Howard10) and the Object and Action Naming Battery (Reference Druks and Masterson11). Potential partcipants were only eligible if they were no longer receiving impairment based speech and language therapy. People with severe visual or cognitive difficulties that reduced their ability to use the intervention were excluded.
The intervention group received independent-use computer therapy in the form of the Step-by-Step program, which contains language exercises designed for people with aphasia (12). The program was configured by a speech and language therapist (S.L.T.) and initial tuition was provided. Participants used their own computer or a loaned laptop computer. Volunteers including speech and language therapy students and existing volunteers from communication support groups contacted participants once per week in the first month and at least once per month thereafter to offer support, assistance, and encouragement. Participants were advised to work through the computer exercises for at least 20 minutes, 3 days a week, for 5 months.
Our analysis is exploratory because it is based upon a small pilot study. It represents an early analysis of the likely cost-effectiveness of self-managed computer therapy for people with aphasia. The aim of the CACTUS trial was to assess the feasibility of conducting a rigorous randomized controlled trial into the effectiveness of self-managed computer therapy compared with usual stimulation rather than to provide robust effectiveness data. Therefore, our analysis cannot be expected to provide conclusive cost-effectiveness results. However, early cost-effectiveness modeling remains of value because it provides insight on the likely cost-effectiveness of the intervention and demonstrates the value of pursuing further research, particularly when value of information analyses are included (Reference Annemans, Geneste and Jolain13;Reference Hartz and John14). Hence, rather than conduct no economic analysis alongside the CACTUS trial, we conducted exploratory analyses to help inform and support future research. We discuss at length the uncertainty associated with our analysis and the implications of this.
METHODS
We conducted a cost utility analysis alongside the CACTUS trial. We took a model-based approach so that costs and outcomes could be extrapolated beyond the end of the trial and so that rare events (such as death) not observed in the trial could be incorporated. Where possible the input parameters required for the model were obtained from the CACTUS trial data. An NHS and personal social service (PSS) perspective was taken. Costs and quality-adjusted life-years (QALYs) were estimated using patient questionnaires administered as part of the CACTUS trial combined with standard cost and valuation sources. Cost-effectiveness was described using an Incremental Cost Effectiveness Ratio (ICER) together with cost-effectiveness acceptability curves (CEACs). A value of information analysis was undertaken to inform future research priorities. A discount rate of 3.5 percent was used for costs and QALYs, as recommended by the National Institute for Health and Clinical Excellence (NICE) (15).
Model Design
A simple economic model was designed to estimate the likely cost-utility of the intervention over time. A three-state Markov model was used, whereby participants could transition from their initial aphasia health state to a response state, or to death. Patients in the response state could relapse to the aphasia state or die. A lifetime period was modeled using month-long cycles. A more elaborate economic model was considered but given the small sample size included in the CACTUS trial, populating this model was deemed unrealistic.
Transition Probabilities
Transitions probabilities were primarily based upon the CACTUS trial data. These are shown in Table 1. The primary clinical outcome measure in the trial related to word retrieval, the proportion of a selection of words that were named correctly at 5 and 8 months compared with baseline. The mean increase in percentage of words named correctly in the experimental group was 17.85 percent at 5 months, compared with −0.64 percent in the control group. We classed an increase of 17 percent or more as a “good response.” A total of 53.3 percent of patients in the intervention group achieved this, compared with 0 percent in the control group (mean difference, 53.3 percent; 95 percent confidence interval [CI] 23.8 percent to 82.8 percent). There was a statistically significant difference between treatment groups for the mean difference in the change in percentage of all treated words named correctly at 5 months from baseline, as reported in the clinical paper (Reference Palmer, Enderby and Cooper9).
Table 1. Model Parameter Values
If a patient demonstrated a good response at 5 months, we assumed that the initial response occurred at month 1. For the first 5 months of the model, patients who achieved a good response could either remain in that state or die. After 5 months, a relapse rate was incorporated to transfer patients back to the “aphasia” state. The relapse rate was estimated by subtracting the proportion of patients who maintained a 17 percent or better increase in the percentage of treated words named correctly at 8 months, from the proportion who could demonstrate that response at 5 months. In the CACTUS trial, 9/17 patients in the intervention group exhibited a good response at 5 months, and 6 of 12 patients followed-up maintained this response at 8 months. Hence, the relapse rate was small, with an estimated probability of 0.008 (0.8 percent) per month.
Transitions from the “aphasia” and “good response” states to death were based upon evidence from the literature on long-term survival following stroke (Reference Brønnum-Hansen, Davidsen and Thorvaldsen16). We used mortality rates for patients who had experienced a stroke 1 or more years previously and applied these rates to both the good response state and the aphasia state for the first 5 years of the model to reflect the duration for which evidence was available from the literature (Reference Brønnum-Hansen, Davidsen and Thorvaldsen16). After this additional mortality was applied based upon Office for National Statistics lifetables (17).
Quality of Life
Participants completed the EQ-5D questionnaire (Reference Dolan18) at baseline, 5 months and 8 months and utility scores for the modeled health states were based on these data. A-priori, we were unsure of the impact that the intervention might have on health state utility. However, we hypothesized that an improvement in word retrieval may lead to improved communication and associated with this, fewer problems undertaking usual activities. It was also thought possible that some patients may experience reduced anxiety or depression. Both of these effects could generate improvements in utility as measured by the EQ-5D. Because people with aphasia often have reading difficulties they were not asked to complete a standard EQ-5D questionnaire. Instead an amended “accessible” version (based on pictures) of the questionnaire was developed. Utility scores used in the model are presented in Table 1. We assumed that utility would be the same in nonresponders in each treatment group. Utility data were available for eight participants who achieved a good response at month 5, and these exhibited an incremental increase in utility of 0.07 compared with those who did not respond (95 percent CI, −0.15 to 0.29). The difference in mean utility between the two 5-month response categories was greater at the 8-month time-point (0.17; 95 percent CI, −0.16 to 0.50). However, due to the low patient numbers, and to take a conservative approach, in the economic model we assumed that the 0.07 increase in utility score was maintained over time, rather than modeling an increasing disparity between responders and nonresponders. The effect on utility was not statistically significant. We ran scenario analysis to explore this uncertain parameter further.
We assumed an age of 68 upon entry to the Markov model, matching the mean age of CACTUS participants. Utility scores were reduced over time according to multipliers estimated by Ara and Brazier (Reference Ara and Brazier19). QALYs were estimated for each cycle of the model by combining utility scores with life-years.
Intervention Costs
Intervention costs included the cost of computers (£495.99) (for those participants that did not have their own computer: 65 percent in the CACTUS trial), the cost of the Step-by-Step software (£250), the cost of microphones (£7.50) required for the program, and the cost of SLT support and training. Time spent by SLTs setting up the intervention and assisting patients were converted into costs using national unit costs (Reference Curtis20). In the CACTUS trial the mean face-to-face time spent by an SLT with each experimental group participant was 5.5 hours, with 0.20 hours of non-face-to-face time incurred (£190.83 per patient). Together with intervention costs, these combined to an estimated cost of £801.60 per patient in the treatment group. Costs for each health state are presented in Table 1.
Other Costs
Patient and carer diaries were used to collect data on changes in health and social services resource use. This included GP, nurse, and other health care professional visits and consultations, as well as hospital admissions, appointments, and prescribed medications. Data were combined with unit cost data from standard sources to calculate costs (Reference Curtis20–22). Diaries were not completed beyond 5 months, and after this time-point, we assumed that these costs were equal to those observed in the control group during the trial period for all patients.
Upon analyzing the patient resource use diaries two significant resource use outliers were identified, as two patients had experienced hospitalizations that were unlikely to have been related to the intervention (they were due to a bowel obstruction and a urinary tract infection). Given the small patient numbers this skewed cost estimates substantially, and therefore these two events were excluded from the analysis. The patient diaries demonstrated that mean health care resource use costs were £67.89 lower amongst patients in the experimental group (£203 per month compared with £270 per month), although this was not statistically significant at conventional levels (95 percent CI, −£210.64 to £346.42). £62.52 of this reduction was due to prescription costs and £5.37 was due to hospitalization costs. The reduction in costs associated with good responders compared with nonresponders came closer to statistical significance (£194.98; 95 percent CI, -£176.20 to £566.15), but this was based upon data from only 3 good responders and, therefore, was not deemed to be robust and was not incorporated in the economic model.
Sensitivity Analysis
Distributions were placed around the following parameters for use in probabilistic sensitivity analysis: Probability of good response in experimental group, Relapse rate, Utility in Aphasia health state, Utility improvement in Good Response state, Percent who require computer, Mean SLT face-to-face time, Mean SLT non face-to-face time, Other health care resource use.
We used gamma distributions for costs, log normal distributions for utilities, and beta distributions for probabilities, with dispersions based upon numbers seen in the trial. While probabilistic sensitivity analysis is useful in characterizing observed uncertainty, the confidence intervals around several parameters are extremely wide, reflecting the small size of the CACTUS trial. For this reason, we ran deterministic scenario analysis on key model parameters to determine which parameters were key drivers of the model under realistic assumptions.
Value of Information
We undertook an evaluation of the expected value of perfect information (EVPI) associated with our model. This represents the maximum value of further research (Reference Griffin, Welton and Claxton23). Further to this, expected value of perfect partial information (EVPPI) analyses were run to assess the value of the uncertainty around specific parameters (Reference Griffin, Welton and Claxton23). We undertook EVPPI analyses on each of the parameters around which distributions were placed for the probabilistic analysis (listed above); the results of these analyses are presented in Appendix A, which can be viewed online.
We estimated the value of information assuming a cost-effectiveness threshold of £20,000 per QALY over a period of 10 years (assuming that it might take 10 years before a new treatment for these patients is developed), using a 3.5 percent discount rate. We ran 500 inner loops and 100 outer loops (50,000 iterations in total) for each analysis based on evidence from the literature investigating the number of loops required to estimate the EVPPI with accuracy (Reference Brennan, Kharroubi and O'Hagan24).
RESULTS
Cost-effectiveness
We estimate that the intervention generates marginal incremental costs of £436.87, and a 0.14 marginal QALY gain, resulting in an ICER of £3,058, as presented in Table 2. Owing to the high levels of parameter uncertainty due to the small size of the CACTUS trial we conducted a probabilistic sensitivity analysis. This suggests that at a cost-effectiveness threshold of £20,000 per QALY the probability of the intervention being cost-effective is approximately 75.8 percent, as demonstrated by the CEACs presented in Figure 1. In the United Kingdom, typically an intervention is classed as cost-effective if it provides one additional QALY for an incremental cost of £20,000 or less (15), and, therefore, the intervention may be classed as cost-effective.
Table 2. Base Case Deterministic Results
QALY, quality-adjusted life-year; ICER, incremental cost-effectiveness ratio.
Figure 1. Cost-effectiveness acceptability curves. QALY, quality-adjusted life-year.
Expected Value of Information
We estimate that the expected per-patient value of perfect information is £143.68. We extrapolated this to a population level by estimating the number of patients that would be likely to receive the treatment over a 10-year time period. Estimates suggest that 11,400 people in Great Britain become aphasic each year following a stroke and that approximately 24 percent of people recover in the first 6 months (2). The clinical results of the CACTUS trial demonstrate that people with less than 10 percent word finding ability at baseline (17.9 percent of patients in the CACTUS trial) do not respond well to the intervention and so may not be eligible for it (Reference Palmer, Enderby and Cooper9). Hence, we estimate that the incidence population that would be eligible for computer intervention is 7,090 (11,400 – (0.24*11,400) – (0.179*11,400*0.76)). Added to this, estimates suggest that there is a prevalent population of 250,000 of people with aphasia (2). Reducing this by 17.9 percent results in a population of 205,250 that would be eligible for the intervention. Assuming that the prevalent population and the incident population are treated, we estimate that, on average, 27,615 patients would be treated per year over a 10 year period.
At the population level, we estimate that the EVPI for the intervention is approximately £37.0 million; however, an underlying assumption of this estimate is that there is 100 percent uptake among the eligible patient population. If only the incident population were treated each year, reflecting one possible implementation scenario, we estimate that the EVPI would be approximately £9.5 million. If perfect information on all model parameters could be obtained for this cost or less, then research to obtain such information would be cost-effective. In reality, it is impossible to obtain perfect information, so this represents a maximum estimate of the value of further research.
Scenario Analysis
Utility gain for responders
We investigated the impact of certain parameters within the model further using scenario analyses. First, we tested the effect of altering the utility gain for patients who achieve a good response from 0.005 to 0.07, with all other parameters remaining equal. The analysis demonstrated that, if the utility gain associated with a good response to treatment was 0.01 or less (seven times less than the mean observed in the CACTUS trial, but well within the 95 percent CI), the ICER would be greater than £20,000. This is presented in Figure 2a.
Figure 2. Scenario analyses results. (a) Impact on the incremental cost-effectiveness ratio (ICER) of different levels of utility gain associated with good response. (b) Impact on the ICER of different relapse rates.
Variations in the relapse rate
We also investigated the effect of altering the relapse rate from 0.8 percent (our base case) to 100 percent, with all other parameters remaining equal. The analysis demonstrated that if the relapse rate was greater than approximately 30 percent per month the ICER would be greater than £20,000. However, of interest, even if the relapse rate was 100 percent, that is, all responders relapse in month 6 (1 month after the intervention ends) the ICER would remain slightly below £30,000. This is presented in Figure 2b.
Altering the utility gain and the relapse rate
The results of the model are clearly sensitive to the utility gain and relapse rate parameters, but one-way deterministic sensitivity analysis suggests that large changes in parameter values are required for cost-effectiveness conclusions to alter. However, we investigated the combined effect of these two parameters and found that their combined impact is potentially much more important. For example, if the utility gain associated with a good response is halved to 0.035 and the relapse rate is increased to 30 percent per month after month 5, the ICER increases to £39,491, and the intervention would no longer be classed as cost-effective given a cost-effectiveness threshold of £20,000.
DISCUSSION
Our results suggest that the computer intervention is likely to represent a cost-effective use of resources. However, our analysis is exploratory and the results should be interpreted with care given the small sample size included in the CACTUS trial; further modeling and analyses are required if and when further data become available. Our probabilistic analysis indicates that we are reasonably confident in our results with a cost-effectiveness threshold of £20,000 per QALY gained, and our EVPI analysis suggests that there is only one parameter (the utility gain associated with a good response) that would provide a valuable focus for future research. However, our scenario analyses demonstrate that the relapse rate also has an important impact on the ICER, particularly when it is altered in combination with the utility gain parameter.
In the CACTUS trial the estimated relapse rate was very low and in our probabilistic analysis the distribution around this parameter was characterized using a beta distribution. The beta parameters were based upon the trial data and across 10,000 iterations the relapse rate was never sampled to be higher than 25 percent. However if, for example, 1 more patient had been observed to relapse from a good response to a nonresponse in the CACTUS trial, the relapse rate would have approximately quadrupled and the beta distribution would have dispersed. In addition, our estimate of the relapse rate may be subject to attrition bias because it was based upon the proportion of good responders at 5 and 8 months (9 of 17 participants and 6 of 12 participants, respectively), It is possible that those lost to follow-up may have been more likely to have relapsed. Hence, it might be hypothesized that, while our probabilistic analysis characterizes the uncertainty observed in the CACTUS trial, the trial may not have provided enough data for some parameters to be characterized appropriately. Owing to this, our analysis may overestimate the confidence that can be associated with our cost-effectiveness results, and may underestimate the value of providing further information on some parameters. Despite this, our population level EVPI estimate remains high in comparison to other interventions in other disease areas (Reference Latimer, Lord and Grant25).
This highlights a more general issue relating to value of information analysis; basing probabilistic sensitivity analysis and its associated EVPI on a single trial and model formulation may underestimate uncertainty and may cause a misrepresentation of population and partial EVPIs.
While we have extensively investigated parameter uncertainty in our model, it is important to also consider structural uncertainty. We decided that it was not possible to populate a more elaborate economic model using data from the CACTUS trial. For instance, it may be preferable to explicitly model different categories of response to treatment (e.g., moderate and major) and future neurological events (which would be expected to impact upon the long-term effectiveness of the treatment). For an exploratory economic analysis such as ours, we deemed a simple model to be adequate, particularly because we incorporated the issue of long-term effectiveness in our model through the relapse rate, and because data were not available to allow us to model a range of response categories. Thus, we anticipate that a more detailed model structure would not alter the results of our analysis given the data currently available. However, a more complex model structure may be developed if and when further data become available.
Importantly, we made two limiting structural assumptions that influence our results. First, we assumed that the intervention could never produce fewer QALYs than the control treatment because the utility gain associated with a good response must be at least zero, and the utility experienced by nonresponders was the same in the intervention group and the control group. This seems reasonable, but it might be argued that the utility in nonresponders in the intervention group may be lower than the utility in the control group. This would be relevant to consider were a larger trial capturing more utility data to be carried out.
Second, we assumed that control group patients had been living with aphasia for some time and that their health state would not improve. This reflects what was observed in the CACTUS trial at 5 months; however, at 8 months 3 patients in the control group reported word finding scores indicative of a “good response.” The impact of this on our cost-effectiveness results can be estimated using our scenario analyses. A proportion of control group patients achieving a good response over time is approximately equivalent to an increase in the relapse rate in the intervention group, as a larger proportion of patients in the two groups end up in the same health state. Taking into account the three patients in the control group that exhibited a good response at 8 months the relapse rate would be 8.7 percent per month, which (as can be seen by Figure 2b) would give an ICER of approximately £10,300.
Our analysis was further limited by the lack of evidence on what constitutes a “good response.” We assumed that anyone who demonstrated a word-finding improvement that was better than the average increase observed in the experimental group achieved a good response, but this is arbitrary. Altering the good response cut-off would change the relative response rate between the control and intervention groups, and would also alter the difference in utility associated with response and nonresponse states. This is worthy of further analysis were a larger trial to be carried out.
Our QALY estimates are based upon an unvalidated “accessible” version of the EQ-5D questionnaire. We have not allocated any more uncertainty to the estimated utilities due to this, as methods to do so are not forthcoming. Hence, we may have underestimated uncertainty and the value of perfect information. It cannot be guaranteed that the responses to this tool and the standard version of the EQ-5D would be perfectly correlated, as representing the EQ-5D questions and responses in pictorial form is open to mis-interpretation.
If patients who would not be expected to respond to the treatment could be identified, the cost-effectiveness case for the intervention would be likely to improve. The CACTUS trial results suggest that patients who could correctly name very low proportions of words at baseline were unlikely to respond to treatment (Reference Palmer, Enderby and Cooper9). Excluding these patients would lower the ICER for the intervention.
Finally, we allocated the full software and computer costs to intervention group patients even though the intervention was taken away at 5 months. In reality, this cost would allow the intervention to be provided for much longer. Hence, intervention costs may be overestimated and our ICER may be conservative.
CONCLUSION
The intervention of usual stimulation plus computer treatment may be cost-effective compared with usual stimulation alone in patients with long-term aphasia given a cost-effectiveness threshold of £20,000 per QALY. However, our economic analysis is exploratory and should be interpreted with care, especially considering the small size of the CACTUS trial. The utility gain associated with a good response to treatment is of greatest importance for future research, but the relapse rate is also important. It is also important to determine whether and at what rate patients receiving standard care are expected to achieve a good response over time. These data are best collected within a larger randomized clinical trial. With an EVPI of £37 million, such a trial (which may cost around £2 million) would appear to represent good value for money.
SUPPLEMENTARY INFORMATION
Supplementary Appendix can be found at: http://dx.doi.org/10.1017/S0266462313000421
CONTACT INFORMATION
Nicholas R. Latimer, BSc, MSc, PhD, (n.latimer@shef.ac.uk), Research Fellow in Health Economics, Health Economics and Decision Science, School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK
Simon Dixon, BSc, MSc, PhD, Professor of Health Economics, Health Economics and Decision Science, School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK
Rebecca Palmer, BA, PhD, Senior Lecturer, University of Sheffield, Sheffield Teaching Hospitals Foundation Trust, 107 Innovation Centre, 217 Portobello, Sheffield, S1 4DP, UK
CONFLICTS OF INTEREST
The authors report a grant to their institution from the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-1207-14097).
