Introduction
A large body of research has suggested that web-based interventions can be effective treatments for depression with comparable effect sizes to face-to-face treatments (Spek et al. Reference Spek, Cuijpers, Nyklicek, Riper, Keyzer and Pop2007a ; Andrews et al. Reference Andrews, Cuijpers, Craske, McEvoy and Titov2010; Cuijpers et al. Reference Cuijpers, Donker, van Straten, Li and Andersson2010). Self-guided forms of web-based treatment (i.e. interventions that patients work through on their own with no guidance) do not rely on having therapists available. These interventions can be made available to a greater number of people at very low incremental cost, thus increasing access and availability. They also maintain anonymity and overcome concern about stigmatization making them more acceptable to many people.
However, meta-analytic studies have also shown that self-guided web-based interventions (i.e. interventions that patients work through on their own without guidance) show less promising results than guided web-based interventions that are delivered with support from a coach or therapist (Gellatly et al. Reference Gellatly, Bower, Hennessy, Richards, Gilbody and Lovell2007; Spek et al. Reference Spek, Cuijpers, Nyklicek, Riper, Keyzer and Pop2007a ; Andersson & Cuijpers, Reference Andersson and Cuijpers2009; Cuijpers et al. Reference Cuijpers, Donker, Johansson, Mohr, van Straten and Andersson2011; Richards & Richardson, Reference Richards and Richardson2012). One explanation for the difference in effectiveness between guided and unguided web-based interventions is that the human support involved in guided interventions increases treatment adherence through accountability to a coach or therapist who is seen as trustworthy, benevolent, and an expert (Mohr et al. Reference Mohr, Cuijpers and Lehman2011). Furthermore, guided web-based interventions often not only involve a supportive coach who helps participants through the program but also more often than unguided interventions include human contact before treatment (e.g. during a diagnostic interview; Johansson and Andersson, Reference Johansson and Andersson2012) or include referral by a therapist (Marks & Cavanagh, Reference Marks and Cavanagh2009; Berger et al. Reference Berger, Caspar, Richardson, Kneubuhler, Sutter and Andersson2011), which may add to feelings of accountability.
In line with the idea of ‘supportive accountability’, higher dropout rates have been found in unguided web-based interventions for depression compared to guided web-based interventions with average levels of adherence estimated at 26% in unguided interventions and 72% in guided interventions (Richards & Richardson, Reference Richards and Richardson2012). In addition, empirical evidence has shown that greater exposure to the intervention is related to better treatment outcomes (Donkin et al. Reference Donkin, Christensen, Naismith, Neal, Hickie and Glozier2011) suggesting that efforts to increase adherence rates in web-based interventions may lead to better outcomes. To gain a better understanding of who may benefit from unguided web-based interventions and how we can improve adherence rates, there is a strong need to identify characteristics of individuals and interventions that are related to treatment dropout, as unguided interventions are much easier to implement and less costly than guided web-based interventions.
A few studies have already investigated this issue (Christensen et al. Reference Christensen, Griffiths and Farrer2009; Waller & Gilbody, Reference Waller and Gilbody2009). However, studies that have been conducted so far often lack the power to find reliable effects of predictors and moderators. In the current study we bring together the data from separate studies and employ a new strategy named individual patient data (IPD) meta-analysis. IPD meta-analysis was developed to address research questions that require large sample sizes and is based on data pooled from individual randomized control trials (RCTs) (Bower et al. Reference Bower, Kontopantelis, Sutton, Kendrick, Richards, Gilbody, Knowles, Cuijpers, Andersson, Christensen, Meyer, Huibers, Smit, van Straten, Warmerdam, Barkham, Bilich, Lovell and Liu2013). In this way it increases the power and precision to detect predictors and moderators. This study aimed to identify socio-demographic, clinical, and intervention characteristics that predict dropout rates in self-guided web-based interventions for depression. In the context of the present paper, the term adherence is defined as the percentage of treatment modules that were completed. Dropout rate was defined as a completion rate of <75% of the intervention modules, as we considered that in most interventions the core treatment elements are administered in this part of the treatment.
Method
Search strategy for identification and selection of studies
We used an existing database of randomized trials of psychological treatments for depression. The database has already been used by several published meta-analyses (http://www.evidencebasedpsychotherapies.org) and its detailed description can be found elsewhere (Cuijpers et al. Reference Cuijpers, van Straten, Warmerdam and Andersson2008). This database has been developed and is periodically updated by a comprehensive literature search of the following health-related databases: Cochrane Central Register of control trials, PubMed, PsycINFO and EMBASE from 1996 to January 2013. In these electronic searches, various key terms covering the concepts of psychotherapy and depression were used in different combinations (both MeSH terms and text words). For a detailed description of the searches the reader is referred to Cuijpers et al. (Reference Cuijpers, van Straten, Warmerdam and Andersson2008). In addition, several systematic reviews and meta-analyses in this research field have been cross-checked throughout the development of this database in order to ensure that no trials were missing. Along with the use of this database, we contacted authors and asked them to provide us with access to the datasets of trials that were not yet published.
Inclusion criteria for studies
We included (a) RCTs, (b) comparing a psychological intervention, (c) delivered through the web, (d) without any form of personal guidance, (e) with a control or comparison group, (f) aimed at adults with depression (based on a clinical interview or on elevated depressive symptoms ratings on self-report measures).
Quality assessment
The validity of the studies included in the present IPD meta-analysis was examined by two independent reviewers (E.K. and D.T.) according to four criteria of the Cochrane Risk of Bias assessment tool (Higgins & Green, Reference Higgins and Green2011; Higgins et al. Reference Higgins, Altman, Sterne, Higgins and Green2011). We tested if the allocation concealment was adequately generated (sequence generation), the allocation was sufficiently concealed (allocation concealment), the knowledge of the allocated intervention was adequately prevented (blinding), and any incomplete outcome data were sufficiently addressed. However, we did not consider that incomplete outcome data could influence the results of the present IPD meta-analysis since the primary aim of this paper was to identify factors influencing treatment dropout. Finally, when the information that was provided in the papers did not provide sufficient details to assess quality, we contacted the primary authors to ask what procedure was actually followed and subsequently we ran sensitivity analysis based on what the papers reported. Disagreement between the reviewers was resolved through discussion, and if needed a third reviewer was consulted (P.C.).
Data extraction and preparation
Two authors independently extracted data included in the present meta-analysis (E.K. and D.T.). We first contacted authors of RCTs that satisfied the inclusion criteria and we asked them whether they would permit us access to their primary datasets. We identified the variables, which were common to all or most of the included datasets. These were the following: randomized group (therapy or control), baseline and follow-up depression scores, age, gender, educational level, employment status, relationship status (being in or not in a relationship), number of modules completed and presence of anxiety symptoms at baseline (yes/no; based on a clinical interview or on elevated anxiety symptoms ratings on self-report measures). Finally, we combined the individual datasets into one large pooled dataset.
Statistical analysis
In this paper, data were extracted only for intervention groups and not for control comparison conditions as we only looked at predictors of treatment adherence. Studies included in the present IPD meta-analysis used measures such as the Center for Epidemiologic Studies Depression Scale (CES-D; Radloff, 1977), the Beck Depression Inventory - I (BDI-I; Beck et al. Reference Beck, Ward and Mendelson1961) or the Beck Depression Inventory - II (BDI-II; Beck et al. Reference Beck, Steer and Brown1996) to monitor change in depression. These depression measures were standardized (transformed into z scores) across the pool of the studies. We also conducted sensitivity analyses to assess the impact of baseline severity on dropout from treatment for CES-D and BDI separately. We analysed the effects of predictors on dropout from treatment using a design-based analysis of the data to account for the clustering of participants within studies. Individual patient data were analysed by a Poisson regression model for patients nested within studies to obtain relative risks (RR) of treatment dropout on the selected factors, adjusted for the other predictors in the Poisson model and taking into account the clustered data structure by obtaining robust (Hubert–White) standard errors based on the first-order Taylor-series linearization method as implemented in Stata version SE 12.1 (StataCorp., 2011). This methodological approach is computationally efficient in synthesizing and estimating the effect of predictors (Zou, Reference Zou2004). We conducted the analysis in three steps. First we conducted a series of bivariate analyses to assess the RR of each factor at a time (the so-called ‘bivariate model’). Then we repeated the analyses with all factors simultaneously entered in the Poisson model (the so-called ‘complete model’). Last, we simplified the complete model by only retaining those factors in the model that were statistically significant by eliminating factors that were not significant (the ‘parsimonious model’). Finally, we performed sensitivity analysis to assess the impact of the included studies’ quality on dropout from treatment and we checked whether the assumption of linearity was met for the relationship between dependent and independent variables.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Results
Selection of included studies
A total number of 14 164 abstracts were identified through bibliographic database searching. After the removal of duplicates, 10 474 abstracts were examined. A total of 1476 full text papers were retrieved for potential inclusion. After the exclusion of 1123 studies, 353 trials were included in the database. We searched through this database and in additional sources (grey literature, researchers on this field) and we identified 13 eligible RCTs for inclusion in the current meta-analysis. We were unable to retrieve the data from three studies (Clarke et al. Reference Clarke, Reid, Eubanks, O'Connor, DeBar, Kelleher, Lynch and Nunley2002, Reference Clarke, Eubanks, Reid, Kelleher, O'Connor, DeBar, Lynch, Nunley and Gullion2005, Reference Clarke, Kelleher, Hornbrook, Debar, Dickerson and Gullion2009) and included 10 RCTs in the present IPD meta-analysis (77%). Fig. 1 presents the study selection process. Overall, the three studies that we did not include (n = 302) were very similar to the 10 included studies, except for the method of recruitment (all participants in these three studies were recruited through a health mental organization in the Northwest United States, while none of the other studies recruited patients this way). The main outcome measures in these studies were the CES-D and the Patient Health Questionnaire-8 items (PHQ-8; Clarke et al. Reference Clarke, Reid, Eubanks, O'Connor, DeBar, Kelleher, Lynch and Nunley2002, Reference Clarke, Eubanks, Reid, Kelleher, O'Connor, DeBar, Lynch, Nunley and Gullion2005, Reference Clarke, Kelleher, Hornbrook, Debar, Dickerson and Gullion2009).
Fig. 1. Study selection process.
Study characteristics
In the present IPD meta-analysis, ten studies with a total of 2705 participants were included. All the examined studies recruited their participants from the community, and they were conducted across six different countries: Australia (n = 2), Germany (n = 2), Spain (n = 1), Sweden (n = 1), Switzerland (n = 1), and The Netherlands (n = 3). The majority of the included studies used self-report outcome measures for depression on which the participants needed to score above a predetermined cut-off point in order to be included in the trial. Seven out of the ten included studies used either the BDI-I or BDI-II as a primary outcome measure while the remaining trials used the CES-D.
All included unguided web-based interventions were based on three different theoretical models of psychotherapy. The majority of the included studies used interventions based on cognitive behaviour therapy (CBT) principles (n = 8). The remaining studies used either problem solving therapy (PST) (n = 1) or they compared web-based CBT with interpersonal psychotherapy (IPT) (n = 1). Table 1 shows selected characteristics of the included studies.
Table 1. Characteristics of included studies
BDI, Beck Depression Inventory; CBT, cognitive and behavioural therapy; CES-D, Centre of Epidemiological Studies for Depression Scale; EDS, The Edinburgh Depression Scale; HADS, Hospital Anxiety and Depression Scale; IPT, interpersonal psychotherapy; K10, Kessler Psychological Distress Scale; MDD, major depressive disorder; Mini DIPS, Mini Diagnostic Interview for Psychiatric Disorders; n, number; SCID-I-CV, Structural Clinical Interview for DSM-IV Axis I disorders; PST, problem solving therapy.
a A positive or a negative sign is given in this column for the following quality criteria respectively: allocation sequence, allocation concealment, blinding of assessors, and incomplete outcome data (whether or not the study used intention-to-treat analysis).
Most of the participants were female (n = 1945/2705, 72%) and most were educated to university level (n = 1933/2705, 71%). The modal age group into which participants fell was 25–34 years (n = 741/2705, 27%). The average score on the CES-D at baseline assessment was 35.5 (s.d. = 11.5), while the average score on the BDI was 28.4 (s.d. = 13.5) indicating a high degree of severity. The average score on the CES-D and the BDI reduced at the post-treatment assessment to 24.2 (s.d. = 13.2, n = 650) and 20.7 (s.d. = 14.8, n = 495), respectively. The majority of the sample reported symptoms of co-morbid anxiety (n = 1689/2705, 71.6%) (Table 2).
Table 2. Demographic and clinical characteristics of the included sample
BDI, Beck Depression Inventory; CES-D, Centre of Epidemiological Studies for Depression Scale; n, number of patients; s.d., standard deviation.
Across the 10 included RCTs, 1090 participants (40%) dropped out before the completion of 25% treatment modules, 1604 (59%) dropped out before completing half of the treatment modules. Further, levels of dropout increased to 70% (1880/2705) when we looked at the number of participants that completed 75% of treatment modules. Finally, only a small percentage (17%, 452/2705) completed all treatment modules.
Quality assessment
All the included studies had acceptable methodological quality. The sequence was adequately generated, and the allocation was adequately concealed. Moreover, all trials used self-report outcome measures, which were administered via the Internet. Therefore, blinding of assessors was considered as adequately addressed across the ten studies of this IPD meta-analysis. However, the participants were not blinded to the interventions, and this may have caused bias. Finally, all included RCTs used intention-to-treat analyses including all the randomized participants in their post-treatment analyses, which indicates that incomplete outcome data were adequately addressed (see Table 1).
Predictors of dropout in self-guided web-based treatment for depression
The results of the bivariate analyses indicated that male gender (RR 1.05, 95% CI 1.01–1.11), participants with a low educational background (primary education: RR 1.23, 95% CI 1.13–1.33), the presence of co-morbid anxiety symptoms (RR 1.18, 95% CI 1.01–1.38) and CBT-based interventions (RR 1.19, 95% CI 1.03–1.39) were related to a higher risk of dropping out. Finally, the chance of dropping out significantly decreased for every 4 years of age increase (RR 0.98, 95% CI 0.97–99). The remaining variables/potential predictors (baseline severity of depression, relationship status, number of intervention modules and employment status) failed to achieve a statistically significant level of p < 0.05 in the bivariate analysis (see Table 3).
Table 3. Predictors to self-guided web-based psychotherapy for depression
CBT v. other, Cognitive behavioural therapy compared to other types of psychotherapy; CI, confidence intervals; N obs, number of observations; RR, risk ratio; substandard error.
Additionally, under the parsimonious model, male gender (RR 1.08, 95% CI 1.03–1.13), lower educational level (primary education: RR 1.26, 95% CI 1.14–1.39), older age (RR 0.94, 95% CI 0.87–1.02) and co-morbid anxiety (RR 1.18, 95% CI 1.01–1.38) remained statistically significant predictors of dropout from treatment. However, in our sample, CBT/not-CBT intervention status was confounded with number of modules. Thus, the effects of intervention type could not be disentangled from the number of modules and we excluded these predictors from the parsimonious model. Finally, depression severity, employment status and relationship status remained non-significant after controlling for the other predictors (see Table 3).
Sensitivity analysis
We analysed the impact of depression severity on dropout for CES-D and BDI scores separately. Individuals who scored higher on CES-D at baseline had a slightly higher risk of dropping out than those with lower scores (RR 1.004, 95% CI 1.003–1.005, observations=1987, p < 0.001). However, the increase in risk was quite small and is therefore unlikely to have clinical relevance. Further, separate analysis for BDI scores at baseline did not produce statistically significant results (p > 0.05). It is important to stress that the results of BDI analysis were based on a considerably smaller, although sufficiently powered, number of participants (n = 718).
Three studies did not report all relevant information regarding allocation concealment in the published papers (although personal contact with the primary authors illustrated that the allocation was adequately concealed) and thus, we decided to run sensitivity analysis based on what the papers reported. We examined the impact of quality of the included studies on treatment dropout. Study quality did not significantly predict treatment dropout (p > 0.05). Further, we controlled for study quality in our final parsimonious model. The predictors remained the same after adjusting for the quality of the included studies.
Discussion
Main findings
The present IPD meta-analysis aimed to identify predictors of treatment dropout in self-guided web-based interventions for depression. We tested the relationship between dropout and several socio-demographic, clinical and intervention characteristics. The multivariate analysis of 2705 individual patients’ data revealed that being male; having attained a lower educational level (primary education); a younger age and having co-morbid anxiety symptoms significantly increased the risk of dropping out before the completion of 75% of treatment modules and thus were related to high treatment's dropout.
Placing our findings in the wider context of the literature
The finding that gender predicted treatment dropout has not been identified by previous literature on self-guided web-based interventions. However, this result may reflect a different coping strategy between the two genders. Previous research has shown that females generally present with a higher effort to cope with depression compared to males (Babwah et al. Reference Babwah, Baksh, Blake, Cupid-Thuesday, Hosein, Sookhai, Poon-King and Hutchinson2006). These efforts might enhance their willingness to continue and complete web-based interventions without any form of guidance. There is also evidence to support the idea that women are generally more conscientious regarding health issues compared to men (Babwah et al. Reference Babwah, Baksh, Blake, Cupid-Thuesday, Hosein, Sookhai, Poon-King and Hutchinson2006). These differences in health attitudes could partly account for the differences in treatment compliance rates between the genders.
A lower educational background has also been identified as a risk for dropping out in previous research and it has been suggested that low educational status is a barrier to adherence to web-based CBT because of greater difficulties in understanding the intervention content and procedure and limited abilities in using information technology which may result in diminished motivation to continue and complete a self-guided web-based treatment (Waller & Gilbody, Reference Waller and Gilbody2009).
Unlike the results from this study that showed that younger age was related to low treatment adherence, previous research showed that younger individuals had higher adherence to web-based treatment (Christensen et al. Reference Christensen, Griffiths and Farrer2009).
Co-morbid anxiety symptoms increased the risk of dropping out of the treatment 16%. It is important, however, to stress that studies included in this meta-analysis were not designed for the treatment of anxiety or to deal with co-morbid anxiety and therefore the reason for this finding is unclear. Further research is needed to clarify this.
None of the remaining variables significantly predicted treatment attrition and results derived by the present IPD meta-analysis were not influenced by quality of the included studies. The lack of a significant effect on adherence of relationship status is consistent with results reported by Christensen et al. (Reference Christensen, Griffiths and Farrer2009). Further, Christensen et al. (Reference Christensen, Griffiths and Farrer2009) concluded that dropout increases with the severity of baseline depression. The findings from the present study suggest that the severity of depression does not significantly predict dropout from treatment. However, when we examined the impact of baseline severity separately for CES-D and BDI we found a significant but small higher risk for dropping out of treatment for patients who scored higher on CES-D at baseline, a result which is consistent with the conclusions of Christensen et al. (Reference Christensen, Griffiths and Farrer2009).
Strengths and limitations
One of the strengths of the present IPD meta-analysis was that it was based on a novel methodological approach that it is considered a gold standard for identifying predictors, moderators and mediators to treatment dropout and outcome. Combining raw individual data from several studies into one single dataset provides adequate power and precision to detect predictors of treatment attrition. Further, the systematic literature search employed by the present IPD meta-analysis reduced the risk of introducing study selection bias into the results.
In spite of the aforementioned strengths it should be noted that the present study has several limitations. Among these limitations was the risk of availability bias. We could access ten RCTs’ individual patient datasets out of 13 eligible studies. Although this is higher than in other IPD meta-analyses (Riley et al. Reference Riley, Simmonds and Look2007), the ten available RCTs might differ in several ways from the three unavailable studies. Moreover, some of the predictor variables were not reported across all the ten RCTs. This might have resulted in lower power to predict effects for some of the variables of interest, although the IPD was better powered to detect a true effect than a single trial. However, such small effects would be less relevant from a clinical or public health perspective.
Moreover, the participants of the present IPD meta-analysis differ from patients in clinical samples. For instance, all the participants were recruited through the community and were proactively seeking help for their symptoms. Thus, the present findings might not be generalized to the whole population with depression but it is representative for help seeking individuals in the community. It should also be borne in mind that four of the included studies conducted a diagnostic interview before inclusion of the participants. (Spek et al. Reference Spek, Cuijpers, Nyklicek, Riper, Keyzer and Pop2007 Reference Spek, Nyklicek, Smits, Cuijpers, Riper, Keyzer and Pop b ; de Graaf et al. Reference de Graaf, Gerhards, Arntz, Riper, Metsemakers, Evers, Severens, Widdershoven and Huibers2009; Vernmark et al. Reference Vernmark, Lenndin, Bjarehed, Carlsson, Karlsson, Oberg, Carlbring, Eriksson and Andersson2010; Berger et al. Reference Berger, Caspar, Richardson, Kneubuhler, Sutter and Andersson2011). This may have enhanced treatment adherence by increasing any feeling of accountability. However, we considered that these studies should be retained in our analyses since they did not provide any guidance throughout treatment. Finally, in the available data intervention type (CBT v. others) is confounded with the number of sessions and thus, it is not possible to reliably attribute dropout to a particular type of intervention or to the number of treatment modules.
All these predictors should be taken into account in future development of self-guided interventions for depression. For example, different features of web-based interventions may be appealing to different individuals and it is important to find out what works best for whom. Future interventions could, for example, employ more audio-visual components such as videos or gaming and less written material for individuals with a lower education. This knowledge will help in utilizing the self-guided form of web-based interventions in the most efficient and effective way. Future studies may need to be tailored to the particular needs of individuals with co-morbid anxiety symptoms, male gender, with a low educational background and young age. Further, future research should also examine dropout at different time points or as a function of exposure to particular types of content, as treatment dropout at different time intervals may represent different processes. Other psychological predictors such as personality styles, motivation and preferences should be included in future trials to inform tailoring. This might prevent dropout in future versions of self-guided web based interventions.
Acknowledgements
The authors would like to thank Erica Weitz, Spyridon Kolovos and Ioannis Gonnianakis for assistance in various parts of this IPD meta-analysis. This research received no specific grant from any funding agency, commercial or not-for- profit sectors.
Declaration of Interest
None.
