Introduction
Major depressive disorder (MDD) and generalized anxiety disorder (GAD) are highly co-morbid, with 58–70% co-morbidity rates (Brown et al. Reference Brown, Campbell, Lehman, Grisham and Mancill2001). Compared with having either disorder alone, co-morbid GAD/MDD is associated with longer episode duration, poorer prognosis and greater impairment (e.g. Kessler et al. Reference Kessler, DuPont, Berglund and Wittchen1999). Although both disorders can be treated effectively with cognitive behavioural therapy (CBT) (Cuijpers et al. Reference Cuijpers, van Straten, Andersson and van Oppen2008; Stewart & Chambless, Reference Stewart and Chambless2009), it remains unclear whether single-disorder CBT protocols provide the most optimal approach for treating individuals with both GAD and MDD. First, there is some evidence that single-disorder treatments may not adequately address co-morbidity (van Balkom et al. Reference van Balkom, van Boeijen, Boeke, van Oppen, Kempe and van Dyck2008). For example, CBT for GAD reduces depressive symptoms immediately post-treatment, but these gains are not sustained over longer-term follow-up (Newman et al. Reference Newman, Przeworski, Fisher and Borkovec2010). Second, MDD and GAD share many common biological, environmental and temperamental risk factors (neuroticism: Andrews, Reference Andrews1990; Hettema et al. Reference Hettema, Neale, Myers, Prescott and Kendler2006), as well as similar cognitive, behavioural and emotional processes that maintain both disorders. For these reasons, CBT protocols for MDD and GAD share many common elements (e.g. cognitive re-structuring) leading to redundancy across treatment protocols (see Chorpita & Daleiden, Reference Chorpita and Daleiden2009). Third, the provision of sequential single-disorder treatments for those with GAD and MDD may not always be practical in clinical settings where resources are scarce, the number of trained clinicians in multiple evidence-based protocols is low, and number of sessions are restricted (Norton, Reference Norton2012).
These reasons have motivated the development of new transdiagnostic treatments that distil common elements of CBT to target shared maintaining factors across anxiety and mood disorders, with promising results (Barlow et al. Reference Barlow, Allen and Choate2004; Norton, Reference Norton2012). Such treatments are acceptable to patients, and effective in treating primary and co-morbid diagnoses (McEvoy et al. Reference McEvoy, Nathan and Norton2009). Preliminary evidence suggests that transdiagnostic treatments for anxiety disorders are as effective as single-disorder treatments (Norton, Reference Norton2012). Transdiagnostic treatments have the added benefit of minimizing burden to patients, clinicians and health care systems (e.g. Lumpkin et al. Reference Lumpkin, Silverman, Weems, Markham and Kurtines2002; Page & Hooke, Reference Page and Hooke2003; Norton, Reference Norton2008), potentially enhance the efficiency of treatment (Chu, Reference Chu2012), and make it easier to disseminate treatments (Weisz et al. Reference Weisz, Chorpita, Palinkas, Schoenwald, Miranda, Bearman, Daleiden, Ugueto, Ho, Martin, Gray, Alleyne, Langer, Southam-Gerow and Gibbons2012).
Internet-based CBT (iCBT) interventions offer the advantage of being more cost-effective than face-to-face treatment (Hedman et al. Reference Hedman, Andersson, Ljótsson, Andersson, Rück and Lindefors2011), and are accessible to many people with anxiety and depression who do not seek face-to-face help (Andrews et al. Reference Andrews, Henderson and Hall2001). iCBT is an effective treatment for depression (Andersson & Cuijpers, Reference Andersson and Cuijpers2009), GAD (Robinson et al. Reference Robinson, Titov, Andrews, McIntyre, Schwencke and Solley2010) and for mixed anxiety disorders (Titov et al. Reference Titov, Andrews, Johnston, Robinson and Spence2010), with large effect sizes (0.88, numbers-needed-to-be-treated = 2.13) and low drop-out rates (Andrews et al. Reference Andrews, Cuijpers, Craske, McEvoy and Titov2010). A recent randomized controlled trial (RCT) by Titov et al. (Reference Titov, Dear, Schwencke, Andrews, Johnston, Craske and McEvoy2011) found that an eight-lesson iCBT programme for co-morbid depression and three anxiety disorders (GAD, social phobia and panic disorder) was more effective than wait-list control (WLC), with moderate effect sizes (0.58 and 0.52 for depression and anxiety, respectively). For an open trial of a five-lesson version, see Dear et al. (Reference Dear, Titov, Schwencke, Andrews, Johnston, Craske and McEvoy2011).
We extended beyond Titov et al. (Reference Titov, Dear, Schwencke, Andrews, Johnston, Craske and McEvoy2011), and designed a shorter six-lesson clinician-assisted iCBT treatment for mixed GAD and MDD, called the Worry and Sadness Program. We sought to explore whether their modest effect sizes could be enhanced by restricting the focus on GAD and MDD, and by placing greater emphasis on treating maladaptive rumination and worry, key maintaining factors for depression and anxiety (Nolen-Hoeksema et al. Reference Nolen-Hoeksema, Wisco and Lyubomirsky2008). Study 1 was a RCT of the Worry and Sadness Program, compared with a WLC group. Study 2 investigated the effectiveness of the same programme, when prescribed by primary care practitioners to their patients via www.crufadclinic.org (now www.thiswayup.org.au/clinic). We expected that adherence would be better in the RCT compared with primary care, whereas the effectiveness of the programme would be similar across both settings.
Study 1: a randomized controlled trial of the Worry and Sadness Program
Method
Design
A CONSORT (Consolidated Standards of Reporting Trials) 2010-compliant (Schulz et al. Reference Schulz, Altman and Moher2010) RCT design was used to compare an immediate treatment group to a deferred-treatment group (WLC). The immediate treatment group was followed up until 3 months post-treatment. The WLC group were enrolled in the iCBT course after the treatment group had completed the programme. A between-groups effect size of 0.6 with power of 80% was expected as achieved in prior studies (e.g. Titov et al. Reference Titov, Dear, Schwencke, Andrews, Johnston, Craske and McEvoy2011). A minimum of 45 participants were required in each group, but 110 were recruited to hedge against attrition.
Participants
Participants were recruited from an existing wait-list of individuals who had previously expressed interest in participating in iCBT, and from an online advertisement posted on the www.virtualclinic.org.au website. Participants applied online to www.virtualclinic.org.au after reading details about the study, including the eligibility criteria for inclusion, as follows: (i) aged over 18 years; (ii) self-identified as suffering from mild or moderate GAD, MDD or mixed anxiety-depressive disorder [International Classification of Diseases, Tenth Revision (ICD-10) F41.2] and with both Generalized Anxiety Disorder seven-item scale (GAD-7) and Patient Health Questionnaire nine-item scale (PHQ-9) scores above clinical threshold; (iii) prepared to provide name, telephone number and address, and the name and address of their local general practitioner; (iv) had access to a telephone, computer and printer; and (v) had maintained a stable dosage for at least 2 months prior to participation if they were receiving current pharmacological and/or psychotherapy treatment, and agreed to not make any changes to their treatment during the entire duration of the study.
Details of participant flow are in Fig. 1. A total of 87 applicants were excluded after completing initial online screening questions. So, 135 applicants met the online selection criteria, provided informed consent, and then participated in a brief telephone interview (nine additional people passed online screening but were unable to be contacted for telephone interview). Trained interviewers administered the Mini International Neuropsychiatric Interview version 5.0.0 (MINI; Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998) to confirm whether the applicant met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria for GAD and/or MDD. A total of 26 individuals were excluded after telephone interview, leaving 109 applicants who met inclusion criteria and were randomized. Random numbers were generated using a random number service (http://www.random.org) by a team member who was not involved in the study. Concealment of allocation was maintained until the applicant met all inclusion criteria and an offer of participation was made. The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital, Sydney, Australia (no. HREC 11/SVH/95), and the trial was registered at the Australian New Zealand Trial Registry (no. ACTRN12611001055998).
Fig. 1. Participant flow diagram for study 1 (randomized controlled trial). iCBT, Internet cognitive behavioural therapy; PHQ-9, Patient Health Questionnaire nine-item scale; MINI, Mini International Neuropsychiatric Interview.
Primary outcome measures
The MINI version 5.0.0 (Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998) MDD, GAD and risk assessment modules were administered to assess current and lifetime DSM-IV diagnoses. Due to practical constraints, other co-morbid diagnoses were not assessed. MINI possesses excellent inter-rater reliability (k = 0.88–1.00) and good concurrent validity with the Composite International Diagnostic Interview (CIDI; Kessler & Ustün, Reference Kessler and Ustün2004).
The PHQ-9 (Kroenke et al. Reference Kroenke, Spitzer and Williams2001) and GAD-7 (Spitzer et al. Reference Spitzer, Kroenke, Williams and Lowe2006) measured depression and generalized anxiety symptoms, respectively, over the past fortnight. Both scales have good psychometric properties, and a cut-off score ⩾10 is used to define probable DSM-IV diagnoses of MDD and GAD (Kroenke et al. Reference Kroenke, Spitzer, Williams, Monahan and Löwe2007; Wittkampf et al. Reference Wittkampf, Naeije, Schene, Huyser and van Weert2007). Internal reliabilities for the current sample were good (current sample α’s: PHQ-9 = 0.75–0.87; GAD-7 = 0.85–0.91). The Kessler 10-item Psychological Distress scale (K-10; Kessler et al. Reference Kessler, Andrews, Colpe, Hiripi, Mroczek, Normand, Walters and Zaslavsky2002) measured non-specific psychological distress over the past 30 days. The K-10 has excellent psychometric properties (Furukawa et al. Reference Furukawa, Kessler, Slade and Andrews2003), and higher scores indicate higher distress (current sample α = 0.76–0.89).
Secondary outcome measures
The 12-item World Health Organization Disability Assessment Schedule II (WHODAS-II; WHO, 2001) measured functional impairment and activity limitation (higher scores indicate higher impairment). The WHODAS-II possesses good psychometric properties (Andrews et al. Reference Andrews, Kemp, Sunderland, Von Korff and Ustun2009) (current sample α = 0.85–0.87). The Beck Depression Inventory – Second Edition (BDI-II; Beck et al. Reference Beck, Steer and Brown1996b) measured depressive symptoms in the past fortnight, and has good psychometrics, including high internal consistency (Beck et al. Reference Beck, Steer, Ball and Ranieri1996a) and comparable psychometric properties for online versus pen-and-paper administration (Hollandare et al. Reference Hollandare, Andersson and Engstrom2010) (α = 0.84–0.92 for the current sample). The Penn State Worry Questionnaire (PSWQ; Meyer et al. Reference Meyer, Miller, Metzger and Borkovec1990) measured trait worry, and has good reliability and validity (Brown et al. Reference Brown, Antony and Barlow1992) (α = 0.87–0.88 for the current sample). The NEO-Five Factor Inventory – Neuroticism Subscale (NEO-FFI-N; Costa & McCrae, Reference Costa and McCrae1985) measured the personality dimension of neuroticism, and has good psychometric properties (Cuijpers et al. Reference Cuijpers, van Straten and Donker2005) (current sample α = 0.58–0.78).
Description of treatment
The Worry and Sadness Program, delivered via https://www.virtualclinic.org.au/, consists of six online lessons to be completed over a 10-week period (19 October to 23 December 2011). Lesson content is presented in the form of an illustrated story about two fictional characters who experience anxiety and depression, and gain mastery over their symptoms using CBT techniques (e.g. activity schedulingFootnote 1Footnote †; see Table 1 for course content). Following each lesson, participants download and print out a lesson summary, which includes practical homework (e.g. graded exposure tasks). Participants had access to: (i) frequently asked questions for each lesson; (ii) ‘patient recovery stories’ from former patients of www.virtualclinic.org.au; and (iii) extra resources on: good sleep, activity planning, assertiveness, pleasant events, conversational skills, hunt for the positives, medications, panic attacks, structured problem solving, thought challenging, worry time, and worry stories (imaginal worry exposure).
Table 1. Lesson content of the Worry and Sadness Program
Outcome measurement
The MINI was administered to all participants at pre-treatment, and at 3-month follow-up for the treatment group (assessors were not blinded to treatment condition at 3-month-follow-up). Self-report outcome measures were administered at pre-treatment (prior to lesson 1), before lesson 4, at post-treatment (1 week after the treatment group finished the programme), and 3-month follow-up (treatment group only), or at matched time points for WLCs. The treatment group completed the K-10 before they commenced each lesson, as a measure to alert the clinician if participants’ scores rose by more than 0.5 s.d. between lessons, indicating a significant increase in distress.
Clinical contact with clinician and therapist
The treatment group participants received regular email and/or telephone contact with their clinician (K.M., the practice manager) until they completed lesson 2, after which contact was made in response to patient request or if the clinician initiated contact because of a deterioration in the K-10 or PHQ-9 score. K.M. was supervised by a therapist (J.M.N., a Ph.D. clinical psychologist). All emails requiring clinical advice were responded to by the therapist. If clinically indicated, or if a patient's K-10 and/or PHQ-9 scores deteriorated, the therapist would make telephone contact with the participant. The clinician, therapist and research support officer (A.M.) participated in weekly meetings to assess and discuss participants’ progress.
Statistical analyses
Significance testing of group differences regarding demographic data and pre-treatment measurements was conducted using independent samples t tests, and χ2 where the variables consisted of nominal data. Intent-to-treat (ITT) marginal model analyses using the restricted maximum likelihood method were used to account for missing data due to participant drop-outs. This approach is appropriate for RCTs with multiple time points (Salim et al. Reference Salim, Mackinnon, Christensen and Griffiths2008), and does not assume that the last measurement was stable (an assumption of the the last observation carried forward approach; see Gueorguieva & Krystal, Reference Gueorguieva and Krystal2004). As the primary outcome measures (GAD-7, PHQ-9, K-10) were also collected at mid-point, effects for the primary measures were modelled with an autoregressive (AR1) covariance structure to account for the correlation between the time-points. Effects for the secondary measures were modelled using an unstructured (UN) covariance structure. Model fit was evaluated using Schwarz's Bayesian criterion. Significant effects were followed up with pairwise contrasts comparing pre-treatment with post-treatment scores. Analyses were performed in SPSS version 20 (IBM, USA). We applied Hedges’ g adjustment to calculate the between-group effect sizes, and adjusted for the correlations between repeated measurement time-points when calculating the within-group pre- to post-treatment effect sizes.
Results
Baseline
The mean age of participants was 44.3 years (s.d. = 12.2, range = 21–80), and 77 were female (77.8% of the sample). The majority were married or living in a de facto relationship (n = 62, 62.6%), were educated with a postgraduate degree (n = 58, 58.6%) and were in full-time (n = 36, 36.4%) or part-time paid work (n = 23, 23.2%). Participants reported moderate levels of depression and anxiety on the PHQ-9 and GAD-7 at the start of the programme (see Table 2 for sample characteristics).
Table 2. Baseline demographics and sample characteristics for the treatment and WLC groups
WLC, Wait-list control; s.d., standard deviation.
a Educational status = highest level of education received.
Diagnostic status according to MINI interviews
In the treatment group, 19 had GAD/MDD (41.3%), 21 had GAD (45.7%) (but with subthreshold MDD) and six met criteria for MDD (13.0%) and had subthreshold GAD. In the WLC group, 28 met criteria for co-morbid GAD/MDD (52.8%), 16 (30.2%) had GAD, and nine (17.0%) had MDD, but each had subthreshold GAD or MDD, respectively. Diagnostic status did not differ between groups at baseline [χ2(2) = 2.52, p > 0.05].
Baseline between-group comparisons
There were no differences between the groups on age, pre-treatment BDI-II, GAD-7, NEO-FFI-N, PHQ-9, K-10, or PSWQ scores (p's > 0.05). However, the control group reported significantly higher disability on the WHODAS-II (t 97 = 2.35, p = 0.02). There were no between-group differences in any other demographic characteristics such as gender, marital status, educational status or employment status, as demonstrated by χ2 analyses (see Tables 2 and 3).
Adherence results
A total of 49 individuals were randomized into the treatment group. Of these, 46 completed pre-treatment questionnaires and were eligible for analysis, and 41/46 completed the total six lessons (89% adherence). Post-treatment and 3-month follow-up data were collected on 43/46 and 40/46 participants, respectively. Of the participants who were considered drop-outs, one completed one lesson only, one completed two lessons, two completed three lessons, and one completed four lessons. Therefore, 60 participants were randomly allocated to the WLC group. Of these, 54 completed pre-treatment questionnaires. One of these withdrew to seek alternative treatment, leaving 53 eligible for the analysis. Post-treatment data were collected on all 53 participants (see Fig. 1 for further details).
Primary outcome measures and effect sizes
Marginal models with group as a fixed factor and time as a repeated factor were conducted separately for each of the dependent measures (see Table 3 for results). All main effects for PHQ-9, GAD-7 and K-10 scores were qualified by significant group × time interactions, all F's (df's 1, 181.84–185.61) >7.91, all p's ⩽ 0.001. Between-group comparisons on the PHQ-9, GAD-7 and K-10 scores revealed that post-treatment scores were significantly lower in the treatment group relative to WLCs, with large observed effect sizes (0.85–1.4). Within-group comparisons for the treatment group revealed large effect sizes. The reductions in the control group on these measures were not significant.
Table 3. Observed results for primary and secondary outcome measures and within- and between-group effect sizes following treatment in study 1
s.d., Standard deviation; df, degrees of freedom; CI, confidence interval; r, inter-correlation between pre- and post- treatment scores used to calculate within-group effect sizes; PHQ-9, Patient Health Questionnaire nine-item scale; WLC, wait-list control; GAD-7, Generalized Anxiety Disorder seven-item scale; K-10, Kessler 10-item Psychological Distress scale; BDI-II, Beck Depression Inventory – Second Edition; PSWQ, Penn State Worry Questionnaire; WHODAS-II, 12-item World Health Organization Disability Assessment Schedule II; NEO-FFI-N, NEO-Five Factor Inventory – neuroticism subscale.
Secondary outcome measures and effect sizes
All main effects for BDI-II, PSWQ, WHODAS-II and NEO-FFI-N scores were qualified by significant time × group interactions, all F's (df's 1, 91.53–93.56) >5.17, all p's ⩽ 0.001 (WHODAS-II; p = 0.02)Footnote 2. Between-group comparisons revealed that all post-treatment scores were significantly lower in the treatment group relative to the control group, with medium (0.56; PSWQ) to large (1.13; BDI-II) between-subjects effect sizes being found. This corresponded to large within-group effects in the treatment group. The reductions in the control group were not significant, with the exception of WHODAS-II scores (effect size = 0.30).
Primary outcome measures and effect sizes at 3-month follow-up
Marginal models with group as a fixed factor and time as a repeated factor were conducted separately to compare mean reductions in scores from post-treatment to 3-month follow-up for the treatment group. Baseline scores were entered as a covariate for their respective analysis. For GAD-7 scores (r = 0.67), the main effect of time was significant (F 1,37.36 = 4.39, p = 0.04). The reduction corresponded to a small effect size (0.26). For PHQ-9 scores (r = 0.64) the main effect of time was not significant (F 1,36.56 = 3.04, p = 0.09). For K-10 (r = 0.61) the main effect of time was significant (F 1,36.06 = 5.89, p = 0.02). The reduction corresponded to a small effect size (0.33) (see Table 3 for means and standard deviations).
Clinical significance
Three criteria for clinical significance were employed: first, pre- and post-treatment PHQ-9 and GAD-7 scores were compared with cut-off scores (⩾10) to provide an index of remission. For the treatment group (n = 46), 26 (56.5%) versus 11 (23.9%) met criteria on the GAD-7 at pre-treatment and post-treatment, respectively, and for the PHQ-9, 27 (58.7%) at pre-treatment reduced to eight (17.4%) above cut-off at post-treatment. For the WLC group (n = 53), 27 (50.9%) met criteria on the GAD-7 at pre-treatment, compared with 24 (45.3%) at post-assessment. A total of 39 (73.6%) were above cut-off on the PHQ-9 at pre-treatment, versus 30 (56.6%) post-assessment. The proportion of participants who were in the clinically depressed and anxious range differed according to group at post-treatment [PHQ-9: χ2(1) = 16.01, p < 0.001; GAD-7: χ2(1) = 4.92, p < 0.05] but not at pre-treatment [PHQ-9: χ2(1) = 2.46, p > 0.05; GAD7: χ2(1) = 0.38, p > 0.05].
Second, following Jacobson & Truax (Reference Jacobson and Truax1991), we calculated reliable change index (RCI) values for the PHQ-9 and GAD-7 scores. RCI values were calculated using test–retest reliability values of 0.83 and 0.84 from Spitzer et al. (Reference Spitzer, Kroenke, Williams and Lowe2006) for GAD-7 scores, and Kroenke et al. (Reference Kroenke, Spitzer and Williams2001) for PHQ-9 scores, respectively. In order to calculate standard error of measurement values, standard deviations were derived from the current sample (GAD-7 pre-treatment pooled s.d. = 4.44; PHQ-9 pre-treatment pooled s.d. = 4.43). For PHQ-9 scores, a change score (from pre- to post-treatment) greater than 4.91 was considered reliable change, and for the GAD-7, a change score greater than 5.07 was considered reliable change. Of the treatment group, 21/46 (45.73%) showed reliable improvements on the PHQ-9 compared with 13/53 (24.5%) in the WLC group [χ2(2,93) = 10.00, p < 0.001]. For the GAD-7, 18/46 (39.1%) of the treatment group participants reliably improved, compared with 5/53 (9.4%) in the WLC group [χ2(2,93) = 18.13, p < 0.001]. Notably, five (9.4%) and six (11.3%) WLC participants reported deteriorated PHQ-9 and GAD-7 scores, respectively, at post-treatment.
Third, we compared the proportion of participants with principal diagnoses at pre-treatment and compared this with follow-up diagnoses for the treatment group. At 3-month follow-up, 32/46 participants (69.6%) no longer met criteria for either disorder. A total of seven participants were diagnosed with GAD at 3-month follow-up. Of these, four had initially been diagnosed with co-morbid MDD/GADFootnote 3.
Time spent per participant and patient satisfaction
The clinician and therapist combined spent on average 23.37 min per participant (s.d. = 12.15, range = 7–60 min) on email and telephone contact in the treatment group over the course of the programme. Treatment group participants were asked to provide a rating ranging from 1 to 10 (where 10 = high level of agreement) about how logical the programme was, their confidence that the programme was successful at teaching them techniques for managing symptoms, and their confidence in recommending the programme to a friend with similar concerns. The results were combined to derive a total treatment satisfaction score, which was high on average (mean = 25.64, s.d. = 3.58, where the highest possible combined total score was 30).
Discussion
This study compared a six-lesson clinician-assisted iCBT programme for mixed depression and anxiety with a WLC group. Adherence was high (89%), and the iCBT programme was more efficacious than WLC on all primary and secondary measures of depression, generalized anxiety and functional impairment. Between 40 and 45% of participants in the treatment group showed reliable improvements immediately following treatment. Importantly, gains were maintained at 3-month follow-up for the treatment group, with evidence of further improvements (albeit small effects) in GAD symptoms and general distress between post-treatment and follow-up. Approximately 70% of participants no longer met diagnostic criteria on structured interview at 3-month follow-up. The impact of our iCBT programme on other co-morbid anxiety disorders (e.g. panic disorder and social phobia) is unknown and awaits further investigation. Nevertheless, these results support growing evidence for the efficacy of iCBT for mixed anxiety and depression (Andersson et al. Reference Andersson, Estling, Jakobsson, Cuijpers and Carlbring2011), and for the primary disorders (Perini et al. Reference Perini, Titov and Andrews2009). Future comparisons with single-disorder treatments will test whether this transdiagnostic iCBT programme provides added benefits in terms of efficacy, acceptability, cost-effectiveness and efficiency.
The non-blinded structured interviews at 3-month follow-up, reliance on self-report measures, and short follow-up period are some limitations of study 1. In addition, the use of a WLC comparison group (instead of an active treatment control condition) is a limitation of this study. Of note, approximately 10% of the WLC group deteriorated whilst waiting for treatment. Active treatment controls could be used in future to minimize the possibility of deterioration occurring, and would offer the advantage of ruling out the role of non-specific therapeutic factors accounting for symptom improvement. Another issue is that it is unknown whether these results would generalize to community patient samples outside of a strictly controlled research setting. In order to address this question, we conducted an effectiveness trial of the Worry and Sadness Program in study 2.
Study 2: adherence and effectiveness of the Worry and Sadness Program in a primary care setting
Method and procedure
We aimed to test the effectiveness of the Worry and Sadness Program in www.crufadclinic.org by making it available to the 1800 clinicians registered with the Clinical Research Unit for Anxiety and Depression (CRUfAD) clinic from 8 August 2011 to 15 December 2011. Registered clinicians identify patients that are likely to benefit from the iCBT courses and prescribe a course to their patient, with a prescription that tells the patient how to enrol, and provides a secure passcode linking the patient to the clinician (participants termed ‘prescription patients’).
The iCBT course at the CRUfAD clinic was the same as in study 1, with the following exceptions: patients had 30 days to complete the first two lessons, and were allowed 90 days to complete the entire course. Once patients completed each lesson and downloaded the homework, they were required to book in a date on which they would commence the following lesson. Reminder emails were sent if patients missed the date. Patients completed the PHQ-9, GAD-7 and WHODAS-II prior to commencing lesson 1, and prior to commencing lesson 6. The K-10 was also administered prior to each lesson. Automatic emails were sent to a patient's supervising clinician to report lesson-by-lesson progress on the K-10. Clinicians were also alerted via email if their patients’ K-10 scores rose over 30 (severe range), rose by more than 0.5 s.d., or if patients missed their nominated lesson date.
Results
Participants
There were 136 prescription patients prescribed the Worry and Sadness Program between 8 August 2011 and 15 December 2011. Patients’ mean age was 39.27 years (s.d. = 13.05, range = 18–78), 88 were female (64.7%), and 75 (55.1%) were living in rural Australia. Prescribing clinicians were: general practitioners (n = 59, 43.4%), 39 (28.7%) were psychologists, 23 were medical specialists (16.9%), six were nurses (4.4%), and nine were other allied health specialists (6.6%). Participants’ depression (mean = 14.14, s.d. = 6.33) and anxiety (mean = 12.19, s.d. = 5.43) symptoms were on average in the moderate range.
Primary outcome measures, effect sizes, and adherence
Prescription patients completed on average, 4.21 lessons (s.d. = 1.87, range = 1–6). Out of 136 participants who began the course, 56 completed all six lessons (41.2% adherence). Completers of the programme were highly satisfied. Of the non-completers, 17 (12.5%) completed one lesson only, 17 (12.5%) completed two lessons, 15 (11.0%) completed three lessons, 14 (10.3%) completed four lessons, and 17 (12.5%) completed five lessons.
The ITT marginal model analyses with time as the repeated variable were conducted separately for each of the dependent measures. All main effects of time for each of the dependent variables were significant (F's1,57.14–69.49 > 50.47, all p's < 0.001). Due to the high level of attrition, estimated marginal means and standards errors are reported here (see Table 4 for observed means and standard deviations). For GAD-7 scores (pre-treatment: mean = 12.14, s.e. = 0.47; post-treatment: mean = 6.90, s.e. = 0.60, r = 0.61), the pairwise comparison was significant (t 1,65.35 = 8.94, p < 0.001), corresponding to a large effect [1.05, 95% confidence interval (CI) 0.67–1.42]. For PHQ-9 scores (pre-treatment: mean = 14.04, s.e. = 0.56; post-treatment: mean = 8.32, s.e. = 0.78, r = 0.57) the pairwise comparison was significant (t 1,62.19 = 7.61, p < 0.001), with a large corresponding effect size (0.94, 95% CI 0.56–1.31). For K-10 scores (pre-treatment: mean = 30.41, s.e. = 0.67; post-treatment: mean = 21.94, s.e. = 0.99, r = 0.62) the pairwise comparison was significant (t 1,60.67 = 9.21, p < 0.001), with a large corresponding effect size (1.07, 95% CI 0.69–1.44). For WHODAS-II scores (pre-treatment: mean = 16.20, s.e. = 0.85; post-treatment: mean = 11.26, s.e. = 0.99, r = 0.66) the pairwise comparison was significant (t 1,57.14 = 7.10, p < 0.001). The reduction corresponded to a medium effect size (0.78, 95% CI 0.40–1.15).
Table 4. Observed results for primary outcome measures for total sample, completer and non-completer samples in study 2a
CI, Confidence interval; s.d., standard deviation; K-10, Kessler 10-item Psychological Distress Scale; WHODAS-II, 12-item World Health Organization Disability Assessment Schedule II; n.a., not applicable; GAD-7, Generalized Anxiety Disorder seven-item scale, PHQ-9, Patient Health Questionnaire nine-item scale.
a Completers, n = 57; non-completers, n = 79.
Completers versus non-completers
Adherence to the Worry and Sadness Program by prescription patients was low (41.2%). Therefore, we compared completers versus non-completers to indirectly investigate possible contributing factors for drop-out. Independent samples t tests showed there were no significant differences between completers versus non-completers of the programme in terms of age, pre-treatment K-10, PHQ-9, GAD-7 and WHODAS-II scores (see Table 4). There were no differences in the type of clinicians who prescribed the course to completers versus non-completers as shown by χ2 analyses [χ2(5) = 5.49, p > 0.05]. These variables were also entered along with rurality as predictors in a multivariate logistic regression model predicting completion status (0 = completed less than six lessons, 1 = completed six lessons). No significant predictors emerged (all p's > 0.05). Independent samples t tests compared the completers and non-completers on lesson-by-lesson K-10 scores. There were no differences, suggesting that patients did not drop-out because they were not getting benefit from the course (see Table 4).
Using a conservative s.d. score of 7.5, we measured the number of people who had experienced benefit of at least 1 s.d. on the K-10 prior to drop-out (this analysis only included individuals who completed two or more lessons) (n = 63). Of the non-completers, 24 (30%) had experienced greater than 7.5 points reduction on the K-10 prior to drop-out, suggesting they had experienced benefit from the iCBT programme prior to drop-out.
Discussion
Study 2 was the first investigation of the effectiveness of our iCBT course for mixed anxiety and depression in patients in primary care. Patients who completed the course were highly satisfied with the programme when supervised by their primary care clinician. Although we do not know whether patients in this study registered for the iCBT course as adjuncts to additional treatments (e.g. psychological therapy and/or medications), patients reported similar reductions in depression and anxiety symptoms, distress and disability as in the RCT in study 1, suggesting that the efficacy of this iCBT programme is generalizable to community settings.
Adherence to the programme was much lower for patients supervised in primary care [41% versus 89% in the RCT (study 1)]. This result mirrors the pattern of poorer adherence to psychological treatments in primary care settings (Cuijpers et al. Reference Cuijpers, van Straten, van Schaik and Andersson2009). The completion rates in study 2 are slightly lower than median completion rates (56%) reported in a meta-analysis of computerized CBT treatments (Waller & Gilbody, Reference Waller and Gilbody2009), and are lower than programme adherence for our iCBT programmes for depression (54%) (Williams & Andrews, in press) and GAD (55%) in primary care (Mewton et al. Reference Mewton, Wong and Andrews2012). However, interestingly, Mason & Andrews (E. Mason and G. Andrews, unpublished observations) recently found better adherence to the Worry and Sadness Program in a sample of patients who were referred by primary care practitioners to be monitored and supervised over the web by clinical psychologists and a psychiatrist at CRUfAD (60%; E. Mason and G. Andrews, unpublished observations). CRUfAD's clinicians are highly familiar with the programme and closely monitor and supervise iCBT patients following a standard protocolFootnote 4. It is possible that due to the higher level of complexity (and broader scope) of the transdiagnostic programme, patients need a greater level of monitoring, frequent reminders, and feedback from primary care clinicians to continue with the programme. Future work is needed to improve the techniques that primary care clinicians use to encourage adherence to iCBT courses in community settings (Hilvert-Bruce et al. Reference Hilvert-Bruce, Rossouw, Wong, Sunderland and Andrews2012), particularly for individuals with complex and co-morbid emotional disorders.
It is noteworthy that we found that 30% of the drop-outs of the programme had experienced significant benefit (on the K-10); this suggests that people may drop-out from iCBT in community settings after experiencing some benefit. It is also possible that participants who experienced benefit may have dropped out because they attributed their improvements to other concurrent treatments (e.g. psychotherapy or medications), thereby influencing their decision to discontinue the programme. A limitation of our study was that we did not assess participants’ use of concurrent treatments. Closer monitoring of the use of concurrent treatments whilst taking part in iCBT will shed light on this possibility. Finally, other limitations of study 2 were the reliance on self-report measures and short follow-up period. Without a control group in study 2, we also cannot rule out the possibility that symptom reductions can be accounted for by other factors (e.g. spontaneous recovery).
General discussion
In two studies, we investigated the efficacy of the Worry and Sadness Program – an iCBT programme for mixed anxiety and depression – in both a research RCT setting and in primary care. A key strength of this study was the measurement of both the effectiveness and efficacy of the same iCBT programme across different settings. Overall, results showed that iCBT for depression and anxiety was effective compared with WLC, and the efficacy of this programme was generalizable to patients who completed the programme in primary care, a ‘real-world setting’ supervised by busy practitioners (effect sizes >1.0). Not surprisingly, adherence was better in the RCT, and relatively low (41%) in primary care. This finding may be attributable to observed sample differences between the studies (e.g. lower depression severity, higher mean age and greater proportion of females in the RCT). Individuals who volunteer to participate in RCTs may also be more motivated to adhere and engage in treatment, more receptive to psychotherapeutic interventions and more willing to try psychological techniques suggested in the programme, than those who are prescribed the programme by their primary care practitioner (who may not have even been interested in receiving psychological treatment). Participants in the RCT were also more rigorously assessed at the outset, and were closely supervised compared with those who were referred to treatment from their primary care practitioner, potentially minimizing drop-out. In summary, our findings suggest that we have an effective and accessible iCBT programme that reduces symptoms of co-morbid depression and anxiety. However, adherence differs dramatically depending on clinician guidance and treatment setting (research versus primary care). In future, greater emphasis needs to be placed on changes in the delivery and design of the programme, and education of primary care practitioners to support patients throughout their prescribed programme in an attempt to improve adherence rates so that we can maximize benefit of iCBT in the ‘real world’.
Acknowledgements
We wish to express our gratitude to Dr Matthew Boulton, Dr Alison Mahoney and Dr Elizabeth Mason for assistance with the design of the treatment protocol. We also wish to thank the three anonymous reviewers for their comments on the original version of the manuscript. A.D.W. is supported by a National Health and Medical Research Council of Australia Fellowship (no. 630746).
Declaration of Interest
None.
