Introduction
Cognitive behavioural therapy for people with psychosis (CBTp) is internationally recommended as a clinically and cost-effective intervention (National Institute of Health and Clinical Excellence, NICE, 2009, 2012; Dixon et al., Reference Dixon, Dickerson, Bellack, Bennett, Dickinson and Goldberg2010; Dickerson and Lehman, Reference Dickerson and Lehman2011). Nevertheless, effect sizes are small (0.2 to 0.37), and there is therefore little room for slippage in routine services if the clinical benefit and cost effectiveness is to be replicated. The evidence-base suggests that specific CBTp competences are required for good outcomes, with delivery of the full range of therapy techniques, and a particular emphasis on behavioural components (Durham et al., Reference Durham, Guthrie, Morton, Reid, Treliving and Fowler2003; Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012; Tarrier and Wykes, Reference Tarrier and Wykes2004). Such therapy is unlikely to be delivered competently in routine practice without specific training and high quality, close supervision (Rollinson et al., Reference Rollinson, Haig, Warner, Garety, Kuipers and Freeman2007; Rakovshik and McManus, Reference Rakovshik and McManus2010; Steel, Tarrier, Stahl and Wykes, Reference Steel, Tarrier, Stahl and Wykes2012; Kimhy et al., Reference Kimhy, Tarrier, Essock, Malaspina, Cabannis and Beck2012). A recent pilot evaluation of a training programme designed to equip therapists to deliver evidence-based CBTp showed that successful completers achieved objective competence levels equivalent to research trial standards and went on to deliver NICE compliant CBTp in routine services, as long as they were given time to practise (Jolley et al., Reference Jolley, Onwumere, Kuipers, Craig, Moriarty and Garety2012). The current study aimed to characterize the therapy delivered by therapists who successfully completed the training, and its clinical impact. We also wished to investigate the association of both therapist competence and therapy content, particularly the use of homework, with clinical change.
Method
Participants
Training
Therapists completed 120 hours of face to face teaching and 76 hours of close (using audiorecording) small group and individual supervision over a period of 12–24 months (depending on mode of study) as part of a postgraduate training programme specializing in CBTp. Therapists were required to submit assignments illustrating their work with three out of four closely supervised cases, representing a course of therapy of at least 16 sessions. They were also required to complete at least 476 hours of supervised practice, and 500 hours of private study (including reading, writing reports and essays, and listening to audiorecordings).
Therapists
The two most recent cohorts (completing their training between 2009 and 2012) comprised 19 therapists: 15 clinical psychologists, 2 nurses and 1 occupational therapist. All were working in UK National Health Service or equivalent mental health settings. Eight of the 19 had extended their studies and were still to complete at the time of data collection, and 2 had withdrawn due to a change in personal circumstances, leaving 9 successful completers.
Cases
Therapists submitted one assessment case report and two intervention case reports, each with an illustrative audio recording of their work with cases from their routine practice, selected by the therapist to best demonstrate their skills in Assessment, Formulation and Intervention, respectively. All cases met criteria for a schizophrenia spectrum diagnosis and had current, distressing positive symptoms of psychosis. Only cases with paired patient-reported outcomes were included in the current study. One intervention case was excluded because post-intervention measures were missing, leaving 17 of a possible 18, and only 3 assessment cases included paired outcome data (from a mid-therapy rating), making a total of 20 therapy cases with paired outcome data. Main presenting problems, personal therapy goals, and number of sessions received were rated from case reports.
Measures
Patient Reported Outcome Measures
(PROMS) were employed to assess change in psychotic symptoms and affect. Psychotic symptoms were assessed using the Psychotic Symptoms Rating Scale (PSYRATS; Haddock, McCarron, Tarrier and Faragher, Reference Haddock, McCarron, Tarrier and Faragher1999). The Delusions subscale comprises six items and the Auditory Hallucinations subscale comprises 11 items, all rated from 0 to 4. A range of affect measures was used by therapists, including the Depression, Anxiety, Stress Scales (DASS-21; Lovibond and Lovibond, Reference Lovibond and Lovibond1995a,b); the Beck Depression Inventory II (Beck, Steer and Brown, Reference Beck, Steer and Brown1996), and the Beck Anxiety Inventory (Beck, Epstein, Brown and Steer, Reference Beck, Epstein, Brown and Steer1988), all of which are 21-item measures rating severity of affective symptoms on a 0–3 scale, and the 34 item Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM; Evans et al., Reference Evans, Connell, Barkham, Margison, McGrath and Mellor-Clark2002), a measure of general distress, which generates a mean total score out of 40, based on the mean item scores from 0 to 4. Each score was converted to a percentage of the total score possible, and average change scores were calculated for psychotic symptoms and for affect. For all measures, symptoms were rated over the previous 7–14 days, with higher scores indicating more severe symptomatology.
Therapy competence was assessed using the Revised Cognitive Therapy for Psychosis Rating Scale (R-CTPAS; Rollinson et al., Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008) and the Cognitive Therapy Scale (CTS; Young and Beck, Reference Young and Beck1980). The R-CTPAS comprises 21 therapy components derived from the Fowler, Garety and Kuipers (Reference Fowler, Garety and Kuipers1995) manual. These are rated for competence of delivery (individualized to the client, appropriately matched and paced), and for frequency of occurrence on a −7 (non-adherent and present throughout the session) to +7 (adherent and present throughout the session) scale. The scale is designed in four subsections (Insight oriented; Assessment and formulation; Active Cognitive Strategies; and Relapse Prevention), and forms three factors: Engagement and Assessment; Relapse Prevention; and Formulation and Schema work, the last two factors comprising Active Therapy (Rollinson et al., Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008; Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012). Fowler, Rollinson and French (Reference Fowler, Rollinson and French2011) recommend that the R-CTPAS is used in combination with the CTS, which measures competence in CBT for a range of problems. The CTS rates 11 elements of CBT from 0 (absent) to 6 (expert delivery). Three subscores are obtained: General Therapeutic Skills (GTS, items 1, 2, 5 and 6); Interpersonal effectiveness (IE, items 3 and 4) and Cognitive Therapy Specific items (CBT, 7–11). The competence criteria for audio recordings were: a substantial dose of adherent therapy (score ≥ 3) on specified items of the R-CTPAS (Item 6, Assessment; Item 8 Formulation; and Items 9–11 Cognitive change work, respectively) and no endorsed item scoring below 3 on the CTS. These criteria were chosen to indicate when “full therapy” had occurred (Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012). In line with other psychosis adherence scales, items could be rated as appropriately absent (Haddock et al., Reference Haddock, Devane, Bradshaw, McGovern, Tarrier and Kinderman2001), but did not then contribute to the scale total. Only 4% of CTS items (8/220) were rated in this way; six were from assessment sessions that did not include some of the active CBT techniques, and the remaining two were from formulation sessions that did not include CBT change techniques.
Procedure
Competence ratings were paired with therapy outcomes from the same course of therapy. All assignments, including therapy recordings, were marked by supervisors, and at least one other marker who was blind to therapy outcome. Marking was quality assured by an external examiner. All supervisors and markers were trained in large research trials to objective standards of competence and had significant clinical and teaching experience, specializing in CBTp. Case reports were pseudonymized prior to submission. Once therapist and client data were paired, all identifiers were removed to fully anonymize the database. Ethical approval was granted by the Psychiatry, Nursing and Midwifery Research Ethics Subcommittee of King's College London (Ref. PNM111228).
Analysis
Symptom and adherence scores were normally distributed, except psychotic symptoms scores, which were skewed towards greater severity. Parametric tests were supplemented with a non-parametric Wilcoxon signed ranks test: as the results were identical, only parametric statistics are presented here. A repeated measures ANOVA was employed to examine change in outcome measures by Time (x2, pre or post) and Outcome-type (x2, affect or psychotic symptoms), co-varying for therapist and number of sessions. Effect sizes were calculated from partial Eta squared values. Correlational analyses were used to examine the associations between therapy quality and positive clinical change.
Results
Patient outcomes
Presenting problems were predominantly voices for nine cases, predominantly delusions for seven cases, and both voices and delusions for four cases (see Table 1). Therapy cases experienced a high level of interference with their lives, reflected in their self-reported therapy goals, the most common of which was to be able to leave the house (n = 9). Other therapy goals were to engage in a specific pleasant or valued activity (n = 6); or to manage distressing symptoms more effectively (n = 5). Pre- and post-treatment symptom scores are shown in Table 2, and reflect severe baseline levels of both psychotic and affective symptomatology, with improvement during therapy for almost all cases. Percentage affect ratings reduced from 50.7 (SD 20.9) to 39.0 (SD 18.1, n = 18, ES = 0.6), and percentage psychotic symptom ratings reduced from 74.0 (SD 17.2) to 52.4 (SD 18.5, n = 17, ES = 1.0). Therapy comprised 17 sessions on average (SD = 7.7; n = 17; range 6–33). Repeated measures ANOVA demonstrated that both affective and psychotic symptoms improved significantly over Time (F(1,14) = 74.0, p<.001; ES 0.8) with no effect of Outcome-type (psychotic symptoms or affect; F(1,13) = 7.6, p = .2) or therapist (F(1,13) = 0.2, p = .6). There was no interaction between number of sessions and improvement (F(1,10) = 1.0, p = .3).
Table 1. Client problems and goals
Table 2. Client outcomes
Notes: BDI-II: Beck Depression Inventory II (Beck et al., Reference Beck, Steer and Brown1996); BAI: Beck Anxiety Inventory (Beck et al., Reference Beck, Epstein, Brown and Steer1988); DASS: Depression, Anxiety, Stress Scales (Lovibond and Lovibond, Reference Lovibond and Lovibond1995); CORE-OM: Clinical Outcomes in Routine Evaluation-Outcome Measure (Evans et al., Reference Evans, Connell, Barkham, Margison, McGrath and Mellor-Clark2002); PSYRATS: Psychotic Symptoms Rating Scale (Haddock et al., Reference Haddock, McCarron, Tarrier and Faragher1999); AH: Auditory Hallucinations; D: Delusions
Therapist competence and therapy content
R-CTPAS scores ranged from 5 to 23.5 (Mean 14.0, SD 5.8), with between two and nine items endorsed per session (Mean items endorsed 5.0, SD 2.3), and no overall non-adherent ratings. Item means and endorsement rates are presented in Table 3, with figures from Rollinson et al.'s (Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008) standardization sample for comparison. R-CTS item scores ranged from 3 to 5 (Mean 3.3, SD 0.3). Overall R-CTS mean was 35.6 (SD 5.2). Session content differed as expected according to stage of therapy. Ratings are shown in Table 4.
Table 3. R-CTPAS item means and percentage endorsement rates
Notes: R-CTPAS: Revised Cognitive Therapy for Psychosis Adherence Rating Scale (Rollinson et al., Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008)
Table 4. Mean therapy component ratings by session type
Notes: a,b denote source of difference by post-hoc Tukey test; R-CTS: Revised Cognitive Therapy Scale (excluding “appropriately absent” ratings); R-CTPAS: Revised Cognitive Therapy Adherence Scale (range = −7 to +7 for each item included); CT: Cognitive Therapy
1Non-parametric Kruskal Wallis, p = 0.02 for F2; p = 0.07 for F3.
Patient outcomes and therapist competence
CTS total, subscale and item scores were not associated with either affective or psychotic symptom outcomes (r values < 0.4, p>.1). On the R-CTPAS, no subscale or factor scores were significantly associated with outcome, but there was a tendency for higher scores on the Assessment and Engagement factor of the R-CTPAS to be associated with less improvement in affect (r = −0.5, p = .05; otherwise r values < 0.4, p values > 0.1). Only four participants were not set homework; neither homework setting nor completion were significantly associated with improved outcomes; on the contrary, change in affect was greater for the four participants for whom no homework was set (r = −0.65, p = .003, n = 18; otherwise r values <0.32; p values >.25).
Discussion
Our pilot findings are consistent with the assertion that high quality training in CBT for therapists has a beneficial effect on clinical outcomes for people with psychosis. Significant changes were reported by service users participating in therapy, both in affective symptoms and in psychotic symptoms. Therapy effect sizes, taking account of the within subject design, were medium for affect, and large for psychosis. Our effect sizes must, of course, be interpreted within the context of this study: small numbers, a naturalistic design, self-report data, no control group, and no independent assessors of patient outcomes. They cannot, therefore, be directly attributed to training. Nevertheless, they are substantially greater than those reported in published meta-analyses and, moreover, represent work in routine services, with a group of clients who were severely unwell (almost half of the group were housebound). We did not routinely collect data on duration of problems prior to therapy, and this is a limitation. However, we do know that most of the therapists were working in service settings for people with longer term problems.
Therapists employed a range of CBTp techniques, at a high frequency, with appropriate and discriminable activity at different stages of therapy, as evidenced by the significant differences in key therapist behaviours between assessment, formulation and intervention stages. Homework was set, and completed, in most cases. Competence criteria for the CTS were met in all cases, but average ratings were lower than those reported for therapists treating anxiety and depression by McManus and colleagues (McManus, Westbrook, Vazquez-Montes, Fennell and Kennerley, Reference McManus, Westbrook, Vazquez-Montes, Fennell and Kennerley2010). However, this may represent the reality of psychosis work and it is possible that the competence ratings achieved in anxiety and depression work do not fully generalize to this rather different field. On the psychosis-specific R-CTPAS, criteria for “full therapy” were met for all intervention cases. Active cognitive therapy strategies, in particular, were well represented, with rates up to six times higher than those reported for the PRP trial (Rollinson et al., Reference Rollinson, Smith, Steel, Jolley, Onwumere and Garety2008). It is likely that this reflects the difference between work with a persisting symptom, help-seeking group, compared to a relapsing group, who did not necessarily want therapy (Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012).
Notwithstanding the quality of therapy delivered, higher levels of competence were not associated with improved clinical outcomes in this pilot sample. This held for all components of therapy: neither active, specific CBT change techniques, nor homework setting were associated with improved outcomes. In common with the PRP trial, a higher level of Assessment and Engagement work was weakly associated with poorer outcomes (Dunn et al., Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith2012). Therapist behaviours designed to promote a therapeutic alliance and build rapport, despite being particularly well executed in our pilot sample, were also unrelated to outcome, in contrast to much of the literature (Bentall et al., Reference Bentall, Lewis, Tarrier, Haddock, Drake and Day2003).
Homework setting, again in contrast to the literature, appeared to be associated with worse affective outcomes. This unexpected finding is difficult to interpret with such small numbers, and visual inspection of scores suggested that, as most clients improved during therapy, the association may have been overly influenced by outliers. Nevertheless, the possibility that active engagement in between-session change techniques may sometimes worsen affective outcomes should be considered. This could occur for positive reasons (e.g. reduced avoidance raising anxiety levels) or because of the insensitive use of change techniques (e.g. inculcating doubt in a belief without an acceptable alternative being in place).
The failure to find an association between competence and outcome may be due to lack of variance in competence scores. Because all therapists had reached a relatively high standard of competence, in order to successfully complete the course, the findings relate only to the effects of increasing levels of competence above an accepted benchmark. The inclusion in a future evaluation of failing or novice therapists should clarify this.
The good clinical outcomes attained suggest that the competence criteria adopted are sufficiently high to replicate the clinical and economic effectiveness of CBTp. It is possible that a lower cut-off could be employed, without jeopardizing outcomes. The cut-offs have a weak evidence base across clinical settings (Rakovshik and McManus, Reference Rakovshik and McManus2010) and further evaluation is required to determine whether the benchmarks represent the minimum standards of competence required to carry out clinically effective work with psychosis.
Implications
If the effect sizes found in our pilot study can be reliably replicated in larger groups of trainees, with more stringent methodology, the cost-savings arising from clinical improvements may offset the high costs of training and staff release. This would increase the viability of high-intensity training as one way to improve access to psychological therapies for people with psychosis (Shafran et al., Reference Shafran, Clark, Fairburn, Arntz, Barlow and Ehlers2009; Kimhy et al., Reference Kimhy, Tarrier, Essock, Malaspina, Cabannis and Beck2012). It may also be possible to reduce training costs: our findings suggest no additional clinical benefit of increasing competence above the cut-off criteria. If more efficient techniques for training (e.g. Dimeff et al., Reference Dimeff, Koerner, Woodcock, Beadnell, Brown and Skutch2009; Westbrook, McManus, Clark and Bennett-Levy, Reference Westbrook, McManus, Clark and Bennett-Levy2012) and assessing therapists to be “good enough” (e.g. Fairburn and Cooper, Reference Fairburn and Cooper2011) can be developed, training times and costs could be reduced.
Limitations
As this study is a small, naturalistic pilot, there are inevitably methodological limitations. We used a range of outcome measures, without a control group and relying on self-report data. There was no extended baseline or follow-up of gains made during therapy. Therapy ratings were taken from a single session, selected for assessment, for each case, and thus may not be representative of the full course of therapy delivered. Measures of therapist competence were not taken before training, so baseline levels of competence may already have been high, and the acquisition of competence cannot be assumed to be attributable to training.
Conclusions
Therapists completing the training programme delivered competent CBT for psychosis and achieved substantial clinical improvements for a client group experiencing high levels of interference with their lives as a result of distressing psychotic symptoms. The medium to large therapy effect sizes achieved demonstrate the potential for services to offset the high costs of training through reductions in the costs of care. This delineates a future pathway for increasing access to effective, evidence-based psychological therapies for people with severe mental health problems. A larger, controlled evaluation of CBTp training is now indicated, with consideration of economic implications.
Comments
No Comments have been published for this article.