Introduction
Psychotic disorders and other severe mental illnesses (SMIs) are associated with a high burden of disease, which is reflected by a variety of impairments in social, economic and daily-life functioning. Schizophrenia has the highest total social burden of disease compared with other psychiatric disorders. A great majority of people with schizophrenia are unemployed (72%), have left school at 16 years of age or earlier (58.1%), are living alone in single houses and have shown impairments in self-care (29.8%) (Jablensky et al., Reference Jablensky, McGrath, Herrman, Castle, Gureje, Evans, Carr, Morgan, Korten and Harvey2000; Vargas et al., Reference Vargas, Strassnig, Sabbag, Gould, Durand, Stone, Patterson and Harvey2014).
One of the core features of psychotic disorders and other SMIs is a substantial impairment in cognitive functioning (Goldberg and Green, Reference Goldberg, Green, Davis, Charney, Coyle and Nemeroff2002; Bowie and Harvey, Reference Bowie and Harvey2006). Impairments have been found in different neurocognitive domains, such as attention, working memory, executive functioning and verbal learning (Goldberg and Green, Reference Goldberg, Green, Davis, Charney, Coyle and Nemeroff2002; Bowie and Harvey, Reference Bowie and Harvey2006; Keefe and Harvey, Reference Keefe, Harvey, Geyer and Gross2012), social cognitive functioning (Green and Horan, Reference Green and Horan2010; Keefe and Harvey, Reference Keefe, Harvey, Geyer and Gross2012) and an insight or metacognitive functioning (Brüne et al., Reference Brüne, Dimaggio and Lysaker2011; Lysaker et al., Reference Lysaker, Vohs, Minor, Irarrázaval, Leonhardt, Hamm, Kukla, Popolo, Luther, Buck, Wasmuth and Dimaggio2015). These cognitive limitations have severe implications for employment, social and everyday living skills, and quality of life (Goldberg and Green, Reference Goldberg, Green, Davis, Charney, Coyle and Nemeroff2002; Green and Horan, Reference Green and Horan2010; Brüne et al., Reference Brüne, Dimaggio and Lysaker2011; Keefe and Harvey, Reference Keefe, Harvey, Geyer and Gross2012; Lysaker et al., Reference Lysaker, Vohs, Minor, Irarrázaval, Leonhardt, Hamm, Kukla, Popolo, Luther, Buck, Wasmuth and Dimaggio2015). Cognitive impairments are also related to a reduced response to psychiatric rehabilitation (PR) programs, with a negative impact on outcomes such as work, social skills and self-care (Wykes and Dunn, Reference Wykes and Dunn1992; Smith et al., Reference Smith, Hull, Romanelli, Fertuck and Weiss1999; McGurk and Mueser, Reference McGurk and Mueser2004).
For PR programs, evidence of the effect on functioning varies from indicative for general programs, like the Boston PR Approach (Gigantesco et al., Reference Gigantesco, Vittorielli, Pioli, Falloon, Rossi and Morosini2006; Swildens et al., Reference Swildens, van Busschbach, Michon, Kroon, Koeter, Wiersma and van Os2011), to substantial for the vocational rehabilitation program Individual Placement and Support (IPS), a model of supported employment (SE) (Kinoshita et al., Reference Kinoshita, Furukawa, Kinoshita, Honyashiki, Omori, Marshall, Bond, Huxley, Amano and Kingdon2013; Modini et al., Reference Modini, Tan, Brinchmann, Wang, Killackey, Glozier, Mykletun and Harvey2016). However, even in a highly evidence-based intervention like IPS a large proportion of people with SMI (around 40–50%) that are motivated to participate in regular employment and education do not succeed. Cognitive impairments are one of the obstacles for this (McGurk and Mueser, Reference McGurk and Mueser2004), and training in cognitive skills and strategies might help to overcome the barriers to finding and sustaining employment and education (McGurk et al., Reference McGurk, Mueser, Xie, Welsh, Kaiser, Drake, Becker, Bailey, Fraser, Wolfe and McHugo2015).
Over the past few decades, several cognitive remediation (CR) techniques have been developed for people with SMI. The effect of these CR interventions on cognitive test performance is shown in multiple studies and meta-analyses (Kurtz et al., Reference Kurtz, Moberg, Gur and Gur2001; Pilling et al., Reference Pilling, Bebbington, Kuipers, Garety, Geddes, Martindale, Orbach and Morgan2002; McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b; Grynszpan et al., Reference Grynszpan, Perbal, Pelissolo, Fossati, Jouvent, Dubal and Perez-Diaz2011; Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011). However, there is no consistent evidence that such improvements generalize to better functioning in real life (Pilling et al., Reference Pilling, Bebbington, Kuipers, Garety, Geddes, Martindale, Orbach and Morgan2002; Corrigan et al., Reference Corrigan, Mueser, Bond, Drake and Solomon2008). This reduces the clinical value of CR as a ‘stand-alone’ intervention for people with SMI. Nevertheless, some studies have indicated the efficacy of CR in relation to functioning, provided that CR is combined with PR (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b; Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011).
When further exploring the effectiveness of such a combined intervention with CR and PR, it should be established on which domains of functioning beneficial results can be achieved. In addition, we need to know whether patient characteristics influence the results. For example, participants at a younger age seem to be more responsive to cognitive interventions than those from an older age group (McGurk and Mueser, Reference McGurk and Mueser2008; Kontis et al., Reference Kontis, Huddy, Reeder, Landau and Wykes2012; Radhakrishnan et al., Reference Radhakrishnan, Kiluk and Tsai2016). On the other hand, a meta-analysis on CR in early schizophrenia showed smaller effect sizes for CR in patients with a short duration of illness than in those with chronic schizophrenia (Revell et al., Reference Revell, Neill, Harte, Khan and Drake2015). Furthermore, it is crucial to determine which characteristics of the combined treatment contribute to the optimal results. For example, we should know whether the approach of the CR intervention moderates effects on functioning. CR can be executed by repetitive exercise to push intrinsic learning (drill and practice), by discussion and the use of methods and strategies to improve cognitive skills (strategy coaching), or by implementing a combination of ‘drill and practice’ and strategy-based coaching (drill plus strategy) (Hurford et al., Reference Hurford, Kalkstein and Hurford2011). Results regarding the most efficacious method have been inconclusive so far. Previous research found that ‘drill and practice’-based CR leads to better cognitive outcomes than a strategy-based approach (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b), but to worse outcomes in real-life functioning (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b; Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011).
The present meta-analysis aimed to test the hypothesis that a combined intervention of PR and CR has a superior effect on real-life functioning and global cognition in people with SMI compared with providing a stand-alone PR or CR intervention, and to determine in which domains of functioning this effect is present. In order to establish whether treatment and patient characteristics moderate the effect of this combined PR plus CR treatment, we conducted a meta-regression analysis. The type of CR approach, treatment goal integration, and the age and education level of the participants were expected to moderate the effect of this combined treatment.
Methods
The review is conducted and reported according the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Moher et al., Reference Moher, Liberati, Tetzlaff and Altman2009). Data were pre-specified in a review protocol, which was not registered in a public database.
Search strategy
Studies for the meta-analysis were identified by conducting searches in PsycInfo, PubMed, Cochrane, CINAHL and EMBASE for English language articles published in peer-reviewed journals up until April 2017. Search terms were used for psychotic disorders, SMI and CR training (see online supplementary data Appendix 1 ). In addition, reference lists from included and excluded studies and from previous reviews were searched. When reported data were insufficient for quantitative data extraction, study authors were contacted to request additional study data.
Study inclusion
Decisions on which studies to include were made independently by two authors (L.W., D.D.). Results were compared and disagreements were resolved by consensus. All randomized controlled trials (RCTs) meeting the following criteria were included: (1) a combined PR and CR intervention is compared with a stand-alone PR or CR intervention. Studies with a ‘head to head’ comparison, comparing the combination of PR and CR with the combination of PR and another evidence-based intervention, were excluded; (2) the PR intervention is consistent with the following definition of PR: to help persons with psychiatric disabilities to increase their ability to function successfully and to be satisfied in the environments of their choice with the least amount of ongoing professional intervention (Anthony et al., Reference Anthony, Cohen, Farkas and Gagne2002); (3) the majority of patients (⩾75%) have a diagnosis of a psychotic spectrum disorder or another SMI (according to the definition of the authors of the article concerned); (4) performance is assessed using at least one ‘functioning’ outcome measure.
Quality assessment
For each included study the risk of bias was assessed by the author L.W. using the Cochrane Collaboration risk of the bias assessment tool (Higgins and Green, Reference Higgins and Green2008) (see online supplementary Appendix 2 for the Risk of bias table). Uncertainty concerning a study was discussed with a second author (M.O.) and resolved by consensus. The risk of bias for each included study was rated for six types of bias: selection bias; performance bias (no blinding of participants and assessors); detection bias; attrition bias; reporting bias; and ‘other sources’ of bias. Performance bias was not rated for insufficient blinding of care providers, because in trials on the effectiveness of psychological interventions this is hardly feasible. The risk of bias for each domain was rated as high risk (seriously weakens confidence in the results), low risk (unlikely to seriously alter the results) or unclear.
Confidence in the pooled results for each outcome measure was assessed independently by two authors (L.W., D.D.), using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) method (Guyatt et al., Reference Guyatt, Oxman, Schuneman, Tugwell and Knottnerus2011). This is a structured assessment of the quality of evidence, taking into account the following factors: risk of bias, inconsistency, indirectness, imprecision and publication bias. The results of the two assessors were compared and disagreements were resolved by consensus. See online supplementary Appendix 3. for the GRADE evidence profile.
Measures
Outcome measures for functioning were grouped into three categories: (a) functioning in work or education (employment rate, hours worked, job duration, wages, work/education quality, work interest/motivation); (b) social functioning (scales and subscales covering specific social skills, number and quality of relationships); and (c) community functioning (generic scales covering independent/daily-life functioning, role adjustment and performance, social and occupational functioning). Some overlap exists between the categories of social functioning and community functioning. This could not be avoided because of the scales used in the included studies. See online supplementary Appendix 4 for an overview of all instruments, subscales and parameters for functioning outcomes in the included studies. To analyze the effect on cognitive functioning, data on global cognition were extracted when available.
In addition to general study-characteristics (sample size, risk of bias assessment), data on the following moderator variables were extracted: (1) treatment characteristics: (a) CR with or without focus on enhancement of social cognition; (b) type of CR approach with ‘drill & practice,’ ‘drill & strategy’ or ‘strategy-based’ execution; (c) focus of PR program on the domain ‘vocational skills,’ ‘social skills,’ or ‘community reintegration skills’ (CRS) (e.g. broad focus on a combination of several domains such as work, education, social skills, leisure, daily-living skills, etc.); (d) treatment goal integration between CR and PR, with CR and PR executed separately with different treatment goals, CR and PR executed separately with integrated treatment goals or both interventions merged into one intervention; (e) treatment intensity of the experimental condition; (2) type of control group: (a) single PR intervention; or (b) PR intervention combined with an ‘active’ control condition; and (3) patient characteristics: (a) age; (b) years of education; and (c) severity of disorder at baseline. For severity of disorder at baseline the mean was calculated from all scores based on percentile scores of norm population ‘schizophrenia’ or ‘psychopathology’, derived from multiple scales. For an overview of definitions for outcome measures and focus of PR programs, see Box 1.
Box 1. Definitions for outcome measures and focus of psychiatric rehabilitation programs
Outcome measures for domains of functioning:
• Vocational functioning: employment rate, hours worked, job duration, wages, work/education quality, work interest/motivation
• Social functioning: social skills, relationships
• Community functioning: independent/daily-life functioning, role adjustment and performance, social and occupational functioning
Type of PR programs, with a focus on:
• Vocational skills: specific focus on skills in work and education
• Social skills: specific focus on social skills
• Community reintegration skills (CRS): broad focus on skills in several domains such as work, education, social skills, leisure, daily life skills, etc.)
Statistical analyses
Calculation of effect sizes
For continuous outcomes the standardized mean difference (SMD) was calculated. For dichotomous outcomes the odds ratio (OR) for events was converted to a SMD following the conversion method of Chinn (Chinn, Reference Chinn2000). All outcomes are reported with 95% confidence intervals (CIs). Random effects models were used to calculate the overall effect and were weighted by the inverse variance (Higgins and Green, Reference Higgins and Green2008). For each outcome, missing data were noted. If information on missing cases was not reported, we contacted the authors. When available, data were used that controlled for missing data (for example, imputed using regression methods). In several of the included studies, time-to-event data were reported inconsistently, and often incompletely. Therefore, it was impossible to analyze these results.
Meta-analytic procedure
Meta-analyses were conducted using RevMan 5.2 (Nordic Cochrane Centre, 2012). Statistical heterogeneity was assessed by visual inspection of forest plots (crossing line of no effect), by χ2-tests (assessing the p-value) and by calculating the I 2 statistic, which describes the percentage of observed heterogeneity that would not be expected by chance. If p < 0.10 and I 2 exceeded 50%, we considered heterogeneity to be substantial.
Calculating moderator effects
For outcomes with a significant heterogeneity (p < 0.10; I 2 > 50%), with at least seven studies that provided data, or at least five studies analyzing the outcome and presenting significant treatment effects, a multivariate meta-regression analysis was carried out. The aim of this analysis was to establish whether associations were present between effect sizes and moderator variables when adjusting for other variables.
Results
Data from 23 studies (1819 subjects) on the effect of the combined intervention of PR and CR were included in the meta-analysis. A total of 2246 records were screened for eligibility. After selecting title and abstracts, 208 full-text articles remained for assessment of eligibility. A total of 182 articles were excluded from the analysis because the article was not a (published) RCT (N = 68), the intervention was not a combination of PR and CR (N = 66), the outcomes were not based on functioning measurements (N = 33), the combined intervention was not compared with a single PR or CR intervention (n = 14), or the article was published in a non-English language (N = 1). Three of the included articles were part of one of the included studies. Only one study was found that (also) compared the effect of a combination of PR and CR to a stand-alone CR intervention. Because the results of one single study cannot be pooled, this comparison was excluded from the meta-analysis and the results from this comparison are described separately. Consequently, a meta-analysis was conducted with one comparison, including 23 studies comparing the effect of the combined intervention of PR and CR to a stand-alone PR intervention. A more detailed description of the study selection is presented in the flow chart in Fig. 1.
Fig. 1. Flow Chart selection of studies conform Prisma Guidelines.
Study characteristics
In the 23 included studies (N = 1819) the combination of CR therapy and PR (N = 962) was compared with a stand-alone PR intervention (N = 857).
Patient characteristics
The mean age of all the participants in the studies was 38.15 years (s.d. = 4.00; range = 28.5–44.07), the mean duration of illness was 14.36 years (s.d. = 5.61; range = 5.80–25.09), the mean number of years of education was 12.43 years (s.d. = 1.48; range = 9.75–15.00) and 36.6% of all participants were female (s.d. = 10.10%; range = 10.59–54.05%). The mean percentile score (derived from multiple scales) was 30.45 for baseline symptom severity (s.d. = 19.84; range = 0.20–71.70), 56.17 for baseline severity in cognitive impairment (s.d. = 25.34; range = 8.80–99.90) and 57.24 for baseline functional impairment (s.d. = 29.11; range = 14.70–99.90).
Characteristics of CR interventions
In nine studies (39.1%) the CR was based on a ‘drill and practice’ approach, while 12 studies (52.2%) had a ‘drill plus strategy’-based CR approach and two studies (8.7%) had a ‘strategy-based’ CR approach. In six studies (26.1%) the CR intervention was extended by social cognitive training.
Characteristics of PR programs
In 11 studies (47.8%) the PR intervention within the combined intervention was focused on ‘vocational skills’ (work or education). In two studies (8.7%) the PR intervention was focused on ‘social skills,’ In ten studies (43.5%) a broad PR program was used, with a focus on several domains of ‘CRS.
Characteristics of combined treatment
In six studies (26.1%) PR and CR were executed separately with different outcome goals, whereas in 12 studies (52.2%) PR and CR were executed separately with integrated treatment goals, and in five studies (21.7%) PR and CR were merged into one intervention. The mean treatment intensity for PR programs could not be extracted. The mean treatment intensity in the CR intervention was 3.67 h per week. An overview of the study characteristics is presented in Table 1.
Table 1. Summary of study characteristics and outcome measures
PR = Psychiatric Rehabilitation; CR = cognitive remediation; AC = Attention Control group; CRS = Community Reintegration skills (PR focused on different domains of community functioning); C = cognitive; S = symptoms; F = functioning; DI = duration of illness; ES = effect size; PT = post treatment; FU = follow-up.
a PR approach: SE = supported employment (first-place-then-train strategy, focus on job placement in competitive employment, including IPS); Non-SE = other programs for vocational rehabilitation; Integrated SE = SE augmented with work-related social skills training; Enhanced SE = providing cognitive information to SE-coaches; VR = Vocational Rehabilitation; Conversation S = Conversation skill training (skills like recognizing (non-)verbal cues, starting and ending a conversation, keeping a conversation going); Broad SS = Broad social skill training (different domains of social skills, like: conversation skills, assertiveness and friendship skills); Broad PR = Broad Psychiatric Rehabilitation (focus on multiple skills for community reintegration, including: skills for work, education, social skills, self-care, daily living skills, medication management, leisure, transportation skills, etc).
b Treatment goals: Apart = CRT and PR are separately executed without integration of and/or adaptation of one homogeneous treatment goal; Goal integrated = both CRT and PR are separately executed with treatment goals adapted to each other by e.g. discussion/bridging groups; Merged = Both CRT and PR have been merged into one intervention;
c Treatment Intensity: CRT: on average 3.23 h p/w of treatment and 33 sessions; cut off point: h p/w ⩽3.2; number of sessions ⩽33; Total: on average 4.69 h p/w and 43.88 sessions; cut off point: h p/w < 4.7; number of session <44; when both sessions and h p/w has been indicated, overall judgement of treatment intensity has been based on the measurement which has relatively the highest deviation from the cutoff point;
d Severity Disorder Baseline: All scores based on percentile scores of norm population schizophrenia or psychopathology, lower percentile score is higher baseline severity; Symptoms based on BPRS, PANSS, CDSS, SAPS & SANS; Functioning based on GAF, QLS, QLS-B, MCAS, BCSM, AIPSS, HoNOS & MMLT; Cognition based on MMSE, WAIS III, WRAT-3, WAIS-R & Raven Progressive Matrices; median percentile score Functioning: 47.05; median percentile score Symptoms: 36.2; median percentile score Cognition: 55;
e Years of Education: median years of education is 12.16 years. Cut-off point is 12.16 years;
f Mean age treatment group = 37.947; cut-off point age <37.95 years.
g Study: Both interventions in this study were sufficient CR interventions, both combined with PR, and were both separately compared with PR alone. Based on this, these two interventions were considered as separate comparisons/studies.
Synthesis of results
A meta-analysis was performed on studies comparing the effect of PR plus CR to the effect of PR alone. The general outcomes of the meta-analysis are presented per outcome category. A more detailed description of all outcomes is presented in Table 2. Forest plots of all outcomes are presented in online supplementary Appendix 5, with the type of PR program categorized in separate subgroups.
Table 2. Summary of outcomes and GRADE level
a. Risk of bias; b. Inconsistency; c. Indirectness; d. Imprecision; e. Publication/reporting bias.
The bold values are the ones that are significant.
Global cognition
The combination of CR and PR led to significant favorable outcomes for global cognition [SMD = 0.31 (0.17–0.45); Z = 4.40; p < 0.01].
Vocational functioning
The combination of CR and PR led to significant favorable outcomes for employment rate [SMD = 0.41 (0.10–0.72); Z = 2.62; p < 0.01] compared with a single PR intervention. The combined intervention also had significant favorable outcomes for hours worked [SMD = 0.31 (0.04–0.58); Z = 2.28; p < 0.05], job duration [SMD = 0.48 (0.30–0.67); Z = 5.10; p < 0.01] and quality of performance in work or education [SMD = 0.76 (0.15–1.36); Z = 2.45; p < 0.05]. Only for wages were these effects not significant [SMD = 0.25 (−0.07 to 0.58); Z = 1.53; p = 0.13].
Social functioning
The combination of CR and PR had significant favorable outcomes for social skills [SMD = 0.24 (0.10–0.38); Z = 3.29; p < 0.01) compared with a single PR intervention. However, these effects were not significant for the number and quality of relationships [SMD = 0.07 (−0.18 to 0.33); Z = 0.55; p = 0.58].
Community functioning
The combination of CR and PR had no significant beneficial effects for independent and daily-living skills [SMD = 0.22 (−0.04 to 0.48); Z = 1.63; p = 0.10], role adjustment and performance [SMD = −0.14 (−0.64 to 0.36); Z = 0.54; p = 0.59], and social and occupational functioning [SMD = 0.06 (−0.09 to 0.22); Z = 0.83; p = 0.41] when it was compared with a single PR intervention.
Moderating effect of patient and treatment variables
Patient characteristics
The number of years of education was a significant negative predictor for employment rate (B = −0.94; p < 0.01), wages (B = −0.74; p < 0.01) and hours worked (B = −0.60; p < 0.01), with a cutoff point (median split) of 12.16 years of education. There was no evidence that the age of the participants, or the baseline severity of the disorder, was a key moderator variable for any of the functional outcomes.
Characteristics of CR intervention
The type of CR approach was a significant predictor for employment rate (B = 0.34; p < 0.01), indicating that the effects are stronger when CR is more focused on a ‘drill plus strategy’ approach than a ‘drill and practice’ approach or a ‘strategy-based’ approach. In addition, post-hoc analysis revealed that effects are significantly smaller when CR is focused on a ‘strategy-based’ approach (B = 0.34; p < 0.05), indicating that mere ‘strategy-based’ CR is the least effective type of CR.
Characteristics of combined treatment
Treatment intensity was a significant negative predictor for employment rate (B = −0.74; p < 0.01), with a cutoff point of 3.2 h of CR per week, and 33 CR sessions. Treatment goal integration between PR and CR was a significant positive predictor for employment rate (B = 0.50; p < 0.05). In addition, when the interventions were not merged, but executed separately, it also led to significant beneficial effects on the employment rate (B = 0.77; p < 0.01). Finally, post hoc sensitivity analyses on the beneficial effect of adjunctive CR on vocational outcomes, indicated no meaningful differences in effect for adding CR to SE programs including IPS (n = 8), as to non-SE programs (n = 8). However, sensitivity analyses indicate a meaningful difference of effect on ‘employment rate’ for adding CR to ‘integrated’ types of SE [SMD = −0.19 (−0.41 to 0.03); Z = 1.68, p < 0.09] compared with adding CR to all other types of vocational PR [SMD = 0.56 (0.26–0.86); Z = 3.67, p < 0.01]. There was no evidence that the type of control group (adding an active control condition or not), the type of PR program (focus on vocational, social or CRS), or the addition of social cognitive training to the CR intervention was a moderator variable for any of the outcomes on functioning.
Quality of evidence
Using the GRADE method (Guyatt et al., Reference Guyatt, Oxman, Schuneman, Tugwell and Knottnerus2011), many outcomes were downgraded by at least one level because of the risk of bias (e.g. incomplete outcome data) and imprecision (the analyses included few participants or events). The GRADE quality level was ‘high’ for the evidence of job duration and global cognition, with no serious risk of bias, inconsistency, indirectness or imprecision. A ‘moderate’ quality level was assigned to the evidence of social and occupational functioning. The results for employment rate, hours worked, the number and quality of relationships, and for social skills were of ‘low’ quality. Finally, the outcomes with a ‘very low’ GRADE quality level were: work and education quality, wages, role adjustment and performance, and independent and daily-life functioning. These outcomes were assigned a ‘very low’ level because of (very) serious risk of bias, imprecision, and inconsistency. The GRADE quality levels, and reasons for downgrading, are presented for each outcome in Table 2 and with more detail in online supplementary Appendix 3.
Discussion
The present meta-analysis on the boosting effect of adding CR to different types of PR on real-life functioning in people with SMI is the most comprehensive to date, containing 23 trials and 1819 participants. A former meta-analysis on the topic of augmenting PR with CR (NICE, 2014), contained 533 participants in six studies published between the years 2005 and 2009. This review focused on the boosting effect of adding CR to vocational PR. A more recent meta-analysis on the effect of CR on vocational outcomes (Chan et al., Reference Chan, Hirai and Tsoi2015) contained nine studies. This review combined studies analyzing the effect of stand-alone CR with studies analyzing the boosting effect of adding CR to vocational PR. Results of these analyses on vocational outcomes were inconsistent, with Chan et al., (Reference Chan, Hirai and Tsoi2015) finding beneficial results and NICE (2014) concluding that the evidence was too limited to make a recommendation. The results of the present meta-analysis on multiple functional outcomes confirm earlier findings of moderator analyses in two reviews on the effectiveness of CR, showing favorable effects for the combined intervention of PR and CR on real-life functioning (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b; Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011). Our results indicate that beneficial effects of the combined intervention over PR alone, are most prominent on the domains of vocational and social functioning. In addition, a favorable effect of the combined intervention on global cognition was found. No beneficial effects were found on community functioning. Beneficial effects on vocational outcomes were best achieved in participants with fewer years of education (less than 12 years), lower intensity of the intervention (less than 33 CR sessions and less than 3 h of CR each week), a ‘drill plus strategy’ CR approach, and integration of treatment goals for PR and CR. The type of PR program, focusing on vocational-, social- or a broad range of community skills, was no significant moderator for any of the outcomes.
When focusing on vocational outcomes with a ‘low’ to ‘high’ GRADE quality level of the pooled results (disregarding outcomes with a ‘very low’ quality level), we found that augmenting PR with CR has a medium effect size on the employment rate of participants, the amount of hours that people work and the duration of their job. These results are in line with the fact that strong correlations are found between cognitive limitations in psychosis (and other SMI) and the level of functioning in work situations (McGurk and Meltzer, Reference McGurk and Meltzer2000; Bell et al., Reference Bell, Bryson, Greig, Corcoran and Wexler2001; Gold et al., Reference Gold, Goldberg, McNary, Dixon and Lehman2002; Dickinson et al., Reference Dickinson, Bellack and Gold2007). The beneficial effects of this combination, compared with PR alone, might be explained by: (a) a better uptake of PR lessons because of improved attention and recall; and (b) increased capacity to apply PR lessons in work situations because of improved executive functioning and planning.
The results on social functioning outcomes indicate that providing CR as an adjunct to PR has a small superior effect on social skills. However, the enhancing effect is not seen in the number of relationships. This might indicate that the correlation between cognitive limitations and the number of relationships is smaller than with social skills, which might be due to the fact that the number of relationships can be influenced by many external factors, such as social support (Couture et al., Reference Couture, Penn and Roberts2006), group context (Gest et al., Reference Gest, Graham-Bermann and Hartup2001), living circumstances, and financial strain (Mattsson et al., Reference Mattsson, Topor, Cullberg and Forsell2008).
The lack of favorable effects from providing CR as an adjunct to PR on community functioning, means that increased cognitive skills show no enhanced results on a sample of different scales measuring several domains of community functioning such as occupational functioning (leisure activities), independent daily functioning and ‘role adjustment and performance’. These results are inconsistent with studies finding that cognition and community functioning are directly and indirectly correlated (Green, Reference Green1996; Goldberg and Green, Reference Goldberg, Green, Davis, Charney, Coyle and Nemeroff2002; Aubin et al., Reference Aubin, Stip, Gelinas, Rainville and Chapparo2009; Keefe and Harvey, Reference Keefe, Harvey, Geyer and Gross2012). This inconsistency may be due to differences in the definition of community functioning that are used. Whereas our definition excludes specific vocational outcomes into a separate domain, several of the before mentioned studies use a broad definition of community functioning, including outcomes on employment and education.
Providing CR as an adjunct to PR shows a significant positive effect on global cognition, although the effect size in our review (0.31) is smaller than seen in previous CR reviews of Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011 (0.45) and McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b (0.41). A possible explanation for this might be the differences in data management. Whereas we extracted data on global cognition as originally reported in the selected studies, and therefore could only include data from nine studies, the other two reviews calculated global cognition for all included studies as an average across multiple reported cognitive outcomes. However, the GRADE appraisal of evidence in our review showed that the results on global cognition are ‘highly’ trustworthy. Therefore, the small to moderate favorable effects on global cognition suggest that the cognitive training within the combined intervention works effectively in enhancing neurocognitive functioning, which might be an indication of a working mechanism for the combined intervention. Enhancing the effect of a stand-alone PR intervention with CR might imply that PR lessons can be learned and applied better because of improved cognitive skills. In addition, there might be other working mechanisms that we did not measure, like improved metacognitive functioning, which might account for a better transfer of cognitive gains to real-life functioning.
The type of PR program was no moderator variable for any of the functioning outcomes. This indicates that superior effects of the combination of PR and CR are present both in specialized PR programs (e.g. IPS) as in broad PR programs (e.g. training of skills in various domains of functioning). In addition, the beneficial effects were seen in PR programs with a variety of vocational and social goals, such as PR programs focusing on education, competitive employment, sheltered employment and social skills. Only for one type of vocational PR (called ‘Integrated supported employment’ in which SE is enriched with work-related social skills training), no further augmenting effect of adjunctive CR was present. As a possible explanation for this, the authors (Au et al., Reference Au, Tsang, So, Bell, Cheung, Yiu, Tam and Lee2015; Tsang et al., Reference Tsang, Bell, Cheung, Tam and Yeung2016) mention a plateau effect induced by the work-related social skills training that had already pushed the effects to the upper limit.
The fact that beneficial effects of adjunctive CR are seen in PR programs with a variety of goals might be due to the fact that cognitive skills such as attention, working memory, executive functioning, and verbal learning are basic requirements for vocational and social functioning. When cognitive limitations occur, this hampers real-life functioning in various activities within these domains. By training cognitive skills in relation to the social context where they are required, one can improve real-life functioning in that particular context.
Patient moderator variables
One significant patient moderator was detected for vocational outcomes; results indicate that beneficial effects can best be achieved in participants with fewer years of education. An explanation for this might be that patients with fewer years of education benefit most from developing their cognitive skills during the rehabilitation program. This finding might imply that a possibility for better tailoring of the CR approach to patients with a higher education level is needed. No other patient moderators were found, suggesting that the combined intervention can be offered to a broad range of patients of various ages and baseline severities of the disorder. However, as seen in the Wykes review on the effects of CR (2011), there was a narrow age range, with 65% of our included studies reporting a mean age between 30 and 40 years. Therefore, as stated by Wykes et al., this conclusion is not as robust until other age groups have been tested in future research.
Treatment moderator variables
Several significant treatment moderators were detected for employment outcomes. Our results confirm the finding of Wykes et al. (Reference Wykes, Huddy, Cellard, McGurk and Czobor2011) and McGurk et al. (Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b) that the beneficial effects of PR programs plus CR are best achieved with the ‘drill plus strategy’ approach of CR, rather than with a ‘drill and practice’ approach. Our data add to this that a ‘drill plus strategy’ approach is also superior to a mere ‘strategy-based’ approach. So, both massed practice and strategy training seem to be of critical importance for enhancing levels of daily functioning. The metacognitive component of strategy training is thought to help people to transfer skills from the training setting into their daily lives (Cella et al., Reference Cella, Reeder and Wykes2015).
When augmenting PR by adding CR, results indicate that the best vocational outcomes can be achieved when the interventions are executed separately, but with integrated treatment goals. This means the goal of improving cognitive skills is targeted at the goal of participating in employment or education and vice versa. To boost the effect of PR programs there should be a focus on how cognitive limitations may impede functioning in PR activities and which strategies can be used to deal with this. These results are in line with the review of Medalia and Saperstein (Reference Medalia and Saperstein2013), which demonstrated that CR is most likely to improve functional skills when the cognitive training is linked to the specific demands in real-world settings.
Finally, our results indicate that beneficial vocational effects are best achieved when the treatment intensity of the CR intervention is not too high (less than 33 CR sessions, with less than 3 h of CR per week). This is in line with the finding that a relatively limited amount of CR (e.g. 5–15 h) is sufficient to improve cognitive functioning (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b). Our result might be explained by the fact that a higher CR intensity can be at the expense of the time available for the PR program and that responsiveness in obtaining work can be reduced when the combined treatment is so intensive that it forms an obstacle in searching for jobs, performing job interviews and being available on the job market.
The results of one single study (Bowie et al., Reference Bowie, McGurk, Mausbach, Patterson and Harvey2012) give the first indication that providing PR as an adjunct to CR, compared with a stand-alone CR intervention, also has a favorable effect on vocational outcomes (0.49), social functioning (0.18) and global cognition (0.71). These results confirm earlier findings on moderator variables in two meta-analyses on the effectiveness of CR (McGurk et al., Reference McGurk, Mueser, Feldman, Wolfe and Pascaris2007a, Reference McGurk, Twamley, Sitzer, McHugo and Mueser2007b; Wykes et al., Reference Wykes, Huddy, Cellard, McGurk and Czobor2011), in which improvements in functioning were best achieved by combining PR with CR, to boost their effects. In their meta-analyses all included studies compared the combination of CR and PR with ‘any other intervention.’ The present study adds to these results in that we compared the combined intervention specifically with a stand-alone CR or PR intervention, which enables a tentative interpretation on the adjunctive effect of respectively CR and PR. Enhancing the effect of a stand-alone CR intervention with PR might suggest that improved cognitive skills can transfer better to daily functioning when applied in the real-world directly, within a PR program, and that enhanced cognitive skills become increasingly relevant for clients when they can be employed within the context of a goal that they have prioritized for themselves.
Strengths and limitations
The strength of this study is the focus on the effect of the combined intervention of PR and CR on real-life functioning. This analysis is clinically relevant because it is highly unusual for CR to be a stand-alone intervention in schizophrenia or other SMI these days. The study helps to understand the nature of the relationship between PR and CR with greater specificity, by comparing the combined intervention with a stand-alone PR and CR intervention, by establishing the effect on separate domains of functioning and by assessing moderator variables.
The main limitation of this meta-analysis is the amount and heterogeneity of included data. The number of included studies was scarcely high enough to enable analysis of moderator variables. Most included studies have a small sample size, with a mean sample of 78 participants (range 34–174). This problem is partly resolved by pooling the results of multiple studies, but optimal information size still was not met for several outcomes. In addition, comparing the effects between different domains of real-life functioning should be done with caution. As not all included studies reported outcomes on the separate domains of functioning (vocational-, social-, and community functioning), effects were calculated for different subsets of studies. As a consequence, effect sizes in the separate domains of functioning might be affected by characteristics of the subsets, including the number of studies, the ceiling- and floor effects, and the variability within the subset. Also the effect on global cognition was analyzed for a subset of nine studies, originally reporting on the effects of the combined intervention on global cognition. Another problem is that many studies did not measure outcomes both post-treatment and at follow-up. Only five studies reported their outcomes on both measurement points (Bell et al., Reference Bell, Bryson, Greig, Fiszdon and Wexler2005; McGurk et al., Reference McGurk, Mueser and Pascaris2005; Silverstein et al., Reference Silverstein, Hatashita-Wong, Solak, Uhlhaas, Landa, Wilkniss, Goicochea, Carpiniello, Schenkel, Savitz and Smith2005; Bowie et al., Reference Bowie, McGurk, Mausbach, Patterson and Harvey2012; Tan and King, Reference Tan and King2013).
Directions for future research
Future research should focus more on the effects of CR on real-life functioning, and how these effects can by enhanced. Within studies on the combined intervention of CR and PR, there should be a greater focus on the effects of PR as an adjunct to CR compared with a single CR intervention. In this research, possible working mechanisms for the favorable effects of the combined intervention should be further examined. For example by testing on boosting effects of improved meta-cognition and interaction effects between improved cognitive skills and real-life functioning. In addition, a broad range of age groups should be tested in order to obtain more knowledge on age as a possible moderator variable. Furthermore, these studies should make a clear distinction between post-treatment and follow-up results, in order to evaluate whether potential improved functioning is sustainable. Moreover, larger sample sizes are needed in this specific field of expertise to achieve a better power on separate outcomes. Finally, meta-analyses on the effect of enhancing PR programs with CR should further analyze which specific domains of real-life functioning the potential effects are most prominent.
Conclusions
The results of this study indicate that adding CR to a broad range of PR programs can boost their effects on vocational- and social functioning. First indications were found that a synergetic mechanism also works the other way around, in which effects of CR can be generalized better to real-life functioning by adding a PR program. At this point, no favorable effects on community functioning were detected. In order to improve vocational functioning, it appears important to integrate and apply cognitive goals of CR within the vocational goals of the PR, and vice versa.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S003329171800418X.
Author ORCIDs
Daniëlle van Duin, 0000-0001-8239-9999.
Acknowledgments
We would like to acknowledge Angita Peters for her help with the literature search and Filip Smit and Simone Onrust for their statistical advice. This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflict of interest
Disclosures: The authors have declared that there are no conflicts of interest in relation to the subject of this study.
Ethical standards
All procedures performed in studies (of any of the authors included in the literature review) involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1975 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study (literature review within a guideline) additional formal consent is not required.
