Introduction
Although numerous evidence-based programs (EBPs) for serious mental illnesses (SMI) exist, the majority are not widely available for clinical use (Medalia et al., Reference Medalia, Erlich, Soumet-Leman and Saperstein2019). Emerging data show that cognitive rehabilitation (CR) programs are feasible and effective in community settings (Reeder et al., Reference Reeder, Pile, Crawford, Cella, Rose, Wykes and Callard2016; Roberts et al., Reference Roberts, Penn, Labate, Margolis and Sterne2010). Cognitive enhancement therapy (CET) is a CR program that is efficacious for people with schizophrenia spectrum disorders (Eack et al., Reference Eack, Hogarty, Cooley, DiBarry, Hogarty, Greenwald, Montrose and Keshavan2009) although has not been widely studied in clinical settings. The present study aims to expand upon previous findings to include more detailed implementation information with a more diverse sample of participants and clinicians.
The present study is an observational examination focused on implementation and outcomes of CETCleveland®, a community-based version of CET in an outpatient clinic. We aim to address the following questions: (1) what are the characteristics of the clinicians and setting utilizing CET and how does the implementation process unfold?; (2) do CET participants recruited by clinicians from an outpatient clinic utilize the intervention as intended (i.e. Adoption)?; and (3) do participants benefit from CET as implemented in an outpatient clinic? Given the limited availability of EBPs and lack of published data from naturalistic environments, the current study aims to address these critical gaps. Please see supplementary material for an expanded version of this report with additional data.
Method
Setting and participants
The current study was conducted at a public, urban academic-affiliated medical center in the midwestern United States unaffiliated with the developers of CET which offers extensive mental health services for people with SMI. Interested participants were referred by clinicians for CET based on the following inclusion criteria: age 18 or above, having a diagnosis of SMI, and neurocognitive impairment. SMI was defined by state/federal criteria (i.e. diagnosis of a major mental disorder and functional disability). All participants enrolled in CET were eligible for the study. Thirty-four participants were invited and completed written informed consent and pre- and post- research testing.
Study design and procedures
Data were collected from two cohorts including four groups of CET total from 2016 to 2018. Study data were collected in two sessions, pre- and post-CET participation. Both sessions included cognitive assessments and symptom interviews. Post-testing included a satisfaction measure. Participants were paid $25 for each study session. The process of implementation was recorded utilizing field notes by L.F. throughout the study period. The study intervention was tested observationally by the researchers after patients were already enrolled in the CET program by clinicians.
CET intervention
The current study utilized a community-based version of CET, CETCleveland®, consisting of neuro- and social cognitive training including computerized drill and practice sessions as well as group-based weekly social-cognitive treatment with active coach support throughout. Treatment is structured in 48 weekly sessions over the course of 12 months.
Measures
Neuro- and social cognition
The MATRICS Consensus Cognitive Battery (MCCB) was utilized to assess cognitive functioning. The MCCB assesses speed of processing, attention/vigilance, working memory, verbal working memory, verbal learning, visual learning, reasoning/problem solving, and social cognition. The Continuous Performance Test (CPT) was replaced with the d2 test of attention due to lack of computer access during testing sessions.
Symptomatology
The Scale for Assessment of Negative Symptoms (SANS) and the Scale for Assessment of Positive Symptoms (SAPS) were used to assess symptomatology.
CET satisfaction
The CET Satisfaction Survey is a 13-item Likert-type questionnaire that assesses participant satisfaction from completely disagree to completely agree. This measure was developed for use in the current study based on the core components of CET (i.e. group activities, coaching and homework).
Data analyses
Descriptive statistics were utilized for demographics, adherence and satisfaction. Implementation information was reported using qualitative description after the first author observed the implementation process, recorded detailed field notes, and later transcribed using a word processor. We included participants with complete pre- to post-data to assess clinical outcomes (cognition and symptoms) with a series of repeated measures t-tests including both CET graduates and drop-outs. All participants in our sample engaged in symptom interviews and cognitive testing. Missing data points were excluded using pairwise deletion. Quantitative data were analysed using SPSS software.
Results
Implementation
Clinician characteristics
During the 2-year study, coaches included N = 11 generalist clinicians who participated in CET training/certification. These included n = 4 Bachelor’s level clinicians, n = 2 Master’s level clinicians, n = 2 psychiatrists, n = 2 nurses, and n = 1 music therapist. The first training year began with N = 7 clinicians participating in certification. Before the second year, n = 1 coach left the agency and n = 2 coaches decided not to continue coaching due to other professional obligations (retention=57.14%). Beginning in the second year, n = 4 new coaches began training. After the second training year, n = 1 coach and n = 1 coach left the agency (new coach retention=75%). Retention for CET certified coaches for 2 years overall was 54.54%.
Intervention training and implementation process
CET implementation was lead in 2012 by three clinicians who engaged in independent learning by purchasing the CET manual and offered educational didactic seminars to interested staff which lacked engagement and did not expand CET beyond the pilot stage. The medical center was awarded a regional foundation grant that allowed for external training to promote program growth and purchase of materials. Training and certification were provided by the Center for Cognition and Recovery (CCR), a non-profit agency that offers CET training using the CETCleveland® manual beginning from 2016 to 2017. The first training year included weekly tele-observation to co-facilitate, monitor fidelity, and provide feedback along with monthly in-person training.
Between the first and second years of implementation, groups moved from two separate locations to one centralized location within a psychiatric rehabilitation program. This move was more convenient for participants and helped overcome the barrier of transportation. To overcome the barrier of coach turnover, two coaches completed training to become CET trainers to have the ability to continuously train future CET coaches, the ‘train the trainer’ method.
Participant demographics
The current study includes N = 34 participants with completed outcome data. Of these, n = 27 participants completed CET and n = 7 dropped out. Participants had a mean age of 39.47 years (SD = 13.14) and were majority male (n = 25, 74.5%). Most participants had a schizophrenia spectrum diagnosis (n = 30, 88.3%). Comorbid diagnoses included mood disorders (n = 8, 23.5%), anxiety disorders (n = 5, 14.7%), PTSD (n = 3, 8.8%) and ADHD (n = 1, 2.9%). The majority of participants identified as African American or Black (n = 18, 52.9%). See Table 1 for full demographic information.
Table 1. Demographic characteristics of participants
Categories that do not sum to 100% indicate missing data.
Intervention adoption
CET satisfaction surveys indicated that the overall satisfaction was favorable, with at least 79.31% positive responses to each question. Participant retention was 79.40%, with 27 of 34 participants graduating. Participants attended a mean of 39.88 days (SD = 6.42) of 55 total sessions and 80.62% of sessions.
Intervention outcomes
Participants significantly improved in three of ten neurocognitive areas, verbal learning, visual learning, and attention. Participants significantly improved in three of five negative symptom subscales, avolition, anhedonia/asociality, and attention. Participants significantly improved in three of four positive symptoms hallucinations, bizarre behaviour, and thought disorder.
Discussion and implications for practice
This study supports implementation of a community-based cognitive enhancement therapy program, CETCleveland®, and reports preliminary positive outcomes in an outpatient psychiatric rehabilitation program in an academic-affiliated medical center. The current study adds to the extremely limited implementation data for evidence-based SMI interventions and highlights CR as an EBP with limited reports in naturalistic settings.
The current study described successful CET early Implementation which was bolstered with significant clinician training. The study setting was suitable for implementation because it has resources to bolster successful implementation. Previously reported training methods for successful implementation were used in this study, including written materials, train-the-trainer, and expert guidance (Herschell et al., Reference Herschell, Kolko, Baumann and Davis2010). This structured training appeared to be a facilitator to the barrier of lacking staff engagement and program growth. In our study, the train-the-trainer method allowed new coaches to be continuously trained and was helpful to overcome staff turnover. In addition to implementation information, our study provides preliminary positive outcomes for CET in a clinical setting.
Our study described CET utilization (i.e. Adoption) in an outpatient program. The majority of participants had diagnoses consistent with published CET samples (i.e. schizophrenia or schizoaffective disorder; Eack et al., Reference Eack, Hogarty, Cooley, DiBarry, Hogarty, Greenwald, Montrose and Keshavan2009) although the current study included a wider range of participants with other SMIs (i.e. PTSD, bipolar disorder) and comorbid mental illnesses. We further extend existing CET data with a racially and ethnoculturally diverse sample, as published CET studies include majority Caucasian samples. Participants reported positive satisfaction rates, adding to a recently published study describing CET as an empowering experience (Faith et al., Reference Faith, Collins, Decker, Grove, Jarvis and Rempfer2019). Additionally, CET appeared to be utilized as intended by participants with high attendance and acceptable retention. These study findings support successful Adoption of CET in a clinical setting.
This study has several strengths, including high external validity regarding clinical adoption of EBPs. Our data add to the limited information available concerning implementation and effectiveness of EBPs for SMI beyond randomized controlled trials. There are also limitations relevant to the current findings. First, the use of multiple t-tests may have increased the possibility of Type I errors. Second, this is an uncontrolled observational study and we are unable to conclude that participant improvements represent effects beyond treatment-as-usual. Additionally, as an unblinded study, researcher expectancy effects are a potential source of bias. The current study used the 12-month version of CET, CETCleveland®, which has some changes in manual content; previous studies report efficacy of 18 months of CET, more research is needed to distinguish these two versions. We were limited to symptom and cognitive measures. Finally, our setting was an academic-affiliated medical center that had resources that may not be accessible in other clinics.
Conclusion and clinical implications
Our results demonstrated successful early implementation and provided preliminary support for clinical utilization of CET. This study has important implications for clinicians planning to implement CET and other EBPs. We demonstrated that CET can be feasibly implemented after addressing implementation barriers such as funding, program growth, and staff turnover. CET effectiveness was promising when delivered by trained clinicians with a diverse group of adults with SMI.
Future studies should continue to investigate effectiveness and implementation of EBPs for people with SMIs, as access to rehabilitation programs continues to be low (Medalia et al., Reference Medalia, Erlich, Soumet-Leman and Saperstein2019). Implementation studies should investigate a variety of settings to elucidate unique challenges for different environments (e.g. rural non-profit clinics), additional measures (e.g. functional outcomes), and follow-up assessment for long-term program adoption. Given its potential and early promising findings, the train-the-trainer method specifically should be further studied within implementation designs to elucidate its impact. Effectiveness studies may include a larger sample with an equivalent comparison group to increase the generalizability of the findings, increase power, and differentiate non-specific effects of the intervention as well as more explicit consideration of individual and cultural diversity factors.
Acknowledgements
We would like to thank the participants for their contributions to the study.
Financial support
This work was supported by the University of Missouri-Kansas City School of Graduate Studies; University of Missouri-Kansas City Department of Psychology and Friends of Psychiatry.
Conflicts of interest
The authors declare none.
Ethics statement
The study was approved by the relevant institutional review board.
Data availability statement
The authors have not made their data available.
Author contributions
Laura Faith: Conceptualization (lead), Data curation (lead), Formal analysis (lead), Funding acquisition (equal), Investigation (lead), Methodology (equal), Project administration (lead), Resources (equal), Supervision (supporting), Writing – original draft (lead), Writing – review & editing (equal); Amber Grove: Conceptualization (supporting), Data curation (equal), Formal analysis (supporting), Investigation (supporting), Methodology (supporting), Project administration (equal), Resources (equal), Writing – original draft (supporting), Writing – review & editing (equal); Stephen Jarvis: Conceptualization (lead), Data curation (supporting), Funding acquisition (lead), Investigation (lead), Methodology (lead), Project administration (supporting), Resources (lead), Supervision (equal), Writing – original draft (supporting), Writing – review & editing (supporting); Melisa Rempfer: Conceptualization (lead), Data curation (equal), Formal analysis (supporting), Funding acquisition (lead), Investigation (equal), Methodology (equal), Project administration (supporting), Resources (equal), Supervision (lead), Writing – original draft (equal), Writing – review & editing (equal).
Supplementary material
To view supplementary material for this article, please visit: https://doi.org/10.1017/S1352465822000017
Comments
No Comments have been published for this article.