Introduction
Anxiety disorders (as specified in the Diagnostic and Statistical Manual of Mental Disorders, fifth edition: DSM-5; American Psychiatric Association, 2013) are the most prevalent mental health conditions in children and adolescents (Sadler et al., Reference Sadler, Vizard, Ford, Marchesell, Pearce, Mandalia and Goodman2018). Childhood anxiety disorders include separation anxiety disorder, social phobia, obsessive compulsive disorder, specific phobias, panic disorder and generalised anxiety disorder (Albano et al., Reference Albano, Chorpita and Barlow2003).
A survey of 9280 adults (Kessler et al., Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005), and 10,123 adolescents (Merikangas et al., Reference Merikangas, He, Burstein, Swanson, Avenevoli, Cui and Swendsen2010) found a lifetime prevalence rate of 28.8–31.9%, with most people reporting onset in childhood (typically in the age range of 6–11 years).
Anxiety disorders can have wide-ranging negative effects on children (e.g. school, relationships) and restrict their development (Bastiaansen et al., Reference Bastiaansen, Koot, Ferdinand and Verhulst2004). They are best treated early in life as this significantly reduces the likelihood of a continued need for support from services long-term when progressing to adulthood (Barrett et al., Reference Barrett, Duffy, Dadds and Rapee2001; Saavedra et al., Reference Saavedra, Silverman, Morgan-Lopez and Kurtines2010); however, early treatment is uncommon (Essau, Reference Essau2005). This is significant considering the cost of anxiety disorders clinically for patient quality of life (Olatunji et al., Reference Olatunji, Cisler and Tolin2007) and financially for society.
The current guidance by the National Institute of Clinical Excellence (NICE) for treatment of anxiety disorders in childhood and adolescence in the UK is the first-line use of cognitive behavioural therapy (CBT) (Beck et al., Reference Beck, Emery and Greenberg1985). CBT is a psychological treatment that focuses on identifying and challenging maladaptive cognitions and behaviours. Although NICE recommendations for managing anxiety primarily relate to adult presentations, applicable guidelines stipulate that children who suffer with an anxiety disorder should be offered CBT (NICE, 2013), as it is currently the most evidenced and effective psychological treatment available (Cartwright-Hatton et al., Reference Cartwright-Hatton, Roberts, Chitsabesan, Fothergill and Harrington2004). NICE guidance (NICE, 2013) indicates that other psychological interventions developed for adults (e.g. short-term psychodynamic psychotherapy) may be considered for young people (typically older adolescents) who are judged to have the emotional and cognitive capacity to engage with such interventions.
CBT can be delivered in individual or group format, and consideration should be given to involving parents or carers (NICE, 2013). Current published literature shows similar effectiveness levels in the short term (follow-up periods of up to 12 months) between individual and group CBT for children aged 5–15 years (Flannery-Schroeder et al., Reference Flannery-Schroeder and Kendall2000; Flannery-Schroeder et al., Reference Flannery-Schroeder, Choudhury and Kendall2005; McKinnon et al., Reference McKinnon, Keers, Coleman, Lester, Roberts, Arendt and Hudson2018; Wergeland et al., Reference Wergeland, Fjermestad, Marin, Haugland, Bjaastad, Oeding and Heiervang2014). Group CBT has become popular in NHS Child and Adolescent Mental Health Service (CAMHS) contexts due to its perceived equivalence to individualised treatments and the needs of services to meet national targets and be seen to be treating an increasing number of children (NHS England, Reference England2016).
An evidenced protocol for treating childhood anxiety is Coping Cat (Crawley et al., Reference Crawley, Kendall, Benjamin, Brodman, Wei, Beidas and Mauro2013; Podell et al., Reference Podell, Mychailyszyn, Edmunds, Puleo and Kendall2010). Coping Cat is a manualised workbook-based intervention designed for individual or group CBT with children between the ages of 7 and 13 years. It was developed originally as a 16-week programme, followed by an 8-week brief version (Crawley et al., Reference Crawley, Kendall, Benjamin, Brodman, Wei, Beidas and Mauro2013). Both programmes include essential components of traditional CBT: psychoeducation, behaviour experiments and exposure, cognitive restructuring, along with added somatic management which may not always be found in manualised CBT programmes, such as relaxation techniques. There is an emphasis on parents or caregivers to take a primary role in assisting their children, and parent sessions are offered to facilitate understanding of the development and maintenance of their child’s anxiety.
Coping Cat is well supported by research evidence, and it is recommended for use by the US government-funded California Evidence-Based Clearinghouse for Child Welfare (CEBC). A meta-analysis of 19 trials, and two subsequent trials, have compared Coping Cat with no treatment and alternative treatments. They found that Coping Cat was substantially more effective in reducing anxiety compared with no treatment, and slightly more effective than alternative treatments (Lenz, Reference Lenz2015; Silk et al., Reference Silk, Tan, Ladouceur, Meller, Siegle, McMakin and Ryan2018; Villabø et al., Reference Villabø, Narayanan, Compton, Kendall and Neumer2018).
Based on this evidence, a CAMHS service has developed an adapted version for group anxiety treatment, entitled ‘Anxiety Gremlins’. Anxiety Gremlins has many similarities to brief Coping Cat: it is given to the same age group for a similar number of sessions and follows CBT principles. There are also some differences; in Coping Cat, the therapist has a manual and the children work through a standardised workbook each session, with fidelity assessed through tapes and supervision. In Anxiety Gremlins, the therapist has a session plan with aims, objectives and session tasks, and the children have worksheets to complete in a pack. The Coping Cat programme recommends group sizes of four to five children, whereas the Anxiety Gremlins programme typically invites a larger number to groups. These changes reflected local drivers [the need for a lower-intensity and more-accessible (higher capacity) first-line intervention] plus practice-based evidence within the local service – where Coping Cat had been piloted at one site, alongside the Anxiety Gremlins modification. Reviewing experiences of facilitators and feedback from young people within the service, the Anxiety Gremlins model was preferred – for its greater flexibility (versus the highly structured Coping Cat protocol) and perceived equity in outcomes. These local adaptations are consistent with guidance for improving access to treatment and a broader (national) trend towards offering brief, low-intensity formats of CBT to children and young people (typically, derivations of the Coping Cat programme; Cresswell et al., Reference Creswell, Waite and Cooper2014) as a routine, first-line intervention.
In research conditions, the Coping Cat programme has demonstrated efficacy; however, when used in practice, a community clinic study found that the clinical anxiety outcomes for Coping Cat (as a manualised, evidence-based treatment) were no better than those observed in usual care (Southam-Gerow et al., Reference Southam-Gerow, Weisz, Chu, McLeod, Gordis and Connor-Smith2010). No evaluation has been conducted into the Anxiety Gremlins adaptation to the programme in practice, and as the programme had been running for several years, it was determined that an evaluation was needed to assess clinical effectiveness.
Aims
The aims of this evaluation were to:
(i) Determine the effectiveness of the Anxiety Gremlins group for treating childhood anxiety disorders.
(ii) Identify factors that were helpful or hindering to the success of the Anxiety Gremlins group.
(iii) Generate recommendations based on the above for staff at CAMHS.
Method
Participant characteristics
Participants were 36 children (19 boys, 17 girls), with a mean age of 10.8 years (range 8–13 years). Social economic status (SES) of participants was estimated using postcode estimates (Smith et al., Reference Smith, Noble, Noble, Wright, McLennan and Plunkett2015). The average sample score was 7.5 (1, most deprived; 10 least deprived), with 27.8% of the sample scoring in the least deprived 10% of SES scores (10), and 19.4% scoring in the lower half of scores (1–5).
Children were referred to the CAMHS in 2018 and 2019 from a range of sources, such as their GP, parent or other professional. Treatment allocation was decided through a multi-disciplinary team discussion, based on an initial face-to-face assessment of presenting difficulties; the Anxiety Gremlins group is the first-line intervention for children with mild-to-moderate anxiety, and the clinicians preferentially recommend this option (for children meeting eligibility criteria) as it is the most rapidly accessible treatment in the service.
Children were eligible for the Anxiety Gremlins intervention if they had mild to moderate anxiety as their main concern and consented to group therapy. Children were excluded if they had active suicidal ideation or had been previously offered CBT. Consideration was given to potential co-morbidities or neurodevelopmental difficulties (whether formally diagnosed or indicated through the assessment process) that may impact upon group suitability, but there were no formal exclusion criteria around this. Children not considered suitable for the group were offered individual interventions or non-directive (e.g. art therapy) groups – and parents may be offered individual or group-based parenting programmes.
Intervention: Anxiety Gremlins
An Anxiety Gremlins session plan can be found in Supplementary File 1. The 8-session intervention (two sessions with parents, six sessions with children) addresses anxiety from a CBT perspective, with a focus on (1) recognising how thoughts can feed anxiety experiences and their behavioural consequences, (2) achieving desired outcomes through positive changes in thinking, and (3) learning behavioural relaxation skills.
Evaluation design
A trainee clinical psychologist joined a clinical psychologist and mental health nurse from a CAMHS to observe and assist with an Anxiety Gremlins group: the ‘observed group’. Service data were collected on children across four separate group interventions delivering the same programme. Groups contained 7 (observed group), 15, 6 and 8 participants, respectively, with an average of 9 per group. The three remaining groups were not observed, and their data accessed retrospectively.
This evaluation used a mixed methods sequential explanatory design (Creswell et al., Reference Creswell, Klassen, Plano Clark and Smith2011). Outcome measures were collected from the groups, and then these data were used to inform facilitator interviews. The quantitative data and qualitative (facilitator-interview) data were then brought together for interpretation (Fig. 1).
Figure 1. The sequential exploratory design for the evaluation.
Aim (i) – Determine effectiveness: quantitative method
Outcome measures
Group effectiveness was measured using the following outcome measures:
Revised Children’s Anxiety and Depression Scale (RCADS) (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000)
The RCADS is a 47-item self-report questionnaire with six subscales covering anxiety disorders and major depressive disorder. It yields a ‘Total Anxiety’ score, used in this evaluation. Items are rated on a 4-point Likert scale from 0 (‘never’) to 3 (‘always’). Raw scores are normed and converted to scaled scores, and clinical thresholds established against the DSM-5 (American Psychiatric Association, 2013). The RCADS is a NICE-recommended valid and reliable psychometric screening (but not diagnostic) tool used to monitor children who may have co-morbid depression or other anxiety disorders (Chorpita et al., Reference Chorpita, Moffitt and Gray2005; NICE, 2013).
Revised Children’s Anxiety and Depression Scale – Parent (RCADS-P) (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000)
The RCADS-P is completed by the parents/carers of a child, with scales the same as the RCADS. Two studies of the psychometric properties in clinical (Ebesutani et al., Reference Ebesutani, Bernstein, Nakamura, Chorpita and Weisz2010) and school (Ebesutani et al., Reference Ebesutani, Chorpita, Higa-McMillan, Nakamura, Regan and Lynch2011) samples with parents have shown high internal consistency and validity.
Child Outcome Rating Scale (CORS) (Miller et al., Reference Miller, Duncan, Brown, Sparks and Claud2003)
The CORS is a brief, 4-item session-by-session measure designed to assess areas of life functioning that are sensitive to therapeutic change. Items are rated from 0 to 10 on visual analogue scales, with higher scores indicating more positive outcomes. The CORS has demonstrated good reliability (α = .84) in a US population of children receiving community mental health services (Bringhurst et al., Reference Bringhurst, Watson, Miller and Duncan2006; Campbell and Hemsley, Reference Campbell and Hemsley2009).
Child Group Session Rating Scale (CGSRS) (Duncan et al., Reference Duncan, Miller, Sparks and Murphy2011)
The CGSRS, adapted for groups of young people aged 6–12 years from the Session Rating Scale (SRS) (Duncan et al., Reference Duncan, Miller, Sparks, Claud, Reynolds, Brown and Johnson2003), is a 4-item visual analogue scale similar to the CORS. It is designed to be a brief tool to measure group-therapy alliance, with evidence of reliability (.86–.90) albeit in an adult population (Quirk et al., Reference Quirk, Miller, Duncan and Owen2013).
Data analysis
All groups
Children’s RCADS total anxiety scores were collected before and after attending the Anxiety Gremlins group programme. At the aggregate level, effectiveness in anxiety symptom reduction was assessed using raw and standardised mean differences (effect sizes) between pre- and post-group scores. Significance testing of overall differences was not focal, although 95% confidence intervals (CIs) are reported for mean differences (enabling the reader to identify statistically significant differences, where present, and understand the degree of imprecision around point-estimates). Effect sizes were estimated with reference to formulae from Lakens (Reference Lakens2013). At the individual level, effectiveness was assessed through the reliable change index (RCI). The RCI enables us to determine whether the magnitude of change for a given client is statistically reliable and clinically significant (Jacobson et al., Reference Jacobson, Follette and Revenstorf1984; Jacobson and Truax, Reference Jacobson and Truax1991) – providing greater insight into the practical meaning of results for clients accessing the service. Based on a reliability of .94 for the total anxiety scale of the RCADS (Kösters et al., Reference Kösters, Chinapaw, Zwaanswijk, van der Wal and Koot2015), a change of 7 points or greater in either direction can be considered reliable change.
Observed group
Additional routinely used service measures were analysed to supplement the measurement of effectiveness for children in the observed group (see Fig. 2). This included parental ratings of anxiety symptom reduction (RCADS-P), life functioning (CORS) and therapeutic alliance (CGSRS). Again, significance testing of overall differences was not focal – particularly here, given the reduced sample size available for analyses.
Figure 2. Assessment schedule for children in the observed group, indicating which measures were taken at each timepoint/session (P, parent; C, child; numbers indicate session numbers; 2 parent sessions and 6 child sessions).
RCADS-P data were analysed in the same way as RCADS data, estimating both aggregate- and individual-level differences between pre- versus post-group scores). For RCI analyses, the reliability of the RCADS-P is reported as .93 (Ebesutani et al., Reference Ebesutani, Bernstein, Nakamura, Chorpita and Weisz2010), therefore a change of 8 or more would be considered reliable. CORS and CGSRS scores were assessed at the aggregate level only, examining raw and standardised mean differences between first and last measurements (i.e. scores in session 1 versus session 6).
Aim (ii) – Helpful and hindering factors, qualitative method
Factors that were helpful or hindering to the process and outcomes of the observed group were explored through feedback interviews with two group facilitators [a clinical psychologist (working in the service for 2 years) and a mental health nurse (in the service for 6 months)]. These interviews were conducted 1 month after the end of the observed group, with each facilitator interviewed separately on the same day. Each interview was scheduled for 30 minutes in length (see Supplementary File 2 for interview schedule). Interviewees were first asked the interview questions, some outcome measures RCADS scores were shared, and then interviewees were invited to elaborate on their responses.
The interviews were analysed using deductive content analysis as outlined by Elo and Kyngas (Reference Elo and Kyngas2008) – this approach maps to the focused and purposive nature of the evaluation, ensuring that analyses attended to a priori questions regarding factors that were helpful or hindering to group processes and outcomes (successful working). The transcripts were read through several times, and then initial codes were generated to fit the following pre-assigned categories (hindering factors, helpful factors). Themes were then generated based on these codes.
Interviews were conducted and analysed by the first author, who also had a peripheral role in facilitating the group (observed group 1) whilst on placement as a trainee clinical psychologist within the service. Given the first author’s dual role, the possibility of positive bias is present.
Aim (iii) – Summarising and generating recommendations
Quantitative and qualitative findings were synthesised, and the findings summarised, with recommendations made at the end of the report.
Participant flow
Across four iterations of the group, 36 children were invited to participate in the Anxiety Gremlins intervention. Figure 3 shows the flow of participants through the groups, indicating the numbers of children invited/retained over the six child-group sessions. Attrition was low overall, with 32 children attending first and last sessions. Figure 3 also depicts levels of attendance for parents of children participating in the Anxiety Gremlins group (parents were invited to attend two parent-group sessions).
Figure 3. Participant flow.
Missing data
The 32 participants who completed Anxiety Gremlins each had a full dataset of child RCADS scores pre- and post-intervention. Of the participants in the observed group where extended data were collected, 93% of CORS and CGSRS data were collected with missing data occurring due to non-attendance (twice) and researcher error (once). The RCADS-P of the observed group included a larger proportion of missing data, as only four of the six parents attended both parent sessions.
Results
Aim (i): Determine effectiveness, quantitative method
Anxiety symptoms
Child
RCADS Total Anxiety scores in Child Session 1 (C1) were compared with those in Child Session 6 (C6). A mean difference of 0.62 (95% CI: –2.04, 3.28) was found between the scores in C1 (M = 58.31, SD = 13.50) and C6 (M = 57.69, SD = 15.54); this equated to a negligible effect size [Hedges’ g av (standardised mean difference for correlated samples) = 0.04]. The Common Language effect size indicates that the likelihood of a child scoring lower for anxiety at final versus first group-session was 53%.
The RCI was used to calculate the difference between scores in C1 and C6 in individual children. Of the 32 children who completed the anxiety gremlins intervention, 22 (68%) made no significant change, five (16%) made a significant improvement, and five (16%) had a significant deterioration.
The RCI was used to measure whether the 10 (31%) children who made reliable change crossed the clinical threshold (see Fig. 4). Three children deteriorated across the clinical threshold, moving from non-clinical scores in C1, to clinical scores in C6. One child improved across the clinical threshold, moving from a clinical score in C1, to a non-clinical score in C6. Of the three children who deteriorated, all entered the group with co-morbidities with major depressive disorder in C1. The child that improved entered the group only with anxiety in C1.
Figure 4. Children who crossed the clinical threshold are represented by the red dashed line. A higher score represents more clinically distressing anxiety.
Parent
RCADS-P Total Anxiety scores in Parent Session 1 (P1) were compared with those in Parent Session 2 (P2). A mean difference of 18.5 (95% CI: –7.12, 44.12) was found between the scores in P1 (M = 101.50, SD = 9.98) and P2 (M = 83.00, SD = 8.41); this equated to a large effect size (Hedges’ g av = 1.46). The Common Language effect size indicates that the likelihood of a parent scoring their child as lower in anxiety at final (P2) versus first session (P1) was 87%.
Scores were also measured using the RCI. Of the four parents who completed both sessions, one (25%) reported no significant change, and three (75%) reported a significant improvement. We then measured whether the parent-reported change-scores crossed the clinical threshold; however, none did, as all parents rated their child’s anxiety as clinically significant in both P1 and P2.
Life functioning and therapy alliance
Comparisons were also conducted on life functioning (CORS) and therapy alliance (GSRS) total scores in C1 and C6 (see Fig. 5). In the CORS, there was a negligible difference [M diff = 0.28 (95% CI: –9.79, 10.36), Hedges’ g av = 0.04] between C1 (M = 15.67, SD = 1.65) versus C6 (M = 15.38, SD = 1.49) – the likelihood of a child reporting improved life functioning at final versus first group-session was 48%. In the GSRS, there was a negligible difference [M diff = 0.37 (95% CI: –7.84, 8.58), Hedges’ g av = 0.03] between C1 (M = 29.85, SD = 7.85) versus C6 (M = 30.22, SD = 9.97) – the likelihood of a child reporting improved alliance at final versus first group-session was 52%.
Figure 5. Life functioning (CORS) and therapy alliance (GSRS) scores across sessions.
Group characteristics
Table 1 presents participant characteristics and primary outcome data by group. Information is presented for four iterations of the Anxiety Gremlins intervention: one that was directly observed (group 1, with additional data collection and contextualisation) and three (groups 2–4) that were analysed retrospectively based on available service records.
Table 1. Primary outcome (RCADS anxiety scores) by group
RCADS, Revised Children’s Anxiety and Depression Scale. Initial and final anxiety scores are RCADS total scores (T-scores). Numbers improved/deteriorated were computed based on reliable change indices. SES, socio-economic status. IMD, Index of Multiple Deprivation (deciles; lower scores = most deprived). Depression: those meeting criteria for co-morbid depression on the RCADS. ES, effect size (negative Hedges’ g av [standardised mean difference] values indicate deterioration [post-group anxiety > pre-group anxiety]). WB, White British. IQR, inter-quartile range.
Whilst effect sizes in groups 3 and 4 were consistent with the overall finding of a negligible effect (≤±.3) on child-reported anxiety, it is notable that there was a small, negative effect (towards deterioration; g av = –0.45) in group 1 and a small, positive effect (towards improvement; g av = 0.42) in group 2. These differences in group-level effect sizes were reflected in differential individual outcomes (50% showing deterioration and 0% showing improvement in group 1 versus 7% showing deterioration and 36% showing improvement in group 2).
There were some apparent differences in the characteristics of groups 1 and 2 (as compared with the other groups and one another). Group 1 participants appear to be more severe/complex (higher levels of anxiety and co-morbid depression pre-group).
Group 2 differed from other groups in size (14 versus 6 completing participants) and demographics (mostly boys/not White British).
Aim (ii): Helpful and hindering factors, qualitative method
Interviews were conducted with two group facilitators. They were 28 min (Deb) and 40 min (Carol) in length. Qualitative content analysis identified three themes in relation to hindering factors and two themes in relation to helpful factors.
Hindering factors
Theme (i): Flow
Both facilitators expressed that the intervention length, date changes and general flow of this group was unhelpful. A typical group should have run over 8 consecutive weeks with dates given to the parents at the start; this group took 10 weeks, with several date changes.
‘it felt like that dragged on a little bit and it didn’t flow properly’ (Carol)
‘the sessions kept changing which probably didn’t help the flow of the group and the dynamics build up quickly if done every week’ (Carol)
The fractured nature of the group was a barrier to the children’s full engagement and some of the learning through consistency may have been lost.
‘I guess they are able to think more about what they are doing and embed the strategies at home, whereas if you leave it a couple of weeks it is forgotten about’ (Carol)
These fractures arose due to staff sickness, rooms being unavailable, and running concurrent with school holidays. The group was scheduled over half-term but needed to be delayed due to a flurry of cancellations. Deb suggested that running groups over half term was common but unhelpful.
‘it’s difficult running sessions over half terms … you know obviously we recap but I don’t think it’s the same as doing the full session’ (Deb)
Theme (ii): Preparation and reflection
The impact on facilitators of the time available before and after the group for preparation and reflection was explored. Carol felt that having more time to reflect beforehand about individual children was missed.
‘What we didn’t do in this group because we didn’t have the time the facilitators didn’t meet beforehand and go through the young people and have some thoughts about initial formulations … a meeting beforehand might have been kind of helpful’ (Carol)
Deb initially felt that the time available was appropriate, but expressed concern about debriefing after each session.
‘No for me I think the time was okay. I don’t think it had an effect on the group but obviously it’s always good for the facilitators if there is enough time after the group to really talk about people’ (Deb)
She felt that if concerns were raised these were discussed, but workload time constraints impacted on the ability of facilitators to hold a full debrief.
‘if we had a concern about one person or not … We kind of always debrief about how the session went. That always happened’ (Deb)
‘…people had to rush off and they are on to the next thing. I think it’s helpful for the facilitators if there is kind of half an hour free after the group to discuss anything or pull it together’ (Deb)
‘one person might be available, but the next person wasn’t’ (Deb)
Theme (iii): Client complexity
Facilitators recognised the complexity of clients recruited into this group as a factor hindering change. Both facilitators felt that the group was a valuable tool for anxiety, but that the clients seen in this group were typically low in mood and may have required a follow-up intervention.
‘they were a bit more low in comparison to previous groups’ (Carol)
‘it’s not waste, it’s indicated and obviously then after the group they are probably somebody who wouldn’t be discharged and the next piece of work would be on the low mood’ (Deb)
Deb expressed how she comes across several clients who she feels need a different type of intervention for low mood prior to acceptance into the anxiety group.
‘sometimes … it feels that the depression is probably overall higher than the anxiety and … the low mood needs kind of maybe addressing prior to the anxiety management group’ (Deb)
Both facilitators reflected on the challenge of referrers identifying clients who were only anxious, due to the frequency of co-morbidity between depression and anxiety. This leads to the Anxiety Gremlins group being selected as the first option, regardless of co-morbidity and RCADS scores.
‘when you are referring on to a waiting list … it can be a bit of a hard call of how much is the anxiety and how much is the low mood’ (Deb)
‘I think that whatever their presentation are and the RCADS are initially … the anxiety management group can be like first port of call’ (Deb)
The complex environments of the children outside of the group were also raised as barriers to change. A primary concern for all children in the group was school, and most children headed to school directly after the group. Due to the age group, some of the children were beginning their transition to secondary school.
‘she fears constantly all the time about school and she knows that’s not gonna change. She knows that after the group she is still going back to school’ (Deb)
‘a lot of them were in year 7 weren’t they, and I guess this might have been a big change from I don’t know primary school moving up to secondary school’ (Carol)
Both facilitators regarded familial instability and lack of support at home as barriers to change, with a recognition that in certain cases a brief anxiety intervention is insufficient in the face of such environmental adversity – although they still perceived that the group could contribute beneficially by developing participants’ coping skills.
‘I don’t think she did the homework. Not sure if she got the support at home’ (Deb)
‘It doesn’t surprise me because I think there’s ongoing factors … but I guess they’re necessarily not gonna be changed within a CAMHS intervention, so I hope we’ve given this young person the skills to at least manage a bit more with these difficulties’ (Carol)
Helpful factors
Theme (i): I’m not the only one!
A prominent strength of the group identified by both facilitators was the recognition for each participant that they were not alone in their suffering.
‘I think that shared experience is probably one of the things they found really helpful and that normalisation of anxiety’ (Carol)
‘hearing other people’s stories helped, and not feeling so alone’ (Deb)
In addition, the children interacted positively with each other, and bonded with their peers. Deb mentioned how this bonding helped them engage in the group.
‘you could see that they helped each other when one of them couldn’t see what was being taught … one of their peers explained it to them … I thought the peers in that particular group were good’ (Deb)
Theme (ii): Beginning the therapeutic journey
The facilitators felt the children made progress more broadly. A basic understanding of anxiety and practice in identifying and discussing feelings were described as positive outcomes.
‘they understood the main principles of CBT and maybe that was enough for what they needed at this time’ (Carol)
‘I think that was kind of a real breakthrough for them being able to see what helps and what doesn’t help … to erm, challenge those thoughts and put them in perspective’ (Deb)
‘they were much more able to recognise their own physical sensations of anxiety and their own thoughts after leaving the group’ (Deb)
Discussion
This evaluation examined the effectiveness of Anxiety Gremlins and identified factors that were helpful or hindering to the group’s success. The purpose of the intervention was to reduce anxiety symptoms (as measured by the RCADS), but the findings of this evaluation do not support its effectiveness; the same number of children significantly deteriorated as improved, with more deteriorating into a clinical condition than improving into a non-clinical one. In anxiety symptoms, life functioning and therapeutic alliance, most children showed very little difference between their first and last sessions. It begs the question, is this a worthwhile intervention?
This is a difficult question to answer. When assessing the outcome of an intervention, it is important not to rely on either outcome measurements or clinical judgement too heavily at the expense of the other (Law and Wolpert, Reference Law and Wolpert2014; Wolpert et al., Reference Wolpert, Ford, Trustam, Law, Deighton, Flannery and Fugard2012). In this evaluation, clinical judgement seemed to be that the group was a helpful intervention: sharing worries as a group helped children understand they were not alone, and they interacted well and formed bonds with similarly affected peers. Conversely, the measurements used have strong psychometric profiles, do not support effectiveness, and we know children are informative and accurate reporters of their own mental health (Deighton et al., Reference Deighton, Croudace, Fonagy, Brown, Patalay and Wolpert2014). Examining outcomes by group, there was evidence of heterogeneity. In principle, the Anxiety Gremlins intervention is protocolised/delivered in a similar way by the same lead facilitators; however, in the absence of fidelity checks, variation between groups could reflect differences in intervention delivery. Conversely – assuming the specific ingredients of the CBT-based intervention remain constant across group iterations – variable outcomes may highlight the importance of common factors (such as group alliance; Cuijpers et al., Reference Cuijpers, Reijnders and Huibers2019) which are probably more sensitive to between-group differences in composition.
Certainly, group composition did vary, and the most contrasting outcomes were evident in groups with distinctive characteristics (groups 1 and 2). Group 1 participants appear to be more severe/complex (higher levels of anxiety and co-morbid depression pre-group) which may have contributed to poorer (deteriorative) outcomes in this group (and this would accord with facilitator reflections). Group 2 differed from other groups in size (over twice the size of other groups) and demographics (mostly boys/not White British) but it is a little harder to link these differences to clinical outcome (perhaps the larger group size contributed positively to group dynamics in this instance?). Clearly further evaluation would be needed to identify important moderators of outcome, but (given the deteriorative outcomes in group 1) the most pressing implication for the service is to improve screening for severity/complexity (which appears contraindicative for the Anxiety Gremlins group).
The use of clinical interviews with facilitators provided insight into the difficulties faced by CAMHS services when selecting appropriate interventions – particularly reflected in the theme of ‘client complexity’. There was an acceptance that well-meaning clinicians refer several children with known co-morbidities as it is the best option available to them (particularly in terms of waiting time) and co-morbidity is so prevalent (Brady and Kendall, Reference Brady and Kendall1992). However, the intervention is not designed for these children and may lead to higher rates of treatment failure (Berman et al., Reference Berman, Weems, Silverman and Kurtines2000). Interviews also highlighted the difficulties involved in replicating rigid research conditions within which interventions such as Coping Cat are developed. Overlap with school breaks, trouble with room bookings, and a shortage of facilitator preparation time were each found to have an impact on ideal group delivery (reflected in themes representing needs for ‘flow’ and ‘preparation and reflection’).
Identifying the ‘ideal’ group delivery ingredients is a further problem. Coping Cat has a fidelity measure, where clinicians can rate each other to ensure compliance to the programme. Anxiety Gremlins incorporates CBT principles into a flexibly applied session plan, with no tailored fidelity measure. Fidelity is an important aspect of intervention assessment and repeatability (Hoffmann et al., Reference Hoffmann, Glasziou, Boutron, Milne, Perera and Moher2014). The absence of suitable measures of CBT fidelity (competence and adherence) or target change-processes (such as anxiety-related cognitions) preclude assessment of whether CBT was delivered or received as intended/in accordance with posited mechanisms of action. This limits the theoretical generalisability of the present findings. It is not clear, for example, whether differences in outcomes (between individuals and across instances of the group) reflect theoretically consistent differences in whether/to what extent target (cognitive-change) processes were activated. Whilst the Anxiety Gremlins protocol ostensibly applies a CBT model, the lack of fidelity/process data makes it difficult to relate current findings to the wider CBT evidence base (i.e. to be confident that findings pertain to the same model of therapeutic change). A general limitation of the CBT evidence base is the relative paucity of empirical support for theorised mechanisms of action (i.e. that positive outcomes are driven by cognitive changes; Moghaddam and Dawson, Reference Moghaddam, Dawson, Dawson and Moghaddam2015) and it is important that relevant evidence is gathered at (and integrated from) all levels (from large-scale trials to local evaluations).
The NHS Future in Mind programme published recommendations that 70,000 more children and young people should have access to high-quality mental health care each year by 2020/21 (NHS England, Reference England2016). In responding to these recommendations, CAMHS services have increased their use of brief and group CBT interventions based on evidence that they are as efficacious as traditional length individual CBT (de Groot et al., Reference de Groot, Cobham, Leong and McDermott2007; Öst and Ollendick, Reference Öst and Ollendick2017). This study highlights potential contraindication for group CBT intervention for anxiety where there are co-morbid depression symptoms. Given that around half of children and young people requiring intervention from a generic CAMHS service (i.e. non-specialist presentations such as eating disorders and psychosis) have a co-morbid presentation (Wolpert et al., Reference Wolpert, Vostanis, Young, Clark, Davies, Fleming, Howey, Howley, Macdougall, Martin, Martin, Payne, Ritchie, Senior, York and Whale2015), it would suggest group interventions should be targeted to certain populations with less severe symptoms and no co-morbid depression symptoms. This highlights a need for the service to consider the assessment and screening process.
Recommendations
The findings of this evaluation are enough to warrant further investigations into the Anxiety Gremlins programme as children are not self-reporting benefits from treatment. Some recommendations are provided below:
The service should compare the content of the session plan to the Coping Cat manual to identify the level of consistency. Fidelity is a critical consideration in enabling successful adaptation/translation of empirically supported treatments from research to practice settings (McHugh et al., Reference McHugh, Murray and Barlow2009). Based on this, adaptations to align more closely with Coping Cat may improve outcomes.
The service should urgently ensure that appropriate interventions for children with co-morbidities (particularly depression) are available to referrers at assessment to avoid ineffectively matched interventions. Clinicians should be encouraged to use their clinical judgement alongside outcome measures to make intervention compatibility decisions. Moreover, given the apparently limited outcome effects of first-line intervention – and indication that a strength of the group may be in preparing children to engage with future supports (‘beginning the therapeutic journey’) – timely access to follow-up treatment options appears crucial.
The service should recognise that disruptions to the group impact on the continuity of learning and ‘flow’ and take steps to minimise these potential disruptions.
Limitations
The children in this evaluation were measured after the group, with no follow-up data or information presented about future outcomes (e.g. number discharged) so were unable to assess any longer-term group effects. In addition, interview feedback was sought from clinicians but not the children or their parents. Parental feedback was also limited in this evaluation by a very small sample size. The applicability of this evaluation to all Anxiety Gremlins groups may be a limitation, due to observed differences between groups. In particular, the qualitative component of the evaluation related primarily to a single group (although facilitators commented on this group from a broader perspective/experience of running the other groups). Moreover, the qualitative component of the evaluation could have been bolstered through additional quality assurance methods, particularly given the dual role of the first author (as interviewer and lead analyst); however, there is a pragmatic need for rapid/adapted methods within practice-based evaluative studies.
Future directions
Following exploration of consistency between content of Anxiety Gremlins and Coping Cat, a future evaluation could be conducted after adapting the intervention, with fidelity measurements to ensure facilitator compliance with the manual. Gathering child and parent feedback on the Anxiety Gremlins intervention would also help to give greater context to self-report outcome measures. Present results (and their divergence from efficacy findings for Coping Cat) may reflect losses in translation to the local context, and it is important the fidelity/process measures are incorporated to understand whether/how local findings relate to the broader evidence-base for Coping Cat and its underpinning CBT model – more broadly, this is a key consideration enabling the interpretation and integration of practice-based evidence for empirically supported treatments.
Acknowledgements
None.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of interest
Christopher Meek, Lucy Redstone and Nima Moghaddam have no conflicts of interests with respect to this publication.
Ethical statements
Authors abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the BPS. Ethical approval was not required as this project was conducted as a service evaluation – and registered as such with the appropriate governance department in the host healthcare organisation. The names of interviewees have been pseudonymised.
Supplementary material
To view supplementary material for this article, please visit: https://doi.org/10.1017/S1754470X20000409
Key practice points
(1) We need practice-based evidence to verify that general efficacy carries into routine clinical effectiveness.
(2) It is vital that clinical applications of CBT are robustly evaluated to enable feedback-informed practice and inform service developments.
(3) Adaptation of ‘evidence-based’ interventions requires careful consideration and fidelity-checking to ensure that core components are retained.
Comments
No Comments have been published for this article.