Hostname: page-component-745bb68f8f-s22k5 Total loading time: 0 Render date: 2025-02-05T00:48:01.579Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing panic disorder-specific competencies: evaluation of the Cognitive Therapy Competence Scale for panic disorder

Published online by Cambridge University Press:  08 September 2020

Sheena Liness ,
Sarah Beale ,
David M. Clark ,
Paul M. Salkovskis ,
Anke Ehlers Open the ORCID record for Anke Ehlers [Opens in a new window]  and
Jennifer Wild
Sheena Liness*
Affiliation:
Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, UK
Sarah Beale
Affiliation:
Institute of Psychiatry, Psychology, and Neuroscience, King’s College London, London, UK
David M. Clark
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
Paul M. Salkovskis
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
Anke Ehlers
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
Jennifer Wild
Affiliation:
Department of Experimental Psychology, University of Oxford, Oxford, UK Oxford Health NHS Foundation Trust, Oxford, UK
*
*Corresponding author. Email: sheena.liness@kcl.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background:

Evidence-based treatment for panic disorder consists of disorder-specific cognitive behavioural therapy (CBT) protocols. However, most measures of CBT competence are generic and there is a clear need for disorder-specific assessment measures.

Aims:

To fill this gap, we evaluated the psychometric properties of the Cognitive Therapy Competence Scale (CTCP) for panic disorder.

Method:

CBT trainees (n = 60) submitted audio recordings of CBT for panic disorder that were scored on a generic competence measure, the Cognitive Therapy Scale – Revised (CTS-R), and the CTCP by markers with experience in CBT practice and evaluation. Trainees also provided pre- to post-treatment clinical outcomes on disorder-specific patient report measures for cases corresponding to their therapy recordings.

Results:

The CTCP exhibited strong internal consistency (α = .79–.91) and inter-rater reliability (ICC = .70–.88). The measure demonstrated convergent validity with the CTS-R (r = .40–.54), although investigation into competence classification indicated that the CTCP may be more sensitive at detecting competence for panic disorder-specific CBT skills. Notably, the CTCP demonstrated the first indication of a relationship between therapist competence and clinical outcome for panic disorder (r = .29–.35); no relationship was found for the CTS-R.

Conclusions:

These findings provide initial support for the reliability and validity of the CTCP for assessing therapist competence in CBT for panic disorder and support the use of anxiety disorder-specific competence measures. Further investigation into the psychometric properties of the measure in other therapist cohorts and its relationship with clinical outcomes is recommended.

Keywords

CBTcompetencedisorder-specific measurespanic disordertraining
Type
Main
Information
Behavioural and Cognitive Psychotherapy , Volume 49 , Issue 2 , March 2021 , pp. 197 - 205
Copyright
© British Association for Behavioural and Cognitive Psychotherapies 2020

Introduction

National Institute for Health and Care Excellence (NICE) guidelines recommend disorder-specific rather than generic cognitive behaviour therapy (CBT) interventions for depression and anxiety disorders (NICE, 2011). Relevant assessment methods are required to evaluate therapists’ delivery of these interventions. The Cognitive Therapy Scale (CTS; Young and Beck, Reference Young and Beck1980) and Cognitive Therapy Scale – Revised (CTS-R; Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001), originally developed for evaluating cognitive therapy for depression, are commonly used to assess therapist competence across disorders. These scales have been successfully adapted with the addition of specialised items to assess CBT for a range of populations, including children and young people (Stallard et al., Reference Stallard, Myles and Branson2014), psychosis (Haddock et al., Reference Haddock, Devane, Bradshaw, McGovern, Tarrier, Kinderman, Baguley, Lancashire and Harris2001) and palliative care (Mannix et al., Reference Mannix, Blackburn, Garland, Gracie, Moorey, Reid, Standart and Scott2006). However, there are currently few scales that have been evaluated to assess therapist competencies in disorder-specific interventions for anxiety presentations. The Competence in Cognitive Therapy for Social Phobia (CTCS-SP) scale (Clark et al., Reference Clark, von Consbruch, Hinrichs and Stangier2007; Von Consbruch et al., Reference Von Consbruch, Clark and Stangier2012), adapted from the CTS, assesses therapist competence in cognitive therapy for social anxiety disorder and demonstrates high inter-rater reliability, test–re-test reliability and internal consistency. Treatment-specific competency measured on the CTCS-SP predicted a large proportion of the variance in clinical outcome for social anxiety (β = .59–79; Ginzburg et al., Reference Ginzburg, Bohn, Höfling, Weck, Clark and Stangier2012). Generic competency measures tend to predict clinical outcome more strongly for depression than anxiety disorders (Liness et al., Reference Liness, Beale, Lee, Byrne, Hirsch and Clark2019; Webb et al., Reference Webb, DeRubeis and Barber2010; Zarafonitis-Müller et al., Reference Zarafonitis-Müller, Kuhr and Bechdolf2014), thus disorder-specific competence measures may present an effective method not only for evaluating specific treatment competencies but also how they relate to patient recovery.

Panic disorder is a common and often disabling mental health condition, with an estimated prevalence of 1.70% in UK adults (Skapinakis et al., Reference Skapinakis, Lewis, Davies, Brugha, Prince and Singleton2011). While panic-focused CBT is the NICE (2011)-recommended treatment, no relevant disorder-specific competency scale is yet available. Secondary analyses of panic disorder treatment trials have uncovered no relationship between general therapist competence, rated using global impression indices, and patient outcome (Boswell et al., Reference Boswell, Gallagher, Sauer-Zavala, Bullis, Gorman, Shear, Woods and Barlow2013; Huppert et al., Reference Huppert, Bufka, Barlow, Gorman, Shear and Woods2001). However, these studies used unvalidated measures of therapist competence and were based on relatively small and highly trained samples of trial therapists (n = 14–21), possibly precluding necessary variance in competence required to identify a relationship with patient outcome. One further study evaluating exposure-based CBT for panic disorder with agoraphobia (Weck et al., Reference Weck, Grikscheit, Höfling, Kordt, Hamm, Gerlach and Lang2016) also found no relationship between therapist competence and patient outcome when rated on a German version of the CTS to assess generic CBT (Weck et al., Reference Weck, Grikscheit, Höfling and Stangier2014) and a competence measure for conducting exposure (Grikscheit et al., Reference Grikscheit, Lang, Kordt, Helbig-Lang, Geisler and Weck2015).

Based on promising evidence from other anxiety disorders (Ginzburg et al., Reference Ginzburg, Bohn, Höfling, Weck, Clark and Stangier2012), the relationship between therapist competence and patient outcome should be assessed using a disorder-specific measure that evaluates the full range of competencies required to deliver CBT for panic disorder. Furthermore, therapists trained in panic-focused CBT achieve good clinical outcomes (Liness et al., Reference Liness, Beale, Lee, Byrne, Hirsch and Clark2019) that are stronger than those attained using psychological treatment-as-usual (Grey et al., Reference Grey, Salkovskis, Quigley, Clark and Ehlers2008). A disorder-specific competence measure may further enhance training, clinical practice and supervision by providing guidance for therapists and detailed protocol-specific feedback. The evaluation of a disorder-specific therapist competence measure for panic disorder, the Cognitive Therapy Competence Scale for Panic Disorder (CTCP; Clark et al., Reference Clark, Salkovskis, Hackman and Grey2002), was the focus of this study.

The CTCP draws on the CTS and CTS-R and panic disorder-specific skills to assess CBT therapist competence for treating clients with panic disorder. The scale was developed to assess the delivery of Clark et al.’s (Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994) treatment protocol (for current manual, see Clark and Salkovskis, Reference Clark and Salkovskis2009); this is one of several evidence-based CBT protocols for panic disorder (e.g. Barlow et al., Reference Barlow, Craske, Cerny and Klosko1989) and is widely used in the UK. The scale items, descriptors for each item, and scoring anchors were developed by consensus of a team of experts in panic disorder.

As on the CTS and CTS-R, CTCP items (see Table 1) were divided into two theoretical subscales: general competency at delivering psychological therapy (items 1–5) and specific CBT for panic disorder skills (items 6–17). While the CTS-R has the same item range for the general subscale (items 1–5: respectively Agenda Setting, Feedback, Collaboration, Pacing and Efficient Use of Time, and Interpersonal Effectiveness), CTCP general competency items were adjusted considerably to reflect the general skills required to deliver the Clark et al. (Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994) protocol. Three items were retained (Agenda, Pacing and Efficient Use of Time, and Interpersonal Effectiveness), although scoring anchors were adjusted, and the remaining two items from the generic CTS-R were replaced with more treatment-relevant general skills. The specific subscale of the CTCP, unlike the CTS and CTS-R, was tailored for all items to focus specifically on elements of panic-focused CBT (see Clark and Salkovskis, Reference Clark and Salkovskis2009). Items are scored between 0 (poor) and 6 (excellent), following the same scale as the CTS with descriptions to anchor scores for each item with ratings of 0, 2, 4 and 6. Table 1 gives the CTCP items by subscale.

Table 1. Cognitive Therapy Competence Scale for Panic Disorder (CTCP; Clark et al., Reference Clark, Salkovskis, Hackman and Grey2002) items by subscale

The primary aim of this project was to evaluate the reliability and validity of the CTCP to assess panic disorder competence and clinical outcome for a sample of CBT trainees who attended a UK Improving Access to Psychological Therapies (IAPT) training programme. We hypothesized that the CTCP would demonstrate:

  1. (1) good inter-rater reliability and internal consistency;

  2. (2) convergent validity with the CTS-R;

  3. (3) a stronger association with panic disorder clinical outcomes than the CTS-R.

Method

Participants

Participants were 60 trainees from the High-Intensity IAPT (HI IAPT) CBT training course at the Institute of Psychiatry, Psychology and Neuroscience, King’s College London, who had submitted a recording of a CBT session for panic disorder. Of the trainees, 78% (n = 47) were female and 22% (n = 13) were male; 80% (n = 48) were White and 20% (n = 12) were of Black, Asian or minority ethnicity (BAME). Mean age was 35.36 years (SD = 7.60). Trainees’ professions were: psychological wellbeing practitioner (40%, n = 24), clinical psychologist (22%, n = 13), counselling psychologist (17%, n = 10), mental health nurse (10%, n = 6), occupational therapist (3%, n = 2), counsellor (3%, n = 2) and other mental-health profession (5%, n = 3). Final grade distribution for trainees was: merit (37%, n = 22), pass (60%, n = 36) and failed/withdrawn (3%, n = 2).

Measures

Therapy competence

The Cognitive Therapy Competence Scale for Panic Disorder (CTCP; Clark et al., Reference Clark, Salkovskis, Hackman and Grey2002) assessed disorder-specific competence for treating panic-focused CBT. The scale (see Table 1) consists of 17 items (competence threshold, mean item score ≥3), which are rated from 0 to 6 (0 = poor, 6 = excellent). Items 1–5 assess general therapeutic skills, while items 6–17 assess panic disorder-specific CBT skills. The reliability and validity of this measure were investigated in the current study.

The Cognitive Therapy Scale – Revised (CTS-R; Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001) assessed overall therapist competence in CBT, and was used to assess convergent validity with the CTCP and compare predictive validity for clinical outcomes in the present study. The scale consists of 12 items (competence threshold, mean item score ≥3), which are rated from 0 to 6 (0 = incompetent, 6 = expert). Items 1–5 (respectively Agenda Setting, Feedback, Collaboration, Pacing and Efficient Use of Time, and Interpersonal Effectiveness) assess general therapeutic skills, while items 6–12 (respectively Eliciting Appropriate Emotional Expression, Eliciting Key Cognitions, Eliciting Behaviours, Guided Discovery, Conceptual Integration, Application of Change Methods, and Homework Setting) assess CBT-specialised therapeutic skills. The CTS-R consistently demonstrates high internal consistency (α range = .75–.97; Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001; Kazantzis et al., Reference Kazantzis, Clayton, Cronin, Farchione, Limburg and Dobson2018; Reichelt et al., Reference Reichelt, James and Blackburn2003). Estimates of inter-rater reliability range considerably across studies [intra-class correlation (ICC) = .57, James et al., Reference James, Blackburn, Milne and Reichelt2001; ICC = .63, Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001; Finn’s r = .88, Kazantzis et al., Reference Kazantzis, Clayton, Cronin, Farchione, Limburg and Dobson2018; ICC = .95, Liness et al., Reference Liness, Beale, Lee, Byrne, Hirsch and Clark2019], with better agreement following rater training (ICC = .38 untrained to .76 trained, Gordon, Reference Gordon2006; r = .44 to .67, Reichelt et al., Reference Reichelt, James and Blackburn2003).

Raters in the current study were course staff with extensive experience practising, supervising and evaluating CBT, and had previously received training in scoring the CTS-R during staff induction as well as ongoing reliability monitoring as part of course procedures. Additional training on the CTCP was provided. Scoring of individual items and item applicability across sessions was discussed in detail during training with inter-rater reliability and ongoing monitoring conducted across the study. Assessors were asked to rate all items. Some items on the CTCP do not apply to all sessions (e.g. in-session behavioural experiments when working on an initial panic formulation and should be rated not applicable, n/a). The submission of mid-treatment active therapy sessions in this study resulted in very few n/a item ratings. All n/a item scores were accounted for appropriately in data analysis. We recommend that, in routine use, where such items are present, the total score be pro-rated to allow comparability across assessments.

Clinical outcome

The self-report Panic Rating Scale (PRS; adapted from Clark et al., Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994) was used to assess the frequency and distress associated with panic attacks. Based on the previous two weeks, patients rated panic frequency on a 5-point scale (0 = no panic attacks, 4 = one or more panic attacks per day), panic-related disability on a 9-point scale (0 = not at all disturbing and/or disabling, 8 = very disturbing and/or disabling), and panic-related agoraphobic avoidance on a 9-point scale (0 = never avoid, 8 = always avoid). Scores for the three scales were added together to generate a total PRS score out of 20. This approach was consistent with Grey et al. (Reference Grey, Salkovskis, Quigley, Clark and Ehlers2008), with the addition of the avoidance rating. The PRS, which is recommended in the relevant treatment manual (Clark and Salkovskis, Reference Clark and Salkovskis2009), has been previously used as a primary outcome measure to assess symptom change in major trials of the Clark et al. (Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994) protocol for panic disorder for CBT clinical trials (Clark et al., Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994, Reference Clark, Salkovskis, Hackmann, Wells, Ludgate and Gelder1999; Öst and Westling, Reference Öst and Westling1995) and in research of therapist training (Grey et al., Reference Grey, Salkovskis, Quigley, Clark and Ehlers2008), and was routinely used to assess treatment outcome for panic disorder on the training course from which the present data were drawn. The PRS demonstrated good internal consistency pre-treatment (ω = .81) and post-treatment (ω = .86) in the current sample.

Panic-related cognitions were assessed with a modified version of the self-report Agoraphobic Cognitions Questionnaire (ACQ; Chambless et al., Reference Chambless, Caputo, Bright and Gallagher1984). Patients were presented with 18 panic-related cognitions and rated the frequency of this cognition on a 5-point scale (1 = never, 5 = always) and also the modified degree to which they believed the cognition while anxious from 0 (do not believe) to 100 (completely convinced this is true). Items from each subscale were added to give a total score ranging from 18 to 90 for frequency and 0 to 1800 for belief. Therapists are encouraged to focus particularly on belief ratings to guide the course of therapy (Clark and Salkovskis, Reference Clark and Salkovskis2009). The ACQ demonstrates good internal consistency (α = .80) and acceptable test–re-test reliability (r = 0.86) in adults with panic/agoraphobia (Chambless et al., Reference Chambless, Caputo, Bright and Gallagher1984). It was not possible to calculate internal consistency for the present sample as case report data only included frequency and belief total scores and not individual items.

Procedure

Tapes were selected from a database of 224 former trainees of the HI-IAPT CBT course at the Institute of Psychiatry, Psychology and Neuroscience, King's College London. As part of coursework, trainees submitted five therapy tapes rated on the CTS-R by a course member and eight reports of clinical cases. Selected panic disorder cases required a recording of a corresponding mid-therapy active treatment session and clinical case outcomes rated on a disorder-specific measure (PRS and/or ACQ). A total of 60 applicable tapes were available and were second-rated with the CTCP by a course member. Each trainee supplied one tape and corresponding case. Trainees were required to use at least one panic disorder-specific measure (PRS and/or ACQ) in their therapy; however, several trainees used only the clinically significant belief subscale for the ACQ. Consequently, reported n values vary across measures.

Results Footnote 1

Descriptive statistics were generated for the total, general and specific subscales of the CTS-R and CTCP and are presented in Table 2. Mean item scores (0–6) were used in all analyses rather than total scores, due to differing scale lengths.

Table 2. Mean CTS-R and CTCP scores for total measure and subscales

CTS-R, Cognitive Therapy Scale – Revised; CTCP, Cognitive Therapy Competence Scale for Panic Disorder.

Reliability

Internal consistency

Cronbach’s alpha coefficients were calculated to assess internal consistency for the total CTCP and for the general and specific subscales. Internal consistency was high for the total measure (17 items; α = .91), general subscale (5 items; α = .79), and specific subscale (12 items; α = .88) of the CTCP. Removal of any item was found to result in a decreased alpha.

Inter-rater reliability

A one-way random effects single-measures intra-class correlation coefficient was calculated to assess inter-rater reliability between pairs randomly selected from five course markers for the CTCP. Inter-rater reliability was good for the total scale [ICC = .84, p < .001, 95% confidence interval (CI) = .54–.95], and for the general (ICC = .70, p = .006, 95% CI = .20–.91) and specific subscales (ICC = .88, p < .001, 95% CI = .62–.97).

Validity

Face validity

Twelve course supervisors with extensive experience in CBT practice, training and competence assessment provided a unanimously positive view of the measure, reporting that the CTCP more effectively captured the competencies required for delivering evidence-based CBT for panic disorder than the CTS-R.

Convergent validity

Pearson’s correlations were calculated between the CTS-R and CTCP for the total measures and for the general and specific subscales. Table 3 presents these correlations. Strong positive associations were found between the CTS-R and CTCP for the total measures and specific subscale, and a moderate positive association was found for the general subscale.

Table 3. Correlations between CTS-R and CTCP totals and subscales

CTS-R, Cognitive Therapy Scale – Revised; CTCP, Cognitive Therapy Competence Scale for Panic Disorder. *Significant at p ≤ .002, **significant at p ≤ .001.

Clinical outcomes

Pearson’s correlations were generated between clients’ percentage change for the PRS and ACQ, and both therapist competence measures. PRS correlations were generated on the subset of patients who scored above caseness criteria at baseline (severity ≥ 4) based on pre-treatment severity in previous studies and expert clinical judgement (Clark et al., Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994; Grey et al., Reference Grey, Salkovskis, Quigley, Clark and Ehlers2008). Table 4 reports these correlations. Positive associations were found between the CTCP and percentage change in the PRS disability and ACQ belief subscales. No other significant association was found between either therapist competence measure and the clinical outcome measures.

Table 4. CTCP and CTS-R predictive validity for clinical change scores

*Significant at p < .05. PRS, Panic Rating Scale (Clark et al., Reference Clark, Salkovskis, Hackmann, Middleton, Anastasiades and Gelder1994); ACQ, Agoraphobic Cognitions Questionnaire (Chambless et al., Reference Chambless, Caputo, Bright and Gallagher1984); CTS-R, Cognitive Therapy Scale – Revised (Blackburn et al., Reference Blackburn, James, Milne, Baker, Standart, Garland and Reichelt2001); CTCP, Cognitive Therapy Competence Scale for Panic Disorder.

Classification of competence

Chi-square tests were conducted to assess whether classifications of competence attainment (mean score ≥3) were equivalent for the CTS-R and CTCP. There was a significant difference in competence classification for overall scores [χ2 (1) = 4.85, p = .03]; this appeared to be driven by trainees who were classified as competent on the CTS-R but non-competent on the CTCP (30% of trainees were classified as competent on the CTS-R but not the CTCP vs 8% vice versa). There was no significant difference for the general subscale [exact χ2 (1) = .003, p = 1.00]. However, a significant difference emerged for the specific subscale [χ2 (1) = 8.28, p = .004] with 25% of trainees classified as competent on the CTS-R but not the CTCP vs 8% vice versa.

Discussion

This study aimed to evaluate the reliability and validity of the Cognitive Therapy Competence Scale for panic disorder. As predicted, the measure demonstrated good internal consistency (α = .79–.91) and inter-rater reliability (ICC = .70–.88) for total and subscale scores. Feedback from markers indicated good face validity. The CTCP mean total and subscale scores demonstrated convergent validity with the CTS-R (r = .40–.54) as expected.

The validity of the CTCP was further supported by its relationship to panic disorder-specific patient clinical outcomes. Associations emerged between the CTCP and percentage decrease in panic-related disability (r = .35) and percentage decrease in belief in panic-related cognitions (r = .29). No associations emerged for the CTS-R, supporting the hypothesis that the CTCP would demonstrate greater predictive validity in clinical outcome. The relatively small sample size (n = 47–53) may have lacked power to detect a small but significant relationship for some outcomes measures, as the relationship between competence and clinical outcome is often small as it is one of many relevant predictors (Webb et al., Reference Webb, DeRubeis and Barber2010). These findings support previous assertions that generic competency measures demonstrate limited predictive validity for outcomes in anxiety disorders (Liness et al., Reference Liness, Beale, Lee, Byrne, Hirsch and Clark2019; Webb et al., Reference Webb, DeRubeis and Barber2010; Zarafonitis-Müller et al., 2018), and that using disorder-specific competency measures is important (Ginzburg et al., Reference Ginzburg, Bohn, Höfling, Weck, Clark and Stangier2012).

Some differences emerged between the CTS-R and CTCP in relation to classification of competence, with 30% of trainees classed as competent on the CTS-R but non-competent on the CTCP. This disagreement appeared to be driven by specific subscale competence. This finding may simply reflect trainees’ uneven acquisition of different skills while developing clinical experience, or might indicate that trainees may have been applying techniques specific to CBT but not within the recommended protocol for the treatment of panic disorder. Given that NICE evidence-based treatment (NICE, 2011) consists of specific interventions detailed in the CTCP, these findings may indicate that the disorder-specific competency rating scale may be more sensitive to true competency in delivering appropriate treatment for this disorder – particularly in light of the clinical predictive validity of the CTCP. Further investigation into the relationship between disorder-specific treatment competencies and general CBT competencies and how these skills may interact to influence clinical outcomes is warranted in larger studies.

While initial findings on the psychometric properties of the CTCP are promising, this study has several limitations. The sample size was relatively small, particularly for clinical outcomes, and was drawn from a single CBT training course. Therapy recordings and clinical cases were self-selected by trainees; however, both were requested to be representative of trainees’ practice and clinical cases were selected with supervisors prior to therapy completion. Further testing in other training cohorts and in experienced therapists is indicated. It was not feasible to assess the measures’ responsiveness to training as cases were drawn from varied time points across the course due to limited numbers. Further investigation using randomisation is recommended to assess whether trainees who use the CTCP to inform therapy and receive feedback on the disorder-specific measure gain greater skill in delivering panic-specific interventions and stronger clinical outcomes than those who use generic measures. Finally, all markers were experienced in delivering and assessing CBT for panic disorder, and therefore the face validity and ease-of-use for inexperienced markers is unknown. Given the promising preliminary findings, further investigation with varied therapist and marker cohorts is recommended.

Overall, preliminary evidence indicates that the CTCP is a reliable and valid measure for assessing therapist competence in CBT for panic disorder. Additionally, this study is the first to our knowledge to find a relationship between therapist competence and clinical outcome for panic disorder; consequently, it supports the use of disorder-specific competence measures for anxiety. Further investigation into the psychometric properties of the CTCP – particularly clinical predictive validity and responsiveness to training –is indicated. If these positive results are generalisable, the CTCP may be used to assess and guide trainee therapists, evaluate treatment of panic disorder in routine care, and monitor fidelity and competence in clinical trials.

Acknowledgements

Many thanks to the tutors, trainees and patients who contributed to the research and to Hannah Parker and Steffen Nestler for their assistance with the study. With thanks to Anne Hackmann and Nick Grey for their input on the CTCP development.

Financial support

This study was funded by a British Association for Behavioural and Cognitive Psychotherapies research grant. Development of the CTCP was supported by grants from the Wellcome Trust: grant nos. 069777 and 200796.

Conflicts of interest

The authors have no conflicts of interest with respect to this publication.

Ethics statement

This study was approved by the King’s College London Psychiatry, Nursing, and Midwifery Research Ethics Committee as part of a larger programme of research on CBT training and clinical outcome: reference number PNM/12/13-50.

Footnotes

1 The Benjamini–Hochberg Procedure (Benjamini and Hochberg, Reference Benjamini and Hochberg1995) was applied to all hypothesis tests to correct for multiplicity, with false discovery rate Q = .10.

References

Barlow, D. H., Craske, M. G., Cerny, J. A., & Klosko, J. S. (1989). Behavioral treatment of panic disorder. Behavior Therapy, 20, 261–282.CrossRefGoogle Scholar
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57, 289–300.Google Scholar
Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., & Reichelt, F. K. (2001). The Revised Cognitive Therapy Scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy, 29, 431–446.CrossRefGoogle Scholar
Boswell, J. F., Gallagher, M. W., Sauer-Zavala, S. E., Bullis, J., Gorman, J. M., Shear, M. K., Woods, S., & Barlow, D. H. (2013). Patient characteristics and variability in adherence and competence in cognitive-behavioral therapy for panic disorder. Journal of Consulting and Clinical Psychology, 81, 443–454.CrossRefGoogle ScholarPubMed
Chambless, D. L., Caputo, G. C., Bright, P., & Gallagher, R. (1984). Assessment of fear of fear in agoraphobics: the body sensations questionnaire and the agoraphobic cognitions questionnaire. Journal of Consulting and Clinical Psychology, 52, 1090–1097.CrossRefGoogle ScholarPubMed
Clark, D. M. & Salkovskis, P. M. (2009). Panic Disorder: Manual for Improving Access to Psychological Therapies (IAPT) High Intensity Therapists. Unpublished manuscript. Oxford: University of Oxford. Available at: https://oxcadatresources.com/ Google Scholar
Clark, D. M., von Consbruch, K., Hinrichs, J. & Stangier, U. (2007). Cognitive Therapy of Social Phobia: Checklist of Therapist Competency. Unpublished manuscript. London: Kings College London.Google Scholar
Clark, D. M., Salkovskis, P. M., Hackman, A. & Grey, N. (2002). Cognitive Therapy for Panic: Checklist of Therapist Competency. Unpublished manuscript. Oxford: University of Oxford.Google Scholar
Clark, D. M., Salkovskis, P. M., Hackmann, A., Wells, A., Ludgate, J., & Gelder, M. (1999). Brief cognitive therapy for panic disorder: a randomized controlled trial. Journal of Consulting and Clinical Psychology, 67, 583–589.CrossRefGoogle ScholarPubMed
Clark, D. M., Salkovskis, P. M., Hackmann, A., Middleton, H., Anastasiades, P., & Gelder, M. (1994). A comparison of cognitive therapy, applied relaxation and imipramine in the treatment of panic disorder. British Journal of Psychiatry, 164, 759–769.CrossRefGoogle ScholarPubMed
Ginzburg, D. M., Bohn, C., Höfling, V., Weck, F., Clark, D. M., & Stangier, U. (2012). Treatment specific competence predicts outcome in cognitive therapy for social anxiety disorder. Behaviour Research and Therapy, 50, 747–752.CrossRefGoogle ScholarPubMed
Gordon, P. K. (2006). A comparison of two versions of the Cognitive Therapy Scale. Behavioural and Cognitive Psychotherapy, 35, 343–353.CrossRefGoogle Scholar
Grey, N., Salkovskis, P., Quigley, A., Clark, D. M., & Ehlers, A. (2008). Dissemination of cognitive therapy for panic disorder in primary care. Behavioural and Cognitive Psychotherapy, 36, 509–520.CrossRefGoogle ScholarPubMed
Grikscheit, F., Lang, T., Kordt, A., Helbig-Lang, S., Geisler, K., & Weck, F. (2015). Entwicklung und Validierung von Instrumenten zur Überprüfung der Therapeutischen Adhärenz und Kompetenz bei der Behandlung der Panikstörung. Zeitschrift für Klinische Psychologie und Psychotherapie, 44, 88–96.CrossRefGoogle Scholar
Haddock, G., Devane, S., Bradshaw, T., McGovern, J., Tarrier, N., Kinderman, P., Baguley, I., Lancashire, S., & Harris, N. (2001). An investigation into the psychometric properties of the cognitive therapy scale for psychosis (CTS-Psy). Behavioural and Cognitive Psychotherapy, 29, 221–233.CrossRefGoogle Scholar
Huppert, J. D., Bufka, L. F., Barlow, D. H., Gorman, J. M., Shear, K. M., & Woods, S.W. (2001). Therapists, therapist variables, and cognitive-behavioral therapy outcome in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology, 69, 747–755.CrossRefGoogle Scholar
James, I. A., Blackburn, I. M., Milne, D. L., & Reichelt, F.K. (2001). Moderators of trainee therapists’ competence in cognitive therapy. British Journal of Clinical Psychology, 40, 131–141.CrossRefGoogle ScholarPubMed
Kazantzis, N., Clayton, X., Cronin, T. J., Farchione, D., Limburg, K., & Dobson, K. S. (2018). The Cognitive Therapy Scale and Cognitive Therapy Scale-Revised as measures of therapist competence in cognitive behavior therapy for depression: relations with short and long term outcome. Cognitive Therapy and Research, 42, 385–397.CrossRefGoogle Scholar
Liness, S., Beale, S., Lee, S., Byrne, S., Hirsch, C. R., & Clark, D. M. (2019). Multi-professional IAPT CBT training: clinical competence and patient outcomes. Behavioural and Cognitive Psychotherapy. Advanced online publication: https://doi.org/10.1017/S1352465819000201 CrossRefGoogle ScholarPubMed
Mannix, K. A., Blackburn, I. M., Garland, A., Gracie, J., Moorey, S., Reid, B., Standart, S., & Scott, J. (2006). Effectiveness of brief training in cognitive behaviour therapy techniques for palliative care practitioners. Palliative Medicine, 20, 579–584.CrossRefGoogle ScholarPubMed
National Institute of Health and Care Excellence (NICE) (2011). Common Mental Health Disorders: Identification and Pathways to Care. Retrieved from Clinical Guideline CG123: https://www.nice.org.uk/guidance/cg123/evidence/full-guideline-181771741 Google Scholar
Öst, L. G., & Westling, B. E. (1995). Applied relaxation vs cognitive behavior therapy in the treatment of panic disorder. Behaviour Research and Therapy, 33, 145–158.CrossRefGoogle ScholarPubMed
Reichelt, F. K., James, I. A., & Blackburn, I. M. (2003). Impact of training on rating competence in cognitive therapy. Journal of Behavior Therapy and Experimental Psychiatry, 34, 87–99.CrossRefGoogle ScholarPubMed
Skapinakis, P., Lewis, G., Davies, S., Brugha, T., Prince, M., & Singleton, N. (2011). Panic disorder and subthreshold panic in the UK general population: epidemiology, comorbidity and functional limitation. European Psychiatry, 26, 354–362.CrossRefGoogle ScholarPubMed
Stallard, P., Myles, P., & Branson, A. (2014). The cognitive behaviour therapy scale for children and young people (CBTS-CYP): development and psychometric properties. Behavioural and Cognitive Psychotherapy, 42, 269–282.CrossRefGoogle Scholar
Von Consbruch, K., Clark, D. M., & Stangier, U. (2012). Assessing therapeutic competence in cognitive therapy for social phobia: psychometric properties of the cognitive therapy competence scale for social phobia (CTCS-SP). Behavioural and Cognitive Psychotherapy, 40, 149–161.CrossRefGoogle Scholar
Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: a meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200–211.CrossRefGoogle ScholarPubMed
Weck, F., Grikscheit, F., Höfling, V., Kordt, A., Hamm, A. O., Gerlach, A. L., … & Lang, T. (2016). The role of treatment delivery factors in exposure-based cognitive behavioral therapy for panic disorder with agoraphobia. Journal of Anxiety Disorders, 42, 10–18.CrossRefGoogle ScholarPubMed
Weck, F., Grikscheit, F., Höfling, V., & Stangier, U. (2014). Assessing treatment integrity in cognitive-behavioral therapy: comparing session segments with entire sessions. Behavior Therapy, 45, 541–552.CrossRefGoogle ScholarPubMed
Young, J., & Beck, A. T. (1980). Cognitive Therapy Scale: Rating manual. Unpublished manuscript, University of Pennsylvania.Google Scholar
Zarafonitis-Müller, S., Kuhr, K., & Bechdolf, A. (2014). The relationship between therapist’s competence and adherence to outcome in cognitive-behavioural therapy – results of a meta-analysis. Fortschritte der Neurologie-Psychiatrie, 82, 502–510.Google Scholar
Figure 0

Table 1. Cognitive Therapy Competence Scale for Panic Disorder (CTCP; Clark et al., 2002) items by subscale

Figure 1

Table 2. Mean CTS-R and CTCP scores for total measure and subscales

Figure 2

Table 3. Correlations between CTS-R and CTCP totals and subscales

Figure 3

Table 4. CTCP and CTS-R predictive validity for clinical change scores

Submit a response

Comments

No Comments have been published for this article.