Introduction
Cognitive behavioural therapy (CBT) is the gold standard of treatment for panic disorder (PD; Arch and Craske, Reference Arch and Craske2008; Overholser, Reference Overholser2000). The central elements of CBT for PD include: psychoeducation, cognitive restructuring (targeting catastrophic thinking and over-estimation of the likelihood of negative outcomes), exposure to feared internal sensations (i.e. interoceptive exposure; IE), and confronting feared situations or activities. Research supports the effectiveness of cognitive strategies in modifying negative interpretations of bodily sensations and adjusting automatic thoughts (Clark et al., Reference Clark, Salkovskis, Öst, Breitholtz, Koehler, Westling, Jeavons and Gelder1997; Craske and Barlow, Reference Craske, Barlow and Barlow2008; Gould et al., Reference Gould, Otto and Pollack1995). Correspondingly, IE has been found to be equally effective in reducing fear of sensations in PD (Arntz, Reference Arntz2002). Other research (Gould et al., Reference Gould, Otto and Pollack1995; Margraf and Schneider, Reference Margraf and Schneider1991) has shown that the combination of cognitive therapy (CT) and IE were highly effective. Despite these findings, research is needed to determine the mechanisms of these techniques and other CBT strategies in the treatment of panic.
A central hypothesis in the cognitive model of PD (Clark, Reference Clark1986; Clark et al., Reference Clark, Salkovskis, Gelder, Koehler, Martin, Anastasiades, Hand and Wittchen1988) is that the catastrophic misinterpretation of physical sensations increases anxious arousal, yielding a panic attack. Individuals tend to normalize unusual physical sensations by ascribing them to situations and when they fail to do that physical or psychological factors are then raised as potential explanations (Austin and Richards, Reference Austin and Richards2001). Research (Beck and Emery, Reference Beck and Emery1985; Clark et al., Reference Clark, Salkovskis, Gelder, Koehler, Martin, Anastasiades, Hand and Wittchen1988) has suggested that this cognitive process is a fundamental mechanism in the development and maintenance of PD. A number of studies have investigated associations between cognitive biases and the interpretation of distress in PD and suggest that individuals with PD tend to experience catastrophic interpretations of bodily sensations (Austin and Richards, Reference Austin and Richards2001; Clark et al., Reference Clark, Salkovskis, Öst, Breitholtz, Koehler, Westling, Jeavons and Gelder1997; Harvey et al., Reference Harvey, Richards, Dziadosz and Swindell1993; Kamieniecki et al., Reference Kamieniecki, Wade and Tsouritos1997; Salkovskis et al., Reference Salkovskis, Clark and Gelder1996; Teachman et al., Reference Teachman, Marker and Smith-Janik2008). The existing research supports cognitive changes via indirect evidence (e.g. McNally and Foa, Reference McNally and Foa1987), which showed that there were no differences in judgement biases between successfully treated patients and healthy controls, and that changes in cognitions are related to symptom reduction (Clark et al., Reference Clark, Salkovskis, Öst, Breitholtz, Koehler, Westling, Jeavons and Gelder1997; Teachman et al., Reference Teachman, Marker and Smith-Janik2008). Additionally, post-treatment self-reported cognitions have been predictive of maintenance of treatment gains (Clark, Reference Clark1999).
It is important to distinguish between techniques (cognitive restructuring or exposure) and processes (changes in catastrophic cognitions or fear conditioning, extinction, habituation, or inhibitory learning), as suggested by Lorenzo-Luaces et al. (Reference Lorenzo-Luaces, Keefe and DeRubeis2016) and more recently by Hofmann and Hayes (Reference Hofmann and Hayes2018). However, there is another distinction that is important to make, within techniques. There is a difference between techniques offered by the therapist and the acquisition of skills by the patient. The same skill can be obtained from various techniques (exposures, cognitive challenging, behavioural experiments, etc.), and the acquisition of the skill should lead to changes in the mechanism (appraisals/anxiety sensitivity, avoidance etc.). Thus, our study allows both objective coding of patient behaviour/statements within a session as well determination of whether the changes in such behaviours are related to changes in symptoms (PDSS), avoidance (MI) and appraisals (ASI). The study does not determine what techniques (psychoeducation, cognitive challenging, exposure) the therapist used to help the patient acquire these skills as there is not a sufficient sampling of sessions throughout treatment.
Whereas the catastrophic interpretation model has gained empirical support via self-report measures, little research has been conducted examining the impact of patient acquisition of CBT skills on outcome, and fewer studies have used an objective examination of skills acquisition on mechanisms or outcomes. One could argue that use of reappraisal is a patient skill that should be related to changes in cognitions. Indeed, Strauss, Kivity and Huppert (in press) examined self-reported use of appraisal and its relationship to outcomes and found that reappraisal was not related to either changes in symptoms or changes in anxiety sensitivity. However, this study used only self-report ratings, which are often not highly correlated with other, objective assessments of reappraisal (e.g., Kivity and Huppert, Reference Kivity and Huppert2018). Therefore, assessing patients’ cognitive and behavioural skills in treatment by an independent evaluator allows for an objective examination of each skill on both mechanisms of change and outcomes.
Previous research (Barber and DeRubeis, Reference Barber and DeRubeis1992, Reference Barber and DeRubeis2001; Strunk et al., Reference Strunk, DeRubeis, Chiu and Alvarez2007, Reference Strunk, Hollars, Adler, Goldstein and Braun2014) has demonstrated a relationship between acquiring CT skills and treatment outcome in depression. Barber and DeRubeis (Reference Barber and DeRubeis1992) developed the Ways of Responding (WOR) scale to assess compensatory skills in CT. Results showed that WOR scores improved during treatment and were linked to a reduction in depressive symptoms (Barber and DeRubeis, Reference Barber and DeRubeis1992). To replicate and extend their findings, Barber and DeRubies (Reference Barber and DeRubeis2001) investigated the acquisition of skills during CT for depression. The results showed a decrease in depression levels and an improvement in WOR total scores after 12 weeks of treatment, and from pre-treatment to post-treatment. Moreover, greater changes in WOR total scores were associated with greater improvements in self-reported depressive symptoms.
Because the WOR measures only the skills learned in therapy, Strunk et al. (Reference Strunk, DeRubeis, Chiu and Alvarez2007) developed the performance of CT strategies (PCTS), a tool for evaluating the cognitive and behavioural skills used in and between sessions in CT for depression. These results suggest that patients’ acquisition of compensatory skills may predict risk for relapse after treatment (Strunk et al., Reference Strunk, DeRubeis, Chiu and Alvarez2007). One limitation of this study is the absence of skill measurement in early stages of treatment, which precludes causal inference about change or acquisition of skills and their relationship to treatment outcomes in CT.
More recently, Strunk et al. (Reference Strunk, Hollars, Adler, Goldstein and Braun2014) developed the Competencies of Cognitive Therapy Scale: Self-Report version (CCTS-SR) and the Competencies of Cognitive Therapy Scale: Therapist-Report version (CCTS-TR). The CCTS-TR is a brief measure completed by a therapist to assess a patient’s ability, frequency and independence of use in three different CT skill domains: automatic thoughts, behavioural activation, and core beliefs. The CCTS-SR also contains items related to the same domains completed by the patient. Results suggest improvement in CT skills over the course of treatment and the CCTS-SR was significantly negatively related to depressive symptoms. The CCTS-TR moderately correlated with the WOR and depressive symptoms. One limitation of this study was the absence of an early evaluation of skills by therapists.
Given that no scale exists to examine in-session patient use of CBT skills in the treatment of PD, we developed the Cognitive Behavioral Therapy Panic Skills (CBTPS) based on Strunk et al. (Reference Strunk, DeRubeis, Chiu and Alvarez2007). The CBTPS examines two constructs: cognitive and behavioural skills. The cognitive subscale contains items relating to patients’ ability to identify, challenge and reassess thoughts and to understand the rationale of treatment. The behavioural subscale contains items relating to patients’ ability to manage and deal with fear via exposure and reducing avoidance. The aim of developing such a scale is that it can help explain the mechanism of change in CBT for PD by: (a) examining cognitive and behavioural change during the course of treatment, and (b) testing whether specific skill acquisition is related to a particular change in PD (e.g. is acquisition of behavioural skills related to reductions in agoraphobia and cognitive skills related to reductions in anxiety sensitivity?). To the best of our knowledge, this is the first proof of concept study to evaluate the relationship between patients’ acquisition of CBT skills and treatment outcome in PD. We hypothesized that change in CBT skills will predict symptom improvement in the sample. More specifically, we predicted that changes in cognitive and behavioural skills will be related to relevant symptom improvement (i.e. anxiety sensitivity and agoraphobia, respectively).
Method
Patients
Twenty patients meeting the DSM-IV criteria for PD (12 women and 8 men, age 22–63 years) were recruited from online and on-campus advertisements to receive free treatment. To be included in the study, patients had to meet all the following criteria: a primary DSM-IV diagnosis of PD, 18 years or older, no concurrent therapy, no prior CBT therapy for PD, and a score of 10 or higher on the Panic Disorder Severity Scale - Independent Evaluator Version (PDSS-IE; Shear et al., Reference Shear, Brown, Barlow, Money, Sholomskas and Woods1997, Reference Shear, Rucci, Williams, Frank, Grochocinski and Vander Bilt2001). Exclusion criteria included history of psychosis, mania, recent history of substance abuse or dependence, and current suicidal ideation or history of suicide attempts. Patients were evaluated by a clinical interview using the Mini International Neuropsychiatry Interview 4.5 Hebrew version (MINI-4.5; Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Sheehan, Amorim and Janavs1998), a structured interview of axis-I disorders of the DSM-IV-TR, and the PDSS-IE. These were administered by both the future therapist and the supervisor to strengthen the reliability of diagnoses. After a complete description of the study to the subjects, written informed consent was obtained from all patients prior to study enrolment. The study was approved by the university’s ethics committee.Footnote 1 Twenty patients who completed treatment were selected for the study.
Therapists
Data were collected in the context of a research seminar for students studying for their Masters in clinical psychology, in which they conducted CBT for PD. All therapists were in their late 20s or early 30s. They were already treating other patients in their practicum sites, but this was their first case of panic with CBT. Each therapist treated only one patient as part of the study. They were trained and received supervision from a clinical psychologist with extensive experience in the treatment of PD (author J.D.H.). All in-office treatment sessions were videotaped in order to supervise the therapists and monitor protocol adherence, and the supervisor provided group supervision after viewing part or all of the sessions.
Treatment
Treatment length was up to 12 weekly individual sessions. All therapy sessions were videotaped, and each session lasted for up to 90 minutes (some of the sessions were held partially or entirely outside the laboratory to perform exposure). Treatment was based on the protocol of Barlow et al. (Reference Barlow, Craske and Meadows2007) for treating PD with additions (Huppert and Baker-Morisette, Reference Huppert and Baker-Morissette2003). All components of the protocol were implemented except those of the breathing and relaxation exercises because of the lack of empirical basis of effectiveness (Craske et al., Reference Craske, Rowe, Lewin and Noriega Dimitri1997; Schmidt et al., Reference Schmidt, Woolaway-Bickel, Trakowski, Santiago, Storey, Koselka and Cook2000). The first sessions (1 to 3 approximately) were devoted to psychoeducation and cognitive restructuring, the next sessions (3 to 5 approximately) were focused on interoceptive exposures (formal exercises which elicit various bodily sensations similar to panic sensations), and the later sessions were dedicated to in vivo exposures (some combined with interoceptive exposures), and consolidation of gains and relapse prevention. Patients were asked to indicate lessons learned at the end of each session. They also received homework after each session (e.g. repeating the interoceptive exposure exercises at home, and conducting an in vivo exposure on their own such as drinking coffee and running and then driving alone in the car). Formal assessments of therapists’ adherence and competence were not carried out; however, review of therapy notes indicated that all patients received all prescribed components of treatment.
Coders
The coding team consisted of three BA female students. Each coder was trained for 20 hours about CBT for PD, the CBTPS, and the coding process. The coding process included watching sessions, recording comments relating to relevant behaviours or statements, and then giving scores based on the CBTPS. In order to examine the change in acquisition of CBT skills during therapy, sessions 3 and 4 were chosen as early stage of therapy (after psychoeducation) and sessions 9 and 10 as late stage of therapy (prior to discussing relapse prevention). Sessions 3, 4, 9 and 10 were coded for all patients (except for three patients due to absence or problems with recording; for those patients only sessions 3, 4 and 9 were coded). In total, 77 sessions were coded. Sessions were randomly assigned to observers for coding. To prevent bias in coding, each coder coded either sessions 3 and 4 or 9 and 10 for each patient. Nine videotapes were coded by three different observers to train the coders in the coding. Then, twenty sessions were coded by at least two of the coders to determine reliability. Coders were blind to outcome data.
Measures
All measures were translated into Hebrew then back translated to English.
Panic Disorder Severity Scale – Self-Report Version (PDSS-SR; Houck et al., Reference Houck, Spiegel, Shear and Rucci2002)
The PDSS-SR is a self-report version of the respective independent evaluator measure for measuring the severity of panic symptoms, which has been shown to have good psychometrics and sensitivity to change (Houck et al., Reference Houck, Spiegel, Shear and Rucci2002).
The Anxiety Sensitivity Index-3
The ASI-3 is an 18-item measure that assesses beliefs about the feared consequences of symptoms associated with anxious stimulation (Taylor et al., Reference Taylor, Zvolensky, Cox, Deacon, Heimberg and Ledley2007). Taylor et al. (Reference Taylor, Zvolensky, Cox, Deacon, Heimberg and Ledley2007) showed good internal consistency and reliability of the ASI-3.
The Mobility Inventory
The mobility inventory (MI) is used to measure levels of agoraphobia (Chambless et al., Reference Chambless, Caputo, Jasin, Gracely and Williams1985). The two subscales have good psychometric properties (Chambless et al., Reference Chambless, Caputo, Jasin, Gracely and Williams1985).
Cognitive Behavioral Therapy Panic Skills
The CBTPS is a new scale developed for the current study to assess the degree of utilization of learned skills in CBT for PD. It is composed of 25 items selected based on the coding of treatment sessions (14 items relating to cognitive skills and 11 items relating to behavioural skills). Each item is scored on a 0–6 scale (0 indicates not exhibiting the skill, and 6 indicates full internalization and implementation of the skill). A score of 4 or higher indicates independent use of the skill without the need for therapist assistance. The total score ranges from 66 (behavioural subscale; see Appendix) to 84 (cognitive subscale; see Appendix).
The CBTPS is based on a number of principles. First, it is based on the ideas underlying the protocol of Barlow et al. (Reference Barlow, Craske and Meadows2007) for treating PD. The treatment focuses on re-evaluating situations by understanding the nature of the disorder, identifying and challenging biased thoughts (particularly of probability over-estimation and catastrophizing), and coping with fear by being exposed to anxiety-provoking triggers and situations (interoceptive and in vivo). Second, the CBTPS incorporates the principles of Huppert and Baker-Morissette (Reference Huppert and Baker-Morissette2003), who focused on helping patients develop panic skills, such as identifying challenging thoughts as part of the process of evaluating safety behaviours. Third, the CBTPS takes into consideration the common tools used in CBT for PD (e.g. 75% of therapists treating panic reported using in vivo exposure and 65% interceptive exposure) and factors associated with negative treatment outcomes (e.g. beliefs that anxiety is dangerous and panic attacks are caused by external factors, refusal to stop safety behaviours, and fear of exposure accompanied with emotional responses; Wolf and Goldfried, Reference Wolf and Goldfried2014).
The CBTPS is divided into two subscales: cognitive and behavioural. The cognitive subscale contains items relating to the ability of patients to identify, challenge and reassess thoughts and to understand the rationale of treatment. The behavioural subscale contains items related to the ability to manage and deal with fear via exposure and reducing avoidance. After choosing the initial items, we conducted a pilot study with a number of videotapes (not included in the current study) to (a) check if it is possible to score the items based on the videotaped session, (b) verify the degree of differences between items, and (c) check for items’ relevance and clarity. During the process of coding, the emphasis was on distinguishing between patients’ stated willingness to perform skills and their actual performance of them (suggesting higher behavioural skill), and identifying avoidance and safety behaviours even when the therapist did not refer to them as such. Coders also differentiated between the use of cognitive strategies to modify dysfunctional thinking patterns (higher skills) versus using them as safety behaviours, as well as distinguishing between performing an exposure to provoke more anxiety (behavioural skill) as opposed to doing so in order to avoid a panic attack (lower level of a behavioural skill). Finally, more active learning (raising questions for reappraisal) versus more passive learning (answering therapist questions but not initiating them) was also considered as having higher levels of cognitive skills. In addition, we examined the items’ relevance by assigning grades to the items, where the grades represent the coder’s degree of confidence in rating the items. A grade of A was given if the coder was confident in coding the item and a grade of B when it was difficult to code the item based on the session. From the 77 sessions (1925 items), only 55 items (2.8%) received a B (29 cognitive items and 26 behavioural items), which indicates that the items were typically relevant to the sessions, and the coders rated the items with confidence more than 97% of the time. Based on the pilot results we removed some items because they examined the same skills, and we added an item relating to the ability to start and initiate a panic attack. Furthermore, we divided some of the items into skills demonstrated in session and those discuss regarding work done outside the session.
Procedures
Patients who responded to the advertisements were first screened by telephone by a graduate student in clinical psychology. Those who were found suitable were invited for a full clinical interview, which included the MINI 4.5 (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Sheehan, Amorim and Janavs1998) and the PDSS-IE (Shear et al., Reference Shear, Brown, Barlow, Money, Sholomskas and Woods1997, Reference Shear, Rucci, Williams, Frank, Grochocinski and Vander Bilt2001). Patients who met inclusion criteria began therapy after a staggered baseline period (ranging from 1 to 6 weeks; the analysis in the current study used only the active therapy sessions). During therapy, patients completed questionnaires before and after each session (ASI, PDSS and MI were only administered at the beginning of each session, as they refer to panic severity and agoraphobic avoidance in the last week). Supervision was conducted on a weekly basis.
Data analysis
To evaluate the new CBTPS scale and examine its relationship to treatment outcome, we used full intent-to-treat longitudinal mixed effects models (LMLM) via the ‘nlme’ package in R (Pinheiro et al., Reference Pinheiro, Bates, DebRoy and Sarkar2017). Models were adjusted for repeated measures with restricted maximum likelihood estimation, a first-order autoregressive covariance structure at the time level and random intercepts and slopes at the patient level. Consistent with Wang and Maxwell’s (Reference Wang and Maxwell2015) recommendations we did not control for linear time effects in our data.
For the purpose of model comparison, we used three model comparison statistics: the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and the Nakagawa and Schielzeth R 2 (Johnson, Reference Johnson2014; Nakagawa and Schielzeth, Reference Nakagawa and Schielzeth2013). Both BIC and AIC are used for model selection, with lower values indicating a better fit. Both calculate the model fit to the data while controlling for overfitting by introducing a penalty term for the number of parameters in the model. Nakagawa and Schielzeth R 2 values were computed using the package ‘r2glmm’ in R (Jaeger et al., Reference Jaeger, Edwards, Das and Sen2017). When comparing two nested models, likelihood ratio tests were used in addition to the other statistics. For all LMLM coefficients, effect sizes were calculated as semi-partial r (r s; Jaeger, et al., Reference Jaeger, Edwards, Das and Sen2017; Nakagawa and Schielzeth, Reference Nakagawa and Schielzeth2013) using the same ‘r2glmm’ package. These effect sizes represent the unique contribution of the predictor above and beyond the contribution of all other predictors in the model.
Results
Reliability of CBTPS
To assess the reliability between observers, nine sessions (randomly chosen) were coded by all three raters. A two-way random effect model was used to compute the inter-class correlation coefficient (ICC) for each subscale. For the cognitive subscale, the ICC coefficient was .79, and for the behavioural subscale the ICC coefficient was .90, suggesting good inter-rater reliability. The internal consistency of the CBTPS was calculated using Cronbach’s alpha for the two subscales separately for each time point. For the cognitive scale, the Cronbach’s alpha ranged from .92 to .98, and for the behavioural scale it ranged from .96 to .98. To test the test–retest reliability of the subscales, Pearson correlations were conducted separately between early sessions (3 and 4) and late sessions (9 and 10). Both the cognitive (r (20) = .82, .70, p < .01, for early and late, respectively) and the behavioural r (20) = .84, .85, p < .01) subscales revealed strong test–retest reliability. Finally, we averaged ratings of early sessions (3 and 4) and ratings of late sessions (9 and 10) together to create the early and late skills ratings.Footnote 2 We predicted lower reliability between early and late scores due to changes via treatment. Both the cognitive (r (20) = .45, p = .047) and the behavioural (r (20) = .56, p = .01) subscales revealed moderate correlations as expected. Finally, we examined the correlation between the subscales. The correlation between early cognitive and behavioural skills was r (20) = .78, p < .01, and between the subscales late in treatment was r (20) = .95, p < .01, suggesting a relationship between cognitive and behaviour skills that increases over treatment.
Changes in CBTPS scores
The averages and the standard deviations for the CBTPS subscale scores and for symptom scores at the early and later therapy stages are presented in Table 1 and Fig. 1. Patients increased significantly from early to late sessions in both behavioural skills (t 19 = –5.1, p < .01, d = 1.05 [0.73–1.46]) and cognitive skills (t 19 = –4.1, p < .01, d = .94 [0.63–1.32]). On average, patients showed independent use of the skills (a score of 4 or higher) in 10% of the behavioural skills early in treatment and 49% at the end of treatment. For the cognitive skills, patients showed proficiency in 22% of the skills early in treatment and 59% at the end of treatment.
Table 1. Mean and standard deviation (SD) of CBTPS subscales and symptom measures at early and late therapy stages
For the CBTPS, early is the average of sessions 3 and 4, late is the average of sessions 9 and 10. For symptom measures, early is at the beginning of session 1, late is the last treatment session (typically session 12). CBTPS, Cognitive Behavioral Therapy Panic Skills; Cog, cognitive subscale; Beh, behavioural subscale; PDSS, Panic Disorder Severity Scale; ASI, Anxiety Sensitivity Index-3; MI, Mobility Inventory.
Figure 1. CBTPS mean subscale scores at sessions 3, 4, 9 and 10.
Relationship between changes in CBTPS and changes in symptoms
Cognitive subscale
As predicted, increases in cognitive skills (CBTPS-Cog) were associated with decreases in PD symptoms (PDSS) (b = –0.14, t 19 = –3.34, p < .01, r s = .43 [.16, .64]). Increases in cognitive skills also predicted decreases in the anxiety sensitivity (ASI) (b = –0.54, t 19 = –3.57, p < .01, r s = .38 [.10, .61]). Finally, increases in cognitive skills were significantly related to decreases in the agoraphobic avoidance (MI) (b = –0.91, t 19 = –3.02, p < .01, r s = .27 [.02, .53]; see Table 2). To further strengthen the analysis, we examined if the cognitive subscale was associated with the PDSS and MI above and beyond the ASI. Results were significant for the PDSS (b = –0.09, t 18 = –2.20, p = 0.04, r s = .30 [.03, .55]) but not MI (b = –0.49, t 18 = –1.39, p = 0.18, r s = .15 [.00, .44]).
Table 2. Prediction of symptom measures by the CBTPS cognitive subscale
PDSS, Panic Disorder Severity Scale; ASI, Anxiety Sensitivity Index-3; MI, Mobility Inventory; CBTPS, Cognitive Behavioral Therapy Panic Skills; Cog, cognitive subscale. **Significantly different from zero at p < 0.01.
Behavioural subscale
As predicted, increase in behavioural skills (CBTPS-behavioural) was associated with decrease in PD symptoms (PDSS) (b = –0.14, t 19 = –3.66, p < .01, r s = .47 [.21, .67]). Increases in behavioural skills were also related to decrease in anxiety sensitivity (ASI-3) (b = –0.57, t 19 = –4.26, p < .01, r s = .43 [.16, .64]). Finally, increases in behavioural skills were related to decreases in agoraphobic avoidance (b = –1.03, t 19 = –3.93, p < .01, r s = .33 [.05, .57]; see Table 3). We conducted model comparisons to examine whether the subscales (both cognitive and behavioural) accounted for the changes in symptoms. The cognitive and behavioural subscales were examined as separate or combined predictors of each outcome measure. According to these models, the behavioural subscale better accounted for improvement (approximately 4% of the variance) than the cognitive subscale on all symptom measures. In addition, it did not appear that including both the cognitive and behavioural skills together in the model contributed to further variance explained beyond the behavioural skills alone (see Table 4). To further strengthen the analysis, we examined if the behavioural subscale was associated with the PDSS and MI over and beyond the ASI. Results were significant for the PDSS (b = –0.09, t 18 = –2.33, p = 0.03, r s = .33 [.05, .58]) and for the MI (b = –0.71, t 18 = –2.12, p < 0.05, r s = .22 [.00, .50]).
Table 3. Prediction of symptom measures by the CBTPS behavioural subscale
PDSS, Panic Disorder Severity Scale; ASI, Anxiety Sensitivity Index-3; MI, Mobility Inventory; CBTPS, Cognitive Behavioral Therapy Panic Skills; Beh, behavioural subscale. **Significantly different from zero at p < 0.01.
Table 4. Model comparison table for prediction of Panic Disorder Severity Scale (PDSS), Anxiety Sensitivity Index-3 (ASI), and Mobility Inventory (MI) from cognitive (Cog) and behavioural (Beh) subscales separately and together
MC, model comparisons; N p, number of parameters in the model; R 2, Nakagawa and Schielzeth (Reference Nakagawa and Schielzeth2013) marginal R 2; χ2, chi-square statistic used in the deviance test; p-value, for chi-square distribution with one degree of freedom; ΔAIC, Akaike information criterion difference; ΔBIC, Bayesian information criterion difference. *p < .05.
Discussion
In this study, we proposed a new scale for measuring the use of specific skills learned in CBT for PD. We chose to develop a measure of skills for PD to facilitate further research on the specific mechanisms of treatment for PD. The CBTPS differs in many ways from existing assessment tools measuring skills acquired in treatment. First, to our knowledge this is the first study to examine CBT skills in PD. Second, examining skills based on session videotapes is rarely done, especially in CBT treatment for PD. Coding sessions by observers allows for measuring characteristics and prediction of outcome. The high inter-rater and between session reliability as well as the strong internal consistency indicate that the scale has strong psychometrics. The findings of the present study support our hypothesis that changes in CBTPS scores (on both subscales) were significantly related to symptom improvement, anxiety sensitivity and agoraphobic avoidance, and that they demonstrated incremental validity above and beyond the ASI as a predictor of symptom outcomes. The findings of this study are generally consistent with previous findings that show a significant relationship between CT skills for depression and treatment outcome, which provides additional support for the mechanisms of CBT (Barber and DeRubies, Reference Barber and DeRubeis1992, Reference Barber and DeRubeis2001; Lorenzo-Luaces et al., Reference Lorenzo-Luaces, Keefe and DeRubeis2016).
The relationship among CBTPS scores and PD outcomes raises the question of whether CBTPS measures skills or treatment outcomes. The CBTPS cognitive subscale reflects understanding of the principles learned in CBT, and the behavioural subscale measures coping skills to deal with the fear of having a panic attack. We cannot rule out the possibility that other factors are responsible for reducing anxiety sensitivity and may lead to symptom improvement. However, during the development of the scale and coding sessions, we came across a number of cases where severity of symptoms contradicted measured skills. For example, at late stages of treatment some patients reported symptom improvement but did not understand the principles of treatment. These patients reported not experiencing PD attacks because they avoided initiating attacks and performing exposures to avoid unnecessary suffering. Therefore, these patients scored low on the CBTPS scale and on the PDSS.
The high correlation between cognitive subscale scores and behavioural subscale scores raises the question of whether the two subscales measure the same variables. Although cognitive subscale items refer to the understanding and implementation of cognitive strategies and behavioural subscale items refer to behaviours, there is considerable overlap between the constructs. For example, the cognitive subscale contains a question: does the patient understand the need to initiate an attack? On the other hand, the behavioural subscale contains a question: is the patient attempting to initiate an attack? We came across cases where patients understood the need to initiate a panic attack but did not initiate it, as well as other cases in which patients initiated a panic attack to appease the therapist without understanding the rationale; therefore, it is important to distinguish between cognitive and behavioural skills themselves. However, our data suggest that the enacting of the skills via behaviours (behavioural subscale) accounted for somewhat more of the variance in outcomes consistently (4%) and that the addition of the cognitive subscale did not contribute to better prediction of outcomes. It is likely that understanding the rationale and the rest of the cognitive work facilitates the behavioural skills, which are necessary but not sufficient. Indeed, the increased correlation between the subscales from beginning to end of treatment indicates that the processes become more related during treatment. This variance could have been due to some cognitive abilities being inferred from expressed behaviours as cognitive constructs may not be as evident as behaviours. Future research using self-report measures and other cognitive assessments could improve reliability on items that had poor reliability. This proof of concept study was a first examination of the scale; however, more research is needed to examine the development of cognitive and behavioural skills as the mechanism of treatment.
Strunk et al. (Reference Strunk, DeRubeis, Chiu and Alvarez2007) failed to show a correlation between the PCTS and WOR scores at post-treatment in CT for depression, but showed that these scores are significant predictors for relapse in follow-up. They suggested that one of the scales measures understanding of the principles of CT and the second measures utilization of learned skills. The CBTPS attempts to capture these two constructs within the measure. Using the CBTPS in longitudinal follow-ups can help determine the contribution of both sets of skills in maintaining gains.
Various studies have shown that a combination of cognitive and behavioural components, as were used in the current study, is the optimal treatment for PD (Arntz, Reference Arntz2002; Margraf and Schneider, Reference Margraf and Schneider1991; McMillan and Lee, Reference McMillan and Lee2010). It is possible that combined treatment is successful not only because it combines two different learning mechanisms, as proposed by Arntz (Reference Arntz2002), but there might be a two-way relationship between the two mechanisms, such that cognitive change facilitates behavioural change and vice versa. For example, it is possible that reassessing the accuracy of catastrophic thoughts allows the patient to initiate a panic attack (e.g. exposure), and that approaching and experiencing an attack allows modification of associations, reinforces messages learned in treatment, and essentially challenges and modifies negative thoughts about panic attacks and their consequences. Using the CBTPS to examine cognitive and behavioural changes at various stages of treatment can help determine whether one change occurs prior to another and whether a specific change is related to symptom changes at a particular stage of treatment (Gloster et al., Reference Gloster, Klotsche, Gerlach, Hamm, Ströhle and Gauggel2014). An in-depth examination of the CBTPS items over sessions may refine our understanding of the impact of each skill and its acquisition on outcomes. Further research relating to developing cognitive behavioural skills session-by-session can shed light on mechanisms of change in treatment and how to facilitate them.
In addition, this measure can help future studies to examine the differential contribution of general CBT skills (e.g. Strunk et al., Reference Strunk, Hollars, Adler, Goldstein and Braun2014) versus disorder specific skills. Given the current trend of transdiagnostic treatments, it will be important to determine if there are specific skills that help patients with specific disorders or whether general skills are sufficient. It is possible that even in transdiagnostic treatments, specific skills are learned (Clark, Reference Clark2009).
Limitations
There are several limitations of this study. First is the absence of follow-up data. Despite the demonstrated significant relationship between CBTPS scores and treatment outcome, measuring symptom levels in follow-up could help patients focus on relevant skills at the end of treatment and determine whether acquired skills are maintained and predict long-term outcomes (e.g. Strunk et al., Reference Strunk, DeRubeis, Chiu and Alvarez2007). Second, the absence of a control group does not allow investigating if the acquisition of these skills is unique to CBT in terms of treating PD. It would be interesting to apply the coding system to other treatments that have been developed for panic, such as Milrod’s panic-focused psychodynamic treatment (Milrod et al., Reference Milrod, Chambless, Gallop, Busch, Schwalberg and McCarthy2015) to determine whether such skills are obtained and predictive of outcomes in non-CBT or psychopharmacological treatments. Third, these results are based on a relatively small clinical sample; it would be good to examine the measure in a larger sample over a full course of treatment instead of just early and late sessions. Furthermore, this study was conducted on novice therapists treating their first case of panic with CBT. It would be important to see if skills acquisition occurs faster in more experienced therapists. Finally, it was difficult to code the use of skills during the early stages of treatment as the first few sessions focus on cognitive appraisals and distortions. There may be a need to use other methods such as a structured interview to determine what skills are used during early sessions. Similarly, evaluating skills pre-treatment could clarify the skills required to learn new skills during therapy. Alternatively, pre-treatment skills could be viewed as potential for capitalization of strength during treatment, and studies could examine compensation versus capitalization in CBT for panic (Cheavens et al., Reference Cheavens, Strunk, Lazarus and Goldstein2012).
To our knowledge, this is the first proof of concept study to examine the impact of CBT skills acquisition on treatment outcomes in PD. We developed a reliable and valid measure for examining patients’ cognitive and behavioural skills in CBT for PD. The CBTPS scale allows examining the mechanism of change and provides an opportunity to deepen our knowledge of treatment for PD. This study sheds light on the link between CBTPS and treatment outcomes. Other studies have shown that scales measuring skills are related to outcome (Barber and DeRubeis Reference Barber and DeRubeis1992, Reference Barber and DeRubeis2001). In other words, it appears that scales measuring patients’ skills are important in terms of understanding the mechanism of change and improving treatment efficacy. However, this is an initial development of the scale, and further studies need to examine the validity of the CBTPS and predict post-treatment relapse in addition to investigating indicators for successful skill learning in therapy.
Author ORCID
Jonathan Huppert 0000-0002-0537-4701
Acknowledgments
The authors would like to thank Hagar Itach, Michal Linetzky, Yael Davidson and Ben Mernick for their hard work in coding the tapes for the study.
Ethical statement
Ethics approval for this study was obtained from the ethics committee of The Hebrew University of Jerusalem.
Conflicts of interest
The authors (A.H., N.Y., A.Y.S. and J.D.H.) declare that they have no conflicts of interest with respect to this publication.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Appendix. Cognitive Behavioral Therapy Panic Skills (CBTPS)
Please rate the following questions using the scale below.
Cognitive Skills
1. Is the patient able to identify his/her automatic thoughts?
2. Does the patient understand the relationship between his/her thoughts and fear?
3. Does the patient understand the rationale behind the cognitive work (that an accurate assessment of the situation and correction of false beliefs will lead to decreasing the severity of the panic symptoms)?
4. Is the patient able to examine evidence relating to automatic thoughts?
5. Is the patient able to properly assess the likelihood of the occurrence of automatic thoughts?*
6. Is the patient able to understand the consequences that could occur if the automatic thoughts were true?*
7. Is the patient able to provide an alternative explanation for experiencing symptoms?*
8. Is the patient able to evaluate the impact of changing automatic thoughts on his anxiety?
9. Does the patient understand the relationship between physiological sensations and automatic thoughts?
10. Is the patient able to produce a rational response to automatic thoughts when they arise?*
11. Does the patient understand the relationship between avoidance and anxiety?
12. Does the patient understand the relationship between safety behaviors and panic symptoms?
13. Does the patient understand the rationale behind interoceptive exposure (the desensitization of physiological sensations that occurs during attacks and detach the association of feelings with fear)?
14. Does the patient understand the need to initiate a panic attack?
* Correcting distorted thinking as an effective reevaluation and not as a safety behavior.
Behavioral Skills
1. Is the patient planning exposures (whether he/she has an intention to perform an exposure)?
2. Is the patient reducing safety behaviors when having an attack in a session?
3. Is the patient reducing safety behaviors when having an attack outside the session?
4. Is the patient trying to provoke physiological symptoms that appear during an attack in a session?
5. Is the patient trying to provoke physiological symptoms that appear during an attack outside the session?
6. Is the patient trying to initiate an attack in session?
7. Is the patient trying to initiate an attack outside session?
8. Is the patient able to initiate an attack in session?
9. Is the patient able to initiate an attack outside session?
10. Is the patient trying to increase the intensity of the attack during a session?
11. Is the patient trying to increase the intensity of the attack outside the session?
Comments
No Comments have been published for this article.