Introduction
Randomized placebo-controlled designs are the standard in pharmacotherapy efficacy trials, but are less common in research evaluating psychosocial interventions. Several investigators have strongly argued in favor of including psychotherapy placebos in clinical trials that examine psychosocial treatments, because placebos provide a way to calibrate the placebo response (Klein, Reference Klein1996; Quitkin, Reference Quitkin1999; Quitkin et al. Reference Quitkin, Rabkin, Gerald, Davis and Klein2000). Indeed, without knowing the placebo response rate, it is difficult to determine the specificity of the intervention under investigation (e.g. Klein, Reference Klein1996).
There is considerable controversy with regards to the magnitude of the placebo effect. Beecher's influential (Reference Beecher1955) article entitled ‘The powerful placebo’ suggested that placebo accounts for significant improvement in approximately 35% of the cases. This estimate for the placebo response rate was widely accepted until Hrobjartsson & Gotsche (Reference Hrobjartsson and Gotsche2001) reported the results of a meta-analytic review of clinical trials in which patients were randomized to either a placebo intervention or no treatment. The results indicated that placebo and no-treatment conditions yielded comparable rates of improvement on binary outcome measures (e.g. treatment response, remission). For continuous outcome measures (e.g. severity, number of symptoms) there was an advantage of placebo over no treatment, but the difference decreased with increasing sample size, suggesting a bias related to the effects of small trials. The difference between placebo and no treatment was significant for the trials with subjective outcomes, but it was not significant for those with objective outcomes. In fact, the placebo only showed a significant effect for the treatment of pain. No significant differences were observed between psychotherapy, pharmacological, and physical placebos.
Despite its rigorous methodology, this meta-analysis has been criticized for various methodological reasons (e.g. Bailar, Reference Bailar2001; Wampold et al. Reference Wampold, Minami, Tierney, Baskin and Bhati2005) and continues to be debated in the literature (Hrobjartsson & Gotsche, Reference Hrobjartsson and Gotsche2007; Wampold et al. Reference Wampold, Imel and Minami2007). Criticisms include the considerable heterogeneity in both the effect sizes and the methodological quality of the studies (Bailar, Reference Bailar2001), but also the fact that the number of trials for some of the Axis-I disorders was very small. For example, only six anxiety trials with continuous outcome measures, and not a single anxiety trial with binary outcome measures, were included in the analysis. In addition to these methodological weaknesses, it may be argued that the term placebo is inaccurate for psychotherapy control conditions. Indeed, unlike pill placebos, psychotherapy placebos typically involve several active components, including therapist contact, support, and education. Moreover, like the active treatment, psychotherapy placebos are presented to patients with a rationale for their efficacy. Accordingly, instead of psychotherapy placebo, we will use the term psychotherapy control condition in this article.
In sum, surprisingly little is known about the effects of psychotherapy control conditions in clinical trials for psychiatric disorders. Important issues that need to be clarified include the magnitude of the psychotherapy control condition effect and the heterogeneity of the effect across disorders. The literature on cognitive behavioral treatments (CBTs) for anxiety disorders is particularly well suited to clarify these issues, because psychotherapy control conditions have been frequently employed in CBT trials for anxiety disorders. These trials have consistently demonstrated the efficacy of CBTs as compared with placebo (for a review, see Hofmann & Smits, Reference Hofmann and Smits2008). The objective of this study was to conduct a meta-analytic review of the psychotherapy control condition effect in randomized controlled CBT trials of adult anxiety disorders. The goal was to estimate the effect of the psychotherapy control condition for the various anxiety disorders in order to aid investigators in the design of future psychotherapy efficacy studies. In addition, we explored the potential moderator effects of clinical characteristics (e.g. number of treatment sessions, diagnostic group) and study features [e.g. study year, assessment type (clinician-rated v. self-report)].
Method
Selection of studies
Our selection criteria were set to limit the sample to high-quality studies involving the evaluation of behavioral and cognitive protocols for adult anxiety disorders. Accordingly, we employed the following inclusion criteria: (a) patients had to be between the ages of 18 and 65 years; (b) patients had to meet the diagnostic criteria of DSM-III-R or DSM-IV for an anxiety disorder; (c) patients had to be randomly assigned to either CBT or psychotherapy control condition; (d) the clinical severity of the anxiety disorder had to be assessed by clinician-rated or self-report measures with sound psychometric properties; (e) the articles had to provide sufficient information to compute effect sizes (i.e. means and standard deviations, t or F values, change scores, frequencies, or probability levels).
Using Medline, PsycINFO, PubMed, SCOPUS, the Institute of Scientific Information, and Dissertation Abstracts International, we entered a combination of the following terms to identify eligible studies: random*, cognitive behavior* therap*, cognitive therap*, or behavior*therap*, panic disorder, agoraphobia, GAD, generalized anxiety disorder, generalised anxiety disorder, OCD, obsessive compulsive disorder, social phobia, social anxiety disorder, specific phobia, simple phobia, PTSD, post-traumatic stress disorder, and acute stress disorder. In addition, we asked experts in the field for relevant studies published in their respective languages. Finally, we conducted manual searches in the lists of references from empirical studies, meta-analyses and review articles.
Data extraction
For each study, the authors independently identified valid and reliable continuous measures for the assessment of clinical severity of the anxiety disorder (i.e. symptom severity, symptom frequency, and quality of life). If binary outcomes were reported, the authors selected the measure that reflected the most conservative indicator of treatment response. When the authors disagreed, they reached consensus through discussion. The numerical data were independently extracted by two research assistants.
Statistical methods
Effect size estimation
We computed the Hedges' g effect size and its 95% confidence interval (CI) for the pre- to post-treatment changes on each anxiety severity measure. This effect size is a variation on Cohen's d that corrects for biases due to small sample sizes (Hedges & Olkin, Reference Hedges and Olkin1985). The formula for computing d is as follows:
where 
is the pretreatment sample mean, 
is the post-treatment sample mean, S difference is the standard deviation of the difference, and r is the correlation between pretreatment and post-treatment scores. Hedges' g can be computed by multiplying d by correction factor J(df) as follows:
where df is the degrees of freedom to estimate the within-group standard deviation. The magnitude of the Hedges' g effect size may be interpreted using Cohen's (Reference Cohen1988) convention as small (0.2), medium (0.5) and large (0.8).
For studies with multiple outcomes, the effect sizes across measures were averaged to yield a single Hedges' g estimate for each study. Because our aim was to generalize beyond the observed studies, we adopted random-effects models instead of fixed models to obtain a summary statistic (Hedges & Vevea, Reference Hedges and Vevea1998). For binary outcomes (e.g. attrition, treatment response), we calculated overall means weighted by sample size.
Publication bias
Also known as the ‘file drawer problem’ (Rosenthal, Reference Rosenthal1979), meta-analyses may overestimate the overall effect size because studies with non-significant findings are often not published. In order to address this issue, we calculated the fail-safe N, which is the number of unretrieved studies required to reduce the overall effect size to a non-significant level (Cooper & Hedges, Reference Cooper and Hedges1994). Rosenthal (Reference Rosenthal1991) has proposed that effect sizes can be considered robust if the fail-safe N is greater than 5k+10, where k reflects the number of studies included in the meta-analysis. We opted to employ this method because it has become a standard for estimating publication bias, but acknowledge that it may underestimate the publication bias (i.e. if most unpublished studies report negative rather than non-significant findings).
Moderator analyses
We fitted two mixed-effects models to the effect size data to examine whether effect sizes varied significantly as a function of diagnostic group and type of outcome measure (clinician-rated v. self-report). Differences across diagnostic groups with respect to binary outcome measures (e.g. response, attrition) were examined using generalized linear models with follow-up pairwise comparisons. To explore the potential impact of study year and treatment dose (defined by number of sessions) on the magnitude of the effect of psychotherapy control conditions, we completed two separate regression analyses. These analyses were conducted using the program Comprehensive Meta-Analysis, version 2 (Biostat, Inc., Englewood, NJ, USA; Borenstein et al. Reference Borenstein, Hedges, Higgins and Rothstein2005).
Results
Study selection
As can be seen in Fig. 1, of the 1165 studies that were initially identified, 19 (454 patients) met all inclusion criteria and were included in the meta-analysis. The most common disorder was post-traumatic stress disorder (PTSD; six trials), followed by acute stress disorder (ASD; four trials) and social anxiety disorder (SAD; four trials), obsessive–compulsive disorder (OCD; three trials), generalized anxiety disorder (GAD; two trials) and panic disorder (PD; one trial). We did not identify any studies involving the randomization of patients suffering from specific phobia. Tables 1 and 2 list the characteristics for each of the studies included in the meta-analysis.
Fig. 1. Flow diagram of study selection process. RCT, Randomized controlled trial.
Table 1. Characteristics of studies included in the meta-analysis
ASD, Acute stress disorder; GAD, generalized anxiety disorder; OCD, obsessive–compulsive disorder; PD, panic disorder; PTSD, post-traumatic stress disorder; SAD, social anxiety disorder; DAV.ID, digital audio visual integration device.
Table 2. Characteristics of measures for each study included in the meta-analysis
ADIS-IV, Anxiety Disorder Interview Schedule for DSM-IV (DiNardo et al. Reference DiNardo, Brown and Barlow1994); ADIS-R, Anxiety Disorder Interview Schedule Revised (DiNardo & Barlow, Reference DiNardo and Barlow1988); ASD, acute stress disorder; ASI, Anxiety Sensitivity Index (Reiss et al. Reference Reiss, Peterson, Gursky and McNally1986); BAI, Beck Anxiety Inventory (Beck et al. Reference Beck, Epstein, Brown and Steer1988); BSI, Brief Symptom Inventory (Derogatis & Melisaratos, Reference Derogatis and Melisaratos1983); CAPS-2, Clinician Administered PTSD Scale, version 2 (Blake et al. Reference Blake, Weathers, Nagy, Kaloupek, Gusman, Charney and Keane1995); CCQ, Catastrophic Cognitions Questionnaire (Khawaja & Oei, Reference Khawaja and Oei1992); CSC, Clinically Significant Change (Jacobson & Truax, Reference Jacobson and Truax1991); CGI-I, Clinical Global Impressions Scale – improvement (Guy, Reference Guy and Guy1976); CIDI, Composite International Diagnostic Interview (World Health Organization, 1997); FDAS, Four Dimensional Anxiety Scale (Bystritsky, Reference Bystritsky1990); FNE, Fear of Negative Evaluation Scale (Watson & Friend, Reference Watson and Friend1969); FQ, Fear Questionnaire (Marks & Mathews, Reference Marks and Mathews1979); GAD, generalized anxiety disorder; HAMA, Hamilton Anxiety Rating Scale (Hamilton, Reference Hamilton1959); IES, Impact of Event Scale (Horowitz et al. Reference Horowitz, Wilner and Alvarez1979); LIFE, The LIFE Base (Keller et al. Reference Keller, Lavori, Friedman, Nielsen, Endicott, Mcdonal-Scott and Andreasen1987); LSAS, Liebowitz Social Anxiety Scale (Liebowitz, Reference Liebowitz1987); LSAS-SR, Liebowitz Social Anxiety Scale – self-report (Baker et al. Reference Baker, Heinrichs, Kim and Hofmann2002); MOCI, Maudsley Obsessional–Compulsive Inventory (Hodgson & Rachman, Reference Hodgson and Rachman1977); OCD, obsessive–compulsive disorder; PADUA, The Padua Inventory (Sanavio, Reference Sanavio1988); PCL, PTSD Checklist (Weathers et al. Reference Weathers, Litz, Huska and Keane1995); PD, panic disorder; PDS, Post-traumatic Stress Diagnostic Scale (Foa, Reference Foa1995); PSWQ, Penn State Worry Questionnaire (Meyer et al. Reference Meyer, Miller, Metzger and Borkovec1990); PTSD, post-traumatic stress disorder; PTSD Symptom Scale, Post-traumatic Stress Disorder Symptom Scale (Foa et al. Reference Foa, Riggs, Dancu and Rothbaum1993); QOL, Quality of Life Scale (Cottraux et al. Reference Cottraux, Note, Albuisson, Yao, Note, Mollard, Bonasse, Jalenques, Guérin and Coudert2000); QOLI, Quality of Life Index (Frisch et al. Reference Frisch, Cornell, Villanueva and Retzlaff1992); RC, reliable change (Jacobson & Truax, Reference Jacobson and Truax1991); SAD, social anxiety disorder; SADS, Social Avoidance and Distress Scale (Watson & Friend, Reference Watson and Friend1969); SCL-90-R-IS, Symptom Checklist 90 Revised – interpersonal sensitivity (Derogatis, Reference Derogatis1977); SCL-90-R-PA, Symptom Checklist 90 Revised – phobic anxiety (Derogatis, Reference Derogatis1977); s.d., standard deviation; s.e., standard error; SF-12, 12-item version of the Medical Outcome Study Self-Report Form (Ware et al. Reference Ware, Kosinski and Keller1996); SIAS, Social Interaction Anxiety Scale (Mattick & Clarke, Reference Mattick and Clarke1998); SISST, Social Interaction Self-Statement Test (Glass et al. Reference Glass, Merluzzi, Biever and Larsen1982); SPAI, Social Phobia Anxiety Inventory (Turner et al. Reference Turner, Beidel, Dancu and Stanley1989); SPDS-C, Social Phobic Disorder Severity and Change Form – change (Liebowitz et al. Reference Liebowitz, Schneier, Campeas, Hollander, Hatterer, Fyer, Gorman, Papp, Davies, Gully and Klein1992); SPDS-S, Social Phobic Disorder Severity and Change Form – severity (Liebowitz et al. Reference Liebowitz, Schneier, Campeas, Hollander, Hatterer, Fyer, Gorman, Papp, Davies, Gully and Klein1992); SPS, Social Phobia Scale (Mattick & Clarke, Reference Mattick and Clarke1998); SRQ-20, The Self-Reporting Questionnaire 20 (Harding et al. Reference Harding, de Arango, Baltazar, Climent, Ibrahim, Ladrido-Ignacio, Murthy and Wig1980); STAI-T, State Trait Anxiety Inventory – trait subscale (Spielberger et al. Reference Spielberger, Gorsuch and Lushene1970); WSAS, Work and Social Adjustment Scale (Marks et al. Reference Marks, Connolly and Hallam1973); YBOCS, Yale-Brown Obsessive Compulsive Scale (Goodman et al. Reference Goodman, Price, Rasmussen, Mazure, Fleischmann, Hill, Heninger and Charney1989); ZSRA, Zung Self-Rating of Anxiety Scale (Zung, Reference Zung1975).
Only two studies (Bryant et al. Reference Bryant, Moulds, Guthrie and Nixon2005; McDonagh et al. Reference McDonagh, Friedman, McHugo, Ford, Sengupta, Mueser, Demment, Fournier, Schnurr and Descamps2005) provided data that were corrected for attrition, i.e. intent-to-treat (ITT) analyses. Unfortunately, we were unable to obtain ITT data from authors who did not include these in the original reports. Accordingly, the subsequent analyses are limited to completer data [19 studies (387 patients) for attrition rates and continuous measures of anxiety disorder severity; 15 studies (317 patients) for measures of response; see Table 1].
Evaluation of study quality
The quality of each study was evaluated according to the following modified Jadad criteria (Jadad et al. Reference Jadad, Moore, Carroll, Jenkinson, Reynolds, Gavaghan and McQuay1996): (a) the study was described as randomized; (b) participants were adequately randomized (e.g. adequate randomization procedure; the study reported withdrawals and drop-outs); (c) participants and evaluators were blinded to treatment condition (i.e. participants and evaluators were not aware whether they received active treatment or placebo intervention); (d) the evaluators were blinded to treatment conditions (i.e. evaluators were not aware which treatment condition participants had received; (e) the description of drop-outs was provided. Independently, the two authors rated each study across the five criteria. By assigning 1 point for each criterion met, total scores could range between 0 and 5. As can be seen in Table 1, the total scores for the study sample ranged from 1 to 3 with a median of 2 (mean=2.11, s.d.=0.66). There was high inter-rater agreement (κ=1).
Description of psychotherapy control conditions
The most commonly employed psychotherapy control condition was supportive therapy (15 studies). Generally, this protocol required therapists to provide a safe environment for self-reflection and be unconditionally supportive, while avoiding techniques specific to CBT (e.g. exposure, cognitive restructuring). Three identified studies included a relaxation protocol and one study used a problem-solving protocol as the psychotherapy control condition. Similar to the supportive therapy placebo, these psychotherapy control conditions provided patients with regular therapist contact and a supportive environment, but also a component with no proven efficacy for the treatment of anxiety disorder under investigation (e.g. relaxation for OCD, PTSD, or SAD; problem-solving for PTSD). Akin to CBT, most of the psychotherapy control protocols (14 studies) included the prescription of homework (e.g. self-monitoring, relaxation practice). The number of sessions ranged from three to 15 with a median of nine (mean=8.79, s.d.=3.69).
Pooled analyses
Fig. 2 depicts the effect sizes for each of the studies included in the meta-analysis and averaged by anxiety disorder. The random effects meta-analysis yielded a mean effect size of 0.45 (95% CI 0.35–0.46, z=8.50, p<0.001) across anxiety disorders. Following Cohen's (Reference Cohen1988) guidelines, this effect size falls in the small to medium range. Use of a random over fixed-effects model was supported by a significant Q statistic [Q(18)=42.78, p=0.001], indicating that the distribution of effect sizes was not homogeneous. The mean weighted response and attrition rates were 25.0% and 14.2%, respectively.
Fig. 2. Pre- to post-treatment effect-size estimates (Hedges' g) for psychotherapy control conditions by anxiety disorder. CI, Confidence interval; ASD, acute stress disorder; GAD, generalized anxiety disorder; OCD, obsessive–compulsive disorder; PD, panic disorder; PTSD, post-traumatic stress disorder; SAD, social anxiety disorder.
Publication bias
The effect size of 0.45 corresponds to a z value of 12.99 (p<0.001). Accordingly, it would require 816 failed trials for the two-tailed p value to exceed 0.05. This finding suggests that the effect size observed in the present study is probably robust (Rosenthal, Reference Rosenthal1991).
Moderator analyses
Comparison between diagnostic groups
As can be seen in Fig. 2, the effect sizes varied somewhat as a function of diagnostic group. Specifically, the effect size was largest for GAD (Hedges' g=0.62, 95% CI 0.08–1.16, z=2.26, p<0.05) followed by PTSD (Hedges' g=0.50, 95% CI 0.33–0.68, z=5.68, p<0.001), SAD (Hedges' g=0.47, 95% CI 0.24–0.71, z=3.90, p<0.001), ASD (Hedges' g=0.46, 95% CI 0.18–0.74, z=3.21, p<0.01), OCD (Hedges' g=0.26, 95% CI 0.13–0.40, z=3.80, p<0.001), and PD (Hedges' g=0.22, 95% CI −0.11 to −0.54, z=1.29, p=0.20). However, this difference in effect sizes did not reach statistical significance [Q(5)=7.51, p=0.19].
Weighted response rates were 29, 22, 15, 34 and 22% for ASD, GAD, OCD, PTSD and SAD, respectively. No response rates were reported in the PD study (Craske et al. Reference Craske, Maidenberg and Bystritsky1995). Generalized linear models with follow-up pairwise comparisons showed that the difference between OCD and PTSD was statistically significant (p=0.009). Finally, attrition rates were largest for GAD (25%), followed by PTSD (16%), SAD (16%), OCD (11%), ASD (10%) and PD (7%). Generalized linear models with follow-up pairwise comparisons revealed that the attrition observed for GAD was greater compared with all other disorders (all p<0.02), except for PTSD (p=0.06) and SAD (p=0.05).
Comparison between clinician-rated and self-report measures
Effect sizes observed for clinician-rated measures (Hedges' g=0.52, 95% CI 0.38–0.66, z=7.23, p<0.001) were not significantly different from those observed for self-report measures [Hedges' g=0.39, 95% CI 0.33–0.46, z=12.05, p<0.001; Q(1)=2.66, p=0.10].
Effect size as a function of treatment dose and study year
The effect size for the improvement on anxiety severity measures tended to be greater for early studies compared with those conducted in more recent years, although the relationship was not statistically significant (b=−0.014, z=−1.54, p=0.12). The magnitude of the psychotherapy control condition effect was also not significantly related to the number of treatment sessions (b=0.008, z=0.81, p=0.42).
Discussion
Psychotherapy control conditions are interventions designed to control for the effect of non-specific factors. These control conditions, therefore, include components that are common to all forms of psychotherapy (e.g. regular contact with a therapist, a supportive environment, instilling belief in the rationale for treatment and in the treatment itself), and lack ingredients that are specific to the intervention under investigation (i.e. exposure and cognitive restructuring for CBT). Accordingly, it can be argued that the inclusion of a psychotherapy control condition (versus waitlist control or pill placebo) in trials aimed at evaluating a psychosocial intervention allows the investigator to isolate the effects of the specific ingredients of that intervention (e.g. Klein, Reference Klein1996)Footnote †.
Some have estimated that the placebo response rate may be as high as 65% (Quitkin, Reference Quitkin1999). In contrast, an influential meta-analysis raised questions about the general efficacy of placebo interventions (Hrobjartsson & Gotsche, Reference Hrobjartsson and Gotsche2001). However, this study only included six anxiety trials with continuous outcome measures, and there were insufficient data available to study the magnitude of the psychotherapy placebo effect for anxiety disorders. In order to fill this gap in the literature, we meta-analytically reviewed the effects of psychotherapy control conditions included in randomized CBT trials for adult anxiety disorders. Our results indicate that psychotherapy control conditions are associated with medium-sized and statistically significant reductions in anxiety severity among patients suffering from the range of anxiety disorders. Interestingly, one out of four patients completing psychotherapy control condition protocols met the criteria for treatment response. These effect sizes and the relatively low attrition rate (14.2%) are parameters that investigators can use for the estimation of the sample size for an adequately powered clinical trial of psychosocial treatments for the anxiety disorders.
Previous work suggests that the response to pill placebo may vary across anxiety disorders (Mavissakalian et al. Reference Mavissakalian, Jones and Olson1990; Piercy et al. Reference Piercy, Sramec, Kurtz and Cutler1996). Indeed, using data from three large pill placebo-controlled trials, Huppert et al. (Reference Huppert, Schultz, Foa, Barlow, Davidson, Gorman, Shear, Simpson and Woods2004) found that patients with SAD and PD evidenced a greater response to pill placebo compared with patients with OCD. Although the small sample of studies included in the present analyses limits us from testing differences across the anxiety disorders, the pattern of findings do not suggest that response to psychotherapy placebo is significantly weaker among patients suffering from OCD. Similarly, the lack of the statistically significant difference between the effect sizes of clinician-rated instruments and self-report instruments may be attributed to the relatively small number of studies used in the analyses.
Unfortunately, the literature of CBT for anxiety disorders only allowed us to examine the uncontrolled effect size estimate of the psychotherapy control condition effect. Accordingly, we cannot rule out that that the observed changes with psychotherapy control conditions merely reflect an effect of time. It should be noted, however, that anxiety disorders tend to be chronic conditions when left untreated (Bruce et al. Reference Bruce, Yonkers, Otto, Eisen, Weisberg, Pagano, Shea and Keller2005). Indeed, waitlist control conditions typically show very minimal, if any changes of anxiety symptoms. Finally, the findings provide little insight into the mechanisms underlying the change that occurs during the control therapy. Psychotherapy includes many non-specific (common) factors that are likely to be present in both the ‘active’ condition and the control condition. These factors are complex and difficult to quantify because they are related to the therapeutic relationship, mastery or control experiences, and attribution of symptom change, among other factors (e.g. Hofmann & Weinberger, Reference Hofmann and Weinberger2007). Future studies should attend to mechanisms underlying the improvements observed with psychotherapy control conditions, including, for example, the possible allegiance effect of the investigator (Luborsky et al. Reference Luborsky, Diguer, Seligman, Rosenthal, Krause, Johnson, Halperin, Bishop, Berman and Schweizer1999) and the effect due to the lack of blinding in psychotherapy trials (Quitkin, Reference Quitkin, Rabkin, Gerald, Davis and Klein2000). As the number of clinical trials that include psychotherapy controls will accumulate over the next years, it will also become feasible to examine possible differences in the effects (and mechanisms) among the different types of psychotherapy control conditions (e.g. relaxation, supportive counseling, anxiety management). Despite these limitations, the data to date, presented in this paper, do provide the necessary information for power calculations of future psychotherapy outcome studies.
Acknowledgements
J.A.J.S. is supported by National Institute of Mental Health (NIMH) grant 1R01MH075889. S.G.H. is a paid consultant of Organon and supported by NIMH grant 1R01MH078308. We thank Angela Berry, Erik Müller, Christiane Suttner and Kristina Korte for their assistance with the data extraction.
Declaration of Interest
None.
