In this issue, Cuijpers and Cristea have prepared a guideline consisting of six pragmatic pieces of advice for researchers to optimise the chances of finding statistically significant efficacy of a treatment over control in a randomised controlled trial (RCT) in the field of psychotherapy. They are: (i) have strong allegiance to the tested therapy; (ii) increase expectation in the therapy in the participants; (iii) utilise weak spots of randomised trials such as unconcealed randomisation, non-blind outcome assessment and unfair handling of dropouts; (iv) use a small sample size; (v) use waitlist control; and (vi) publish only those that confirm your allegiance. These ‘tips’ are so powerful that, the authors claim, following them would enable you to prove a psychotherapy to be effective even when it is not.
Cuijpers and his team are well-known avid producers of systematic reviews of psychotherapies. Systematic reviews and meta-analyses require critical appraisal of each included (and excluded) trial and critical overview of the totality of thus selected evidence. They have also conducted a number of influential randomised controlled trials of psychotherapies themselves. The guideline, therefore, is written on the background of their rich experiences in this field.
So, who should use this guideline?
The first group of uses would be people like me, who have developed their own programme of psychotherapy. Such would probably apply to every psychotherapist, as we sometimes half-jokingly note that there are as many schools of psychotherapy as there are psychotherapists. For instance, I certainly had strong allegiance to all the therapies that I examined in my own RCTs and I am sure that would have instilled expectation to the same effect among the participants that we had successfully recruited into the trials. One trial used non-blind self-report as its primary outcome (Furukawa et al.
Reference Furukawa, Horikoshi, Kawakami, Kadota, Sasaki, Sekiya, Hosogoshi, Kashimura, Asano, Terashima, Iwasa, Nagasaku and Grothaus2012) and another involved 37 participants only (Watanabe et al.
Reference Watanabe, Furukawa, Shimodera, Morokuma, Katsuki, Fujita, Sasaki, Kawamura and Perlis2011). And, of course, all used waitlist controls (Watanabe et al.
Reference Watanabe, Furukawa, Shimodera, Morokuma, Katsuki, Fujita, Sasaki, Kawamura and Perlis2011, Reference Watanabe, Horikoshi, Yamada, Shimodera, Akechi, Miki, Inagaki, Yonemoto, Imai, Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015; Furukawa et al.
Reference Furukawa, Horikoshi, Kawakami, Kadota, Sasaki, Sekiya, Hosogoshi, Kashimura, Asano, Terashima, Iwasa, Nagasaku and Grothaus2012). I am sure that there are many other followers of Cuijpers et al's recommendations among those who conduct psychotherapy trials today. Future trialists in this field should wisely learn from this guideline.
The second group of users would be people, again like me, who conduct systematic reviews and critical appraisal of psychotherapy literature. Traditional systematic reviews nowadays usually pay good attention to risks of bias pertaining to proper randomisation, blinding or intention-to-treat principle and publication bias. It is probably then surprising if a recent systematic review did not point out the risk involved in non-blinding when they reviewed internet CBT for anxiety disorders by claiming that it cannot be avoided in this field (Olthuis et al.
Reference Olthuis, Watt, Bailey, Hayden and Stewart2015). Other aspects that may lead to overestimation of efficacy of a psychotherapy have traditionally not received enough attention in systematic reviews. Some reviews still lump various control conditions into one comparison and do not take the effect of researcher allegiance into account when they synthesise evidence (Butler et al.
Reference Butler, Chapman, Forman and Beck2006). In fact, more trials are at high risk of bias for researcher allegiance recently than before (time trend p < 0.01) (Chen et al.
Reference Chen, Furukawa, Shinohara, Honyashiki, Imai, Ichikawa, Caldwell, Hunot and Churchill2014). Future reviews in this field should now regularly pay good attention to the factors enumerated by the guideline of Cuijpers et al.
The practicing psychologists and psychiatrists should form the third group of users, even when they do not themselves develop therapies, run trials of therapies or conduct systematic reviews of trials. Using this guideline, they can now more critically and validly appraise the evidence base of the psychotherapies that they practice. For example, I can now recommend this guideline and urge my colleagues who are all practicing psychiatrists or psychologists when we review research studies in our journal club.
But at the end of the day, how should the ultimate consumers of medical literature, namely patients, their families and the policy makers, use this guideline? Do they remain at the mercy of the bulk of literature consisting of original RCTs that followed this guideline, of systematic reviews that ignored this guideline in their evidence synthesis, and of practicing psychotherapists who may be all too easily convinced of the effectiveness of the therapies that they practice?
I leave the answer to each and every reader of Epidemiology and Psychiatric Sciences. Prime time has come for the world psychotherapy, psychology and psychiatry academia to move forward so that this guideline will be obsolete and unnecessary, soon.
You have
Access
In this issue, Cuijpers and Cristea have prepared a guideline consisting of six pragmatic pieces of advice for researchers to optimise the chances of finding statistically significant efficacy of a treatment over control in a randomised controlled trial (RCT) in the field of psychotherapy. They are: (i) have strong allegiance to the tested therapy; (ii) increase expectation in the therapy in the participants; (iii) utilise weak spots of randomised trials such as unconcealed randomisation, non-blind outcome assessment and unfair handling of dropouts; (iv) use a small sample size; (v) use waitlist control; and (vi) publish only those that confirm your allegiance. These ‘tips’ are so powerful that, the authors claim, following them would enable you to prove a psychotherapy to be effective even when it is not.
Cuijpers and his team are well-known avid producers of systematic reviews of psychotherapies. Systematic reviews and meta-analyses require critical appraisal of each included (and excluded) trial and critical overview of the totality of thus selected evidence. They have also conducted a number of influential randomised controlled trials of psychotherapies themselves. The guideline, therefore, is written on the background of their rich experiences in this field.
So, who should use this guideline?
The first group of uses would be people like me, who have developed their own programme of psychotherapy. Such would probably apply to every psychotherapist, as we sometimes half-jokingly note that there are as many schools of psychotherapy as there are psychotherapists. For instance, I certainly had strong allegiance to all the therapies that I examined in my own RCTs and I am sure that would have instilled expectation to the same effect among the participants that we had successfully recruited into the trials. One trial used non-blind self-report as its primary outcome (Furukawa et al. Reference Furukawa, Horikoshi, Kawakami, Kadota, Sasaki, Sekiya, Hosogoshi, Kashimura, Asano, Terashima, Iwasa, Nagasaku and Grothaus2012) and another involved 37 participants only (Watanabe et al. Reference Watanabe, Furukawa, Shimodera, Morokuma, Katsuki, Fujita, Sasaki, Kawamura and Perlis2011). And, of course, all used waitlist controls (Watanabe et al. Reference Watanabe, Furukawa, Shimodera, Morokuma, Katsuki, Fujita, Sasaki, Kawamura and Perlis2011, Reference Watanabe, Horikoshi, Yamada, Shimodera, Akechi, Miki, Inagaki, Yonemoto, Imai, Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015; Furukawa et al. Reference Furukawa, Horikoshi, Kawakami, Kadota, Sasaki, Sekiya, Hosogoshi, Kashimura, Asano, Terashima, Iwasa, Nagasaku and Grothaus2012). I am sure that there are many other followers of Cuijpers et al's recommendations among those who conduct psychotherapy trials today. Future trialists in this field should wisely learn from this guideline.
The second group of users would be people, again like me, who conduct systematic reviews and critical appraisal of psychotherapy literature. Traditional systematic reviews nowadays usually pay good attention to risks of bias pertaining to proper randomisation, blinding or intention-to-treat principle and publication bias. It is probably then surprising if a recent systematic review did not point out the risk involved in non-blinding when they reviewed internet CBT for anxiety disorders by claiming that it cannot be avoided in this field (Olthuis et al. Reference Olthuis, Watt, Bailey, Hayden and Stewart2015). Other aspects that may lead to overestimation of efficacy of a psychotherapy have traditionally not received enough attention in systematic reviews. Some reviews still lump various control conditions into one comparison and do not take the effect of researcher allegiance into account when they synthesise evidence (Butler et al. Reference Butler, Chapman, Forman and Beck2006). In fact, more trials are at high risk of bias for researcher allegiance recently than before (time trend p < 0.01) (Chen et al. Reference Chen, Furukawa, Shinohara, Honyashiki, Imai, Ichikawa, Caldwell, Hunot and Churchill2014). Future reviews in this field should now regularly pay good attention to the factors enumerated by the guideline of Cuijpers et al.
The practicing psychologists and psychiatrists should form the third group of users, even when they do not themselves develop therapies, run trials of therapies or conduct systematic reviews of trials. Using this guideline, they can now more critically and validly appraise the evidence base of the psychotherapies that they practice. For example, I can now recommend this guideline and urge my colleagues who are all practicing psychiatrists or psychologists when we review research studies in our journal club.
But at the end of the day, how should the ultimate consumers of medical literature, namely patients, their families and the policy makers, use this guideline? Do they remain at the mercy of the bulk of literature consisting of original RCTs that followed this guideline, of systematic reviews that ignored this guideline in their evidence synthesis, and of practicing psychotherapists who may be all too easily convinced of the effectiveness of the therapies that they practice?
I leave the answer to each and every reader of Epidemiology and Psychiatric Sciences. Prime time has come for the world psychotherapy, psychology and psychiatry academia to move forward so that this guideline will be obsolete and unnecessary, soon.