Introduction
Ninety percent of people presenting to health care services with mental health difficulties have their care managed only in the primary care setting (Goldberg and Huxley, Reference Goldberg and Huxley1992; Paykel and Priest, Reference Paykel and Priest1992). Medication has tended to be the mainstay of treatment, but practitioners also use a range of other consultation or counselling skills (Olfson, Weissman and Leon, Reference Olfson, Weissman and Leon1995). Cape, Barker, Buszewicz and Pistrang (Reference Cape, Barker, Buszewicz and Pistrang2000a, Reference Cape, Barker, Buszewicz and Pistrangb) noted that these skills have therapeutic potential, but concluded that current psychological management of emotional problems by GPs lacks an evidence-base. Additionally, 30% of people who see their GP will have some mental health component to their physical illness (Goldberg and Huxley, Reference Goldberg and Huxley1992; Tiemans, Ormel and Simon, Reference Tiemens, Ormel and Simon1996). It seems surprising; therefore, that we know so little about which aspects of psychological management will help these patients. It is also notable that, given the prevalence of mental health difficulties in primary care, there is so little in the way of systematic postgraduate training in primary mental health care in the UK (The Sainsbury Centre for Mental Health has highlighted the need for a national primary care mental health training strategy: SCMH, 2002). One difficulty in developing such a strategy is that we do not yet know which psychological treatment approaches are most effective in the primary care setting.
Cognitive behaviour therapy (CBT) has been rigorously researched in secondary care settings (e.g. Kovacs, Reference Kovacs1980; Teasdale, Reference Teasdale1985; Clark, Salkovskis and Ost, Reference Clark, Salkovskis and Ost1997). It also has high face validity for workers in primary care (France and Robson, Reference France and Robson1997), and may be delivered as a brief intervention (e.g. Barkham, Shapiro, Hardy and Rees, Reference Barkham, Shapiro, Hardy and Rees1999; Mynors-Wallis, Reference Mynors-Wallis1996). There is also limited evidence as to its effectiveness in the primary care setting (Cape, Reference Cape1996; Mynors-Wallis, Reference Mynors-Wallis1996; Scott, Tacchi, Jones and Scott, Reference Scott, Tacchi, Jones and Scott1997), but a study by King, Davidson, Haines, Sharpe and Turner (Reference King, Davidson, Haines, Sharp and Turner2002) has cast doubt on its usefulness in routine general practice.
CBT training in primary care would therefore seem a promising solution, as indicated by Williams (Reference Williams2002), who surveyed 42 general practice vocational training schemes, reporting that only a small number of these schemes provided any formal teaching of CBT. However, scrutiny of CBT training in routine general practice settings justifies a pessimistic view.
Studies of CBT training in primary care
Gask, Williams and Harrison (Reference Gask, Williams and Harrison1995) described CBT training in general practice for trainee GPs. They received 8 weekly one-day training sessions in CBT, and results indicated that CBT skills could be taught successfully, but cannot necessarily be applied to post-graduate, mainstream general practice settings.
Gask, Usherwood, Thompson and Williams (Reference Gask, Usherwood, Thompson and Williams1998) detailed the evaluation of training for 20 experienced general practitioners, who received 5 two-hour sessions. Positive changes in their confidence in using a talking therapy, and improvement in their management approaches, were reported, though there was no use of the trained skills after training. Davidson, King, Sharpe and Taylor (Reference Davidson, King, Sharpe and Taylor1999) described a randomized trial, with trainee GPs. A two-day course was supported by three two-hour supervision sessions over the following three months. The numbers in the study were small (intervention group 8, control group 7), but some changes were found in confidence, reported use of “verbal counselling techniques”, and in mood diaries. The authors concluded affirmatively that the effectiveness of CBT training merited further investigation.
Studies of training and clinical outcome in primary care
Morriss et al. (Reference Morriss, Gask, Ronalds, Downes-Grainger, Thompson and Goldberg1999) trained 8 GPs and assessed the outcomes for 112 patients (compared with 103 controls), all of whom presented with somatized mental disorder. The study showed that a CBT management approach improved outcomes in “part-somatizers” (i.e. for those psychologically-minded participants). Morriss et al. (Reference Morriss, Gask, Ronalds, Downes-Grainger, Thompson and Goldberg1998) also described successful training of GPs in CBT management approaches, providing an economical evaluation. But King et al. (Reference King, Davidson, Haines, Sharp and Turner2002) showed that CBT training (for 84 GPs) resulted in no better knowledge of, or attitudes towards, depression, relative to a control group. Training also had no discernable impact on patients' outcomes.
In summary, the stark conclusion from the results of literature searches on CBT training in primary care is that there is only limited evidence in the literature that brief training in CBT is effective in changing knowledge or practice by clinicians in routine primary care, with the exception of the adoption of some behavioural approaches. This may be in part because of the difficulty of conducting research in routine primary care settings, but there are also some clear weaknesses within the above literature. For example, in King et al. (Reference King, Davidson, Haines, Sharp and Turner2002) only three sessions were spent on CBT approaches, carried out in quite large groups, and outside the individual practices. No opportunity for case discussion or ongoing support was offered, unlike previous successful training of GP registrars (Davidson et al., Reference Davidson, King, Sharpe and Taylor1999). Furthermore, they appear to have been technique (rather than conceptualization) driven. A further difficulty of the King et al. (Reference King, Davidson, Haines, Sharp and Turner2002) study was that the patients included might not have been suitable for brief CBT.
Additionally, none of the reviewed studies have described the level of expertise of the CBT trainers. Given these considerations, it is perhaps not surprising that there is little evidence of effective CBT training in primary care. Even with more systematic training efforts, there is only limited evidence of effective CBT training in secondary care (Myles and Milne, Reference Myles and Milne2004), and indeed in other areas of staff training (Milne, Keegan, Westerman and Dudley, Reference Milne, Keegan, Westerman and Dudley2000).
Objectives
In this context of poorly designed training, this paper describes a programme based on current CBT approaches, and exploratory evaluation of brief CBT training in primary care. We aim to:
1. Outline an evidence-based training package for primary care practitioners.
2. Illustrate how this can be evaluated.
3. Reflect on ways forward.
Method
The training
The first author designed the three and a half hour (1 session) workshop, being the maximum period that could be negotiated for full primary care teams. The content was aimed to be formulation-driven rather than disorder specific. The course followed a similar structure to PRAXIS, an interactive CD-ROM for teaching foundational level CBT (Myles, Reference Myles2003) and the foundational level course training described by Myles and Milne (Reference Myles and Milne2004). This seemed important, as interested members of primary care may be expected to go on to expand their training via this resource. The workshop session was developed into a training package, with visual material, course leader's manual (CBT: the important bits for primary care – copies available from the authors), practitioner's manual, patient support materials, and video vignettes.
Measures
Kirkpatrick's (Reference Kirkpatrick, Craig and Bittel1967) outcome evaluation framework was partly adopted, i.e. covering the acceptability of the training, knowledge gain, and self-reported behaviour change in practice (we did not attempt to measure impacts on the care system). Three validated questionnaires were adapted for the study, while the remaining two measures had to be specifically devised. The measures took about 15 minutes to complete, and were developed with consultation, review and discussion involving eight supervisor-level CBT practitioners, four consultant-level clinical psychologists, and three general practitioners with an interest in CBT (the expert group).
Measures of acceptability
Two questionnaires of learner's satisfaction with the training were developed. The first questionnaire (administered immediately post-workshop) examined satisfaction using a Likert scale format, plus the opportunity for structured individual comment on various elements of the workshop. Questions covered: understanding; application of skills to the workplace; and overall satisfaction with the training. The second acceptability questionnaire (administered at the end of the full training package) looked at attitudes to both the workshop and to the case discussion element of the training.
Measure of knowledge 1
A 10-item multiple-choice questionnaire (CBT-PC MCQ) was used, based on Myles and Milne (Reference Myles and Milne2004). Their questionnaire had good face and content validity, and test-retest reliability was also favourable (r = 0.85). The questionnaire was adapted, with permission, to suit the primary care training and the course content. The expert group confirmed the face and content validity of this declarative knowledge assessment tool. Test-retest reliability (two week interval) was r = 0.96 (p < .01) based on seven participants. Trainees were given the questionnaire before the workshop, after the workshop, and at the end of the case discussion part of the course. The trainees were asked to answer every question, and to choose one correct answer from the four choices.
Measure of knowledge 2
A procedural knowledge questionnaire was developed, which asked the trainee to describe what he or she would do in CBT-relevant clinical interview situations; this was based on Milne (Reference Milne1984) but adapted to suit, again in consultation with the expert group. The inter-rater reliability of this scale was r = 0.94 (p < .01), based on N = 12 data sets.
Measure of implementation of learning in practice
Although a direct observational approach would be the best way to estimate change in practice, it was not practical for this study. There were no available scales “off the shelf” to measure self-reported CBT practice; therefore there was again a need to develop or adapt a suitable instrument.
The Cognitive Therapy in Practice Questionnaire was developed from a secondary care questionnaire devised by Twaddle (Reference Twaddle2000). The adjustments were carried out to ensure that the scale was more suitable for the primary care setting, following consultation with the expert group. The scale asks the practitioner to estimate the number of patients with whom the cognitive therapy approaches covered within training had been used over the last two-week period. The practitioner was also asked to estimate how many patients he or she saw each week over the previous two weeks.
The trainers
Nine members of the Department of Psychological Services (Northumberland Locality), who were of supervisor standard in CBT (levels 4 and 5, supervisor or specialist level, in Northumberland CBT Resource Directory; Moore and Findlay, Reference Moor and Findlay2003), were invited to take part in the study, both as potential primary care trainers (and as consultants for credibility checks on materials to be produced). Five supervisors agreed to be trainers. This included three clinical psychologists, one nurse therapist, and a counselling psychologist. All of these practitioners were also experienced in work in primary care. The trainers attended a three-hour interactive workshop, to prepare them to lead the CBT training and case discussion course in primary care. The supervisors were given course materials and the trainer's manual. They were allocated to practices taking part in the training.
Participants
Four Northumberland primary care localities took part in the training, chosen because staff in these practices had previously shown an interest in CBT or attended earlier events. A novel feature of the present study was that these teams were offered the training in their own practice.
In all, 32 practitioners took part in the training. However, only those who took part in the workshop plus at least three of the four case discussion groups, and who completed and returned all assessment forms, were included in the study. This number was 25, made up of a mix of professions, but predominantly general practitioners.
Procedure
Ethical approval was received for this study from the LREC. Workshop leaders followed the pre-prepared structure described within the training manual wherever possible. The workshops lasted between 3–4 hours. The evaluation took place before and after the workshops. The evaluation questionnaires were also used as a way of priming the course participants to essential elements of course content.
As CBT case discussion groups in primary care are a novel approach, there was no pre-prescribed structure for the groups, and participants tended to choose cases with clear links to CBT theory, or chose to review particular areas of theory and applications of CBT in primary care as their main areas of attention during the sessions.
Statistical analysis
Most of the data collected in this study were interval data (i.e. number of patients; number of correct responses; the rating data on the satisfaction questionnaires can only be seen as ordinal data, and as such were subjected to non-parametric test analysis). The Kolmogorov-Smirnov test was used to assess normality of distributions for data analysed using parametric tests. This test indicated that data collected by the main questionnaires of the study were normally distributed.
A power calculation was carried out to help to estimate a sample size that would ensure adequate power for this simple before and after design. The results suggested that a sample of n = 14.3 would be sufficient; (power = 0.8, effect size mu-mu/s = 0.8 alpha = .05, sigma = 1).
Results
Sixteen participants completed and returned forms for the second part of the training, which left this part of the study just over the number required (15) from our power calculation. Table 1 summarises the sample.
Table 1. Demographic details of primary care professionals in the study (n = 25)
Satisfaction with the training and the training materials
The results from the two acceptability scales suggest that the training was acceptable to participants, both after the initial workshop and at the end of the training.
Measures of knowledge
Table 2 summarises the results for the measures of knowledge before and after the initial workshops. There were highly significant increases in knowledge on both questionnaires following the initial CBT workshop. The greatest gains shown in Table 2 were on the procedural knowledge questionnaire, where mean scores increased from 5.2 to 12.9.
Table 2. Knowledge mean scores before and after the workshops (N = 25)
There were highly significant gains on knowledge scores after the course (Table 3); this was particularly notable on the procedural knowledge questionnaire, where mean scores rose from 5.2 to 16.3.
Table 3. Measure of knowledge mean scores before and after the whole course (N = 16)
Measures of implementation of learning in practice
One important aim of this study was to see if what had been learned in the training would be applied in practice. Table 4 shows mean results for the In Practice questionnaire before and after the training.
Table 4. Global scores on the In Practice Questionnaire before and after training (n = 15)
Results from this questionnaire indicated a highly significant move to more self-reported use of CBT techniques, with more patients in practice following the training. It is important to ensure that these results are not simply a confound, due to an increase in the numbers of patients being seen by our practitioners at the end of the training period. Therefore, we conducted a repeated measures t-test on the estimates of numbers of patients seen before and after the study. This indicated that there was no significant difference in patient numbers between the two time points.
The CBT areas addressed within the training that were most used at the end of the study were: problem solving, giving of CBT patient self-help, and using some elements of the CBT approach.
Types of cases discussed in the sessions
Leaders collected details of the types of cases presented at the case discussion sessions: these are varied both in complexity and in type of presentation/diagnosis. No patients with severe/enduring mental health problems were presented for discussion during the sessions. Depression was a common presentation, but often co-morbidly presenting with other difficulties.
Discussion
Gask et al. (Reference Gask, Usherwood, Thompson and Williams1998) reported that general practitioners failed to use the cognitive elements of the approach, following their brief CBT training programme. By contrast, the present sample of practitioners reported a statistically significant increase in sharing the CBT model with their patients, and in their use of CBT formulations. Cognitive approaches were also reported as being used. The most popular approaches from the CBT training menu, accounting for more than half of the CBT activity, were: problem solving; use of patient information; use of elements of CBT; and sharing the cognitive model with the patient.
These relatively favourable findings require an explanation. The fact that all trainers were experienced trainers/supervisors in CBT may have increased the effectiveness and credibility of the training. Data on the level of expertise of trainers, or satisfaction with the competence of trainers, have not been reported in previous published studies of CBT training in primary care. It may be important to have skilled CBT supervisors (rather than practitioners) to deliver such training. Another possible explanation is the carefully structured content. Cape et al. (Reference Cape, Barker, Buszewicz and Pistrang2000a, Reference Cape, Barker, Buszewicz and Pistrangb) postulated that three components of psychological treatments, common to both primary and secondary care, were necessary to make progress in the psychological management of mental health problems in general practice. These three core components are all present in the self-reported practice of our small study group, namely establishing a therapeutic relationship, assisting the patient in understanding his problems and promoting change in behaviour, thoughts or emotions.
As trainers we have constantly emphasized assisting the patient to understand their own difficulty, for patients with less severe problems. The rationale is that the the practitioners reported greatest change in the area of using “some elements of the CBT approach”. It may be that given the very different time frame available to primary care practitioners, an adapted version of secondary care CBT is all that is possible.
Critical review
One limitation of the study is that, of 25 primary care clinicians entering the study, only 16 returned data for the duration of the whole study. This may have introduced sampling error into the results. Other important weaknesses were the reliance on self-report instruments, and the pre-post research design. These mean that we may have received favourably biased information, and that we cannot infer that the training interventions were responsible for the obtained findings. Third factors, such as incidental learning and the motivational impact of the study, may actually have produced the improvements. Future research should overcome these and the other weaknesses within the present study by utilising some objective measures of learning and transfer, such as supervisor ratings of competence in using CBT, audits of case notes, and direct observation of clinical practice (see Milne, Reference Milne2007 for a review of such measures). In terms of introducing a research design that permits causal inferences to be drawn about the effects of training, in the practice context often the only feasible option is to introduce a double baseline assessment. This should have an interval approximating to the training period, to minimize other threats.
Implications of the findings for General Practice
Previous studies of brief CBT for primary care have yielded disappointing results, and have potentially discouraged postgraduate training, in particular of the cognitive elements of this approach (e.g. Gask et al. Reference Gask, Usherwood, Thompson and Williams1998). One obvious way to remedy this is to include more robust CBT training at trainee level for general practitioners, health visitors and possibly other clinical members of the primary health care team. Implications from the present study are to utilise experienced CBT supervisors; to consider patient selection carefully, and to ensure the opportunity for practitioners to have case discussion on a regular basis.
However, the present study suggests that, even with brief CBT training, experienced practitioners can absorb and apply the concepts of the approach, and that changes in practice are then reported. Practitioners also remain enthusiastic about the approach, but time commitments (both in attending training and in applying the techniques) continue to be a difficulty.
Conclusion
The present study indicates that improvements in procedural and declarative knowledge, and small changes in routine practice, may occur following brief CBT training of experienced practitioners. This is a relatively favourable outcome, which we attribute to the use of experienced CBT supervisors, careful programming and mixed learning methods, adherence to a training manual, and to the careful selection of patients. However, the methodological limitations of the present study indicate the need for a more rigorous trial of the approach, before undertaking studies of patient outcomes.
Acknowledgements
Thanks are given to Lainey Fraser, Catriona Grey, Maureen Leyland, Gerry Morrow and Pamela Myles.
Comments
No Comments have been published for this article.