Introduction
Clinical supervision has made significant progress over the past two decades, including an emergent international consensus on competencies, better measurement options, and improved feedback systems (Watkins & Milne, Reference Watkins, Milne, Watkins and Milne2014). However, much remains to be done and a key challenge is developing supervisory competence through enhanced training, then sustaining it through suitable support procedures (Milne & Reiser, in press). Such training transfer efforts require an approach that is ‘competency-based, evidence-based, particularized, and accountable’ (Watkins, Reference Watkins2012, p. 193). Within the same special issue as Watkins (Reference Watkins2012), Milne & Reiser (Reference Milne and Reiser2012) specified supervision guidelines as part of a cycle of activities that could contribute to improved training transfer. However, the need for guidelines and manuals has been long recognized within clinical supervision, in terms of informing training, enhancing our understanding of the training intervention, and in enabling high-fidelity research (Watkins, Reference Watkins and Watkins1997). Rather, the dearth of sound guidelines and manuals may well illustrate the general neglect of supervisor training, as reflected in the oft-quoted refrain that ‘something does not compute’ (i.e. the illogical discrepancy between the professional importance of supervision, and the neglect of training and support; Watkins, Reference Watkins and Watkins1997, p. 604). By contrast, clinical practice guidelines have increased significantly in number recently (Beauchamp et al. Reference Beauchamp, Drapeau and Dionne2015), and have become prominent through the Improving Access to Psychological Therapies (IAPT) programme (Clark, Reference Clark2001). It seems that improved written guidance may also be one way to make things compute in relation to clinical supervision. In this review, the aim is to consider how manuals and guidelines can contribute to improving cognitive-behavioural therapy (CBT) supervisors. The review objectives are to clarify how manuals and guidelines are defined; to list their intended functions; to acknowledge the pitfalls and barriers to success; to consider the promising boosters to success; to conduct a review, to summarize and evaluate the currently available resources; and to draw conclusions on how we might best improve written guidance on CBT supervision in future. The evaluation is based on the criteria used to judge complex interventions (Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009), particularly the innovation challenges of dissemination and implementation.
Definitions of manuals and guidelines
What do we mean by the terms ‘manual’ and ‘guideline’? Partly because manuals and guidelines are rare within the supervision literature, and partly to tease out the best-available approaches, this review draws on definitions within therapy, supervision and training (i.e. Dobson & Shaw, Reference Dobson and Shaw1988; Kendall et al. Reference Kendall, Chu, Gifford, Hayes and Nauta1998; Chambless & Ollendick, Reference Chambless and Ollendick2001; Beauchamp et al. Reference Beauchamp, Drapeau and Dionne2015). This is not to assume that these different activities are the same, and important differences will be noted. However, these diverse activities broadly share the same definitions. Training manuals are written statements designed to inform and facilitate the trainers’ capacity to lead experiential workshops. For instance, a manual may suggest programmes of activity (e.g. learning objectives; links to competence frameworks), may suggest learning exercises, or may link to training resources (e.g. slide shows, video material, supervision guidelines). The goal is implementation fidelity: the consistent repetition of a particular approach to training (e.g. across different trainers and settings). By contrast, therapy manuals typically add an emphasis on adherence to a theoretical model, and may be more prescriptive and detailed. But like clinical guidelines, manuals are usually aspirational, recommending some training practices, but ultimately encouraging the trainer to exercise professional judgement in drawing on ideas and materials within the manual, as appropriate to a particular context and participant group. That is, manuals are typically support tools which package the best-available knowledge (research evidence, theory, and expert's judgements about best practice), and resources (training aids and materials), making them readily accessible to trainers in a non-prescriptive and flexible style, intended to gently guide practice development.
By contrast, guidelines are systematically developed recommendations that are based on the best-available research evidence moderated by expert consensus, intended to change processes and thereby improve outcomes on a national scale (NICE, 2012). These recommendations are designed to be read by all interested parties, and are thought to help by improving communication, by clarifying standards, by enabling staff training, and by enhancing decision-making. In terms of decision-making, a guideline (like a manual) should capture the distilled essence of the available knowledge, presented in a form which represents a service to the reader by highlighting key variables in a practical context, while acknowledging the quality of the available research evidence.
Guidelines are much more common than manuals in clinical supervision, though there will typically be considerable variation in the way that these are developed or presented. At one end of the spectrum are informal guidelines, which have not been developed systematically and rely solely on the author's experience (e.g. Pretorius, Reference Pretorius2006; Gordon, Reference Gordon2012). There is usually no collaborative process of development, nor any institutional endorsement, nor any large-scale dissemination or implementation scheme. For instance, most books on supervision include some guidelines, and some organizations issue guidance from time to time, written by an expert and issued through the organization (e.g. the BABCP; Lewis, Reference Lewis2005). By contrast, a formal guideline is developed through a collaborative process that may be highly systematized (e.g. expert consensus-building; systematic literature reviews), and widely disseminated, for example the NICE system in the UK (Leng et al. Reference Leng, Baillie and Raj2008) and the Agency for Healthcare Research and Quality in the USA. A rare example in the supervision literature is the four guidelines developed by Milne & Dunkerley (Reference Milne and Dunkerley2010). In summary, while it is possible to draw some clear and important differences between manuals and guidelines across different professional activities, it is also the case that significant overlaps exist. This means that in some cases it becomes difficult to distinguish them, and for this reason in this review the term ‘written guidance’ will often be preferred to subsume both manuals and guidelines (and variants thereof). In this confused situation, it is reassuring to note that all such materials reviewed below at least share the goal of improving CBT supervision.
The intended functions of manuals and guidelines
Whether distinctive or overlapping, what can we expect of manuals and guidelines? Manuals have a strong history within CBT, reflecting the CBT emphasis on precision, technique, and quantification (Dobson & Shaw, Reference Dobson and Shaw1988; Wilson, Reference Wilson1996). Just as in manual-based therapy (King & Ollendick, Reference King and Ollendick2008), high-quality training implies a training manual as part of the ‘gold standard’ for effective training, alongside an experiential workshop and linked clinical supervision (Sholomskas et al. Reference Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro and Carroll2005). However, manuals which are not supported by complementary training methods are unlikely to have any effect (Miller et al. Reference Miller, Yahne, Moyers, Martinez and Pirritano2004; Sholomskas et al. Reference Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro and Carroll2005). Manuals appear to be valuable in part because they may limit therapist drift (i.e. low adherence or fidelity to the intervention). To illustrate, based on an online survey of over 100 clinicians using family-based therapy (FBT) for eating disorders, more than half of the sample failed to adhere to FBT guidelines (Kosmerly et al. Reference Kosmerly, Waller and Robinson2015). But those who said that they had referred to the therapy manual were significantly more likely to adhere to the therapy protocol.
Similarly, a ‘pressing need’ in the advancing field of clinical supervision is thought to be manual-driven education, linked to transfer of training issues (including supervision-of-supervision; Watkins, Reference Watkins2014). Supervisor training manuals are rarely as well developed or as rigorously evaluated as in the staff training field, but there is reason to believe that they can be beneficial, when combined with appropriate training methods (Miller et al. Reference Miller, Yahne, Moyers, Martinez and Pirritano2004; Milne, Reference Milne2010), possibly helping to achieve good clinical outcomes (Sholomskas et al. Reference Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro and Carroll2005). Although difficult to implement, from the available evaluations (almost entirely in general medicine) it appears that clinical guidelines also ‘have a positive impact on quality’ (Leng et al. Reference Leng, Baillie and Raj2008, p. 355) and on processes of care, linked to significant improvements in clinical outcomes (Grimshaw & Russell, Reference Grimshaw and Russell1993; Beauchamp et al. Reference Beauchamp, Drapeau and Dionne2015). In general, clinical benefits are likely to accrue when these gold-standard elements are combined with suitable systemic work to support transfer (Beidas & Kendall, Reference Beidas and Kendall2010; Milne & Reiser, in press).
In addition to potentially raising the quality of care and improving clinical outcomes, manuals and guidelines also hold promise as they can:
-
• provide a clear and transparent, operational definition of training and supervision, enabling dissemination, replication in research, and consistent implementation in practice (e.g. by comprehensively defining the content of a training programme);
-
• indicate the resources that are required (including ‘doses’ of training elements; number and nature of supervision sessions);
-
• enable fidelity to be assessed (enable an adherence check regarding whether an intervention was manipulated correctly);
-
• indicate specifically how competencies can be developed (e.g. detail experiential learning exercises; provide descriptions of supervisory techniques);
-
• enable observation and the rating of competence;
-
• maximize attention to the active factors (mechanisms, such as active reflection) and minimize the non-specific factors;
-
• highlight the anticipated mini-impacts (e.g. awareness-raising through reflection);
-
• improve adherence and transfer (e.g. through providing detailed recommendations and illustrations, such as video demonstrations);
-
• enhance clinical effectiveness.
These intended functions are based on a review of therapy guidelines (Addis & Krasnow, Reference Addis and Krasnow2000). They can be re-stated more systematically in terms of five successive levels of intervention ‘fidelity’ (Borelli et al. Reference Borelli, Sepinwall, Ernst, Bellg, Czajkowski, Breger, DeFrancesco, Levesque, Sharp, Ogedegbe, Resnick and Orwig2005). On this logic, the first function that written guidance can fulfil is to articulate a conceptualization or model: the ‘design’ that underpins an intervention like therapy or supervision. The next fidelity challenge is ‘adherence’; did the intervention correspond to the design? If levels one and two are achieved, the next issue is ‘implementation’: was the intervention conducted skilfully? This leads logically to asking about the effect of these three aspects of fidelity: did they have the intended ‘outcome’? Last, Bellg et al. (Reference Bellg, Borelli, Resnick, Hecht, Minicucci, Ory and Czajkowski2004) suggest that we also need to know about the ‘impact’; did an intervention's outcome generalize? Applied to written guidance, this definition of fidelity clarifies the different functions that can be achieved and highlights the necessity of addressing all five criteria in order to improve an intervention like supervision.
We can make this fidelity logic more concrete, and link it to CBT supervision, by considering how each of these five levels relate to the current issues in supervisor development. The ‘design’ aspect relates to the model of CBT supervision, as expressed in expert review of the literature (e.g. Reiser, Reference Reiser, Watkins and Milne2014). ‘Adherence’ relates to competence frameworks, specifying what should be done in the name of CBT supervision (e.g. Roth & Pilling, Reference Roth and Pilling2008). In this sense, competence is a demonstration that the right thing (i.e. the given model) has been done right (i.e. with adherence). Competence is the combination of measurable knowledge, skills and attitudes that are required to perform an occupational role to the standard required in employment (Falender et al. Reference Falender, Shafranske and Ofek2014). An example that they provide is ‘Forming a supervisory alliance’. ‘Implementation’ concerns the proficiency with which the competencies are conducted (e.g. Milne et al. Reference Milne, Reiser, Cliffe and Raine2011a ). The ‘outcome’ of these fidelity steps can be measured by mini-outcomes (like the supervisee's learning; Milne et al. Reference Milne, Reiser, Cliffe and Raine2011a ), but is more commonly judged by reference to clinical outcomes, such as reduced symptomatology. ‘Impact’ is less often assessed in CBT, but could include indicators of maintenance and generalization, such as the effect of supervisor training on supervision, and in turn on the supervisee's therapy (Reiser & Milne, Reference Reiser and Milne2014).
Pitfalls and barriers to success
However, as detailed below, these potential benefits have rarely been reaped within supervisor training, due to the limited extent and quality of research. Which additional barriers exist? Another reason for the paucity of sound manuals and guidelines is the generally negative perceptions held by the majority of mental health practitioners. For example, they have been regarded as encouraging a ‘cook-book’ mentality, and of placing practitioners in a professional straightjacket (King & Ollendick, Reference King and Ollendick2008). They may also be regarded as inflexible, stifling creativity and adversely affecting practice (Gupta & Warner, Reference Gupta and Warner2014). To illustrate such negative perceptions, Gyani et al. (Reference Gyani, Shafran, Layard and Clark2013) summarized interviews with 33 CBT and other therapists in the UK. These therapists preferred to rely on their clinical experience or on theory when making clinical decisions, while regarding research evidence as suspect (e.g. due to using unrepresentative samples) and prizing applicability over evidence. These interviewees, echoing the earlier observations by Dobson & Shaw (Reference Dobson and Shaw1988), noted that manuals neglect therapist variables (such as the therapeutic alliance), and may similarly overlook other critical aspects of competence (e.g. expertise).
For such reasons, guidelines and manuals have had a troubled implementation history across the entire healthcare spectrum, rarely being assimilated readily or fully into the professional practice of any discipline anywhere (Carroll & Nuro, Reference Carroll and Nuro2002; Grol & Grimshaw, Reference Grol and Grimshaw2003). To illustrate, according to a survey of 164 psychotherapists and trainee clinical psychologists (Lucock et al. Reference Lucock, Hall and Noble2006), therapy manuals and evidence-based guidelines were rated as relatively unhelpful, though CBT adherents within this sample gave them a significantly higher rating. Overall, clinical supervision was rated as the most helpful influence on their clinical practice (5.2). A survey by Addis & Krasnow (Reference Addis and Krasnow2000) also reported that CBT adherents held a significantly more positive attitude to manuals and guidelines, though the actual reported frequency of their use was strikingly low (only 5% of respondents used manuals ‘often’ or ‘exclusively’. A more recent survey corroborated these findings, again reporting that CBT practitioners were the most favourably disposed to manuals, but indicating that fewer than 10% of therapists made routine use of manuals (Becker et al. Reference Becker, Smith and Jensen-Doss2013). By neglecting to detail their training and supervision activities, researchers have not helped to foster EBP. For example, only 15 of the 31 trials of CBT for schizophrenia that contributed to the draft NICE guidelines (2013) detailed how the therapists were trained. This guideline concluded that therapists could have entered their studies with different levels of competence, and the impact of any specific training was ‘impossible to determine’ (p. 248). Supervision fares no better. Roth et al. (Reference Roth, Pilling and Turner2010) scrutinized 27 CBT trials, finding that 11 did not state how frequently supervision occurred, and only 14 clarified the format (e.g. 1:1).
Fulfilling the promise: boosters to success
To place these findings in context, the contents and delivery of complex interventions like supervision are rarely described with sufficient detail to permit replication (Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009). Recommendations have been developed to improve such reporting (http://interventiondesign.co.uk). This guidance indicates that there are eight broad themes or characteristics that should be specified: the intervention's content (aims and objectives; competencies); setting, provider and receiver characteristics (resources; context; participants); mode of delivery (formats, such as 1:1; group; remote); the intensity (e.g. duration); and adherence (Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009). To these could be added something about the appropriate evaluation focus: What are the intended outcomes of the intervention? (i.e. is it targeting thoughts, feelings or behaviours? Who should the intervention reach?, and, What degree of generalization is intended?; Chorpita et al. Reference Chorpita, Daleiden and Weisz2005). Written guidance should ideally address all of these characteristics. For example, a training manual can specify learning outcomes and incorporate related evaluation instruments. Guidelines can clarify the desired nature of supervision and how adherence might best be assessed. Table 1 provides an illustration of applying these eight characteristics to a supervision manual, illustrating the richness of the information that can be obtained.
Table 1. Specifying a manual's content, based on the complex intervention approach (Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009)
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary-alt:20160921100722-33209-mediumThumb-S1754470X15000720_tab1.jpg?pub-status=live)
Material in the Table is based on Reid et al. (Reference Reid, Rotholz, Parsons, Morris, Braswell, Green and Schell2003).
Following some preliminary clarifications, I next review the available research on supervision manuals and guidelines, analysing them primarily by reference to these eight characteristics, closing with a consideration of the implications for the optimal approach to written guidance.
Review of the available supervision manuals and guidelines
This section summarizes the available supervision manuals and guidelines. It appears that there are very few formal manuals for training supervisors. Scrutiny of the five located randomized controlled trials (RCTs) of supervision suggests that they are absent or limited even where RCT convention dictates a manual (e.g. Uys et al. Reference Uys, Minaar, Simpson and Reid2005; Heaven et al. Reference Heaven, Clegg and Maguire2006; Bradshaw et al. Reference Bradshaw, Butterworth and Mairs2007; White & Winstanley, Reference White and Winstanley2009). To illustrate, in a further RCT, manuals were developed to guide supervisors in their application of one of two approaches to fostering the therapeutic alliance, a ‘process focus’ or a ‘skill focus’ (Bambling et al. Reference Bambling, King, Raue, Schweitzer and Lambert2006). However, these were more like a general framework than instructions, with some theory, case examples and encouragement to practise (M. Bambling, personal communication, 30 April 2014). In short, none of the five RCTs which manipulated supervision actually included a sound supervision manual. Likewise, no formal manual was located within the 11 controlled studies of supervisor training that were reviewed by Milne et al. (Reference Milne, Sheikh, Pattison and Wilkinson2011b ), although several papers described the training in considerable detail. For example, the supervisor training reported by Ducharme et al. (Reference Ducharme, Williams, Cummings, Murray and Spencer2001) approximated to a manual, but without that term being used. Their research paper described the training content and methods, making replication a possibility (and fully satisfying two of the eight recommendations in Table 1), but, in addition to omitting key details, the study concerned implementation and minimized research rigour (including evaluations of outcome), a recurring theme within this literature. Similarly, the systematic review of supervision's clinical outcomes by Reiser & Milne (Reference Reiser and Milne2014) indicated that manuals were rare, and that only six of the twelve reviewed studies even specified the elements of supervisor training. More positively, ten of these studies referred to a guideline or a model or theoretical rationale, and there were signs that such details were increasingly reported over time. But overall, Reiser & Milne (Reference Reiser and Milne2014, p. 154) concluded that there was ‘a disturbing deficiency in specifying, implementing and documenting training practices’, which they considered reflected the limited progress in manualizing supervisor training.
In contrast with the above omissions, I next review the available published studies of written supervisor guidance (i.e. supervisor training manual, supervision guidelines, or both). The aims are to clarify what we currently know, and to identify how we can best make progress. I will describe all such written material, and where the information is available I will describe how it was developed and evaluated, then offer a critique, in relation to recommended practice (i.e. Chorpita et al. Reference Chorpita, Daleiden and Weisz2005; Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009). However, I should stress that generally this literature is scientifically weak, in that much of this information is missing.
The earliest work included developing a supervisor training curriculum, using research and expert consensus to consider how the professional standards within counselling supervision could best be attained (Borders et al. Reference Borders, Bernard, Dye, Fong and Nance1991). The first full supervisor training manual was reported by Neufeldt (Reference Neufeldt1994) and was designed to operationalize Bernard's discrimination model and to foster reflective practice. Helpfully, the manual included transcripts and detailed, step-by-step procedures for 26 supervisory well-defined skills (e.g. ‘assist supervisee to conceptualize case’, ‘present a developmental challenge’). It was intended to be used in conjunction with weekly seminars, including educational role-plays and reflections on video-presented material.
Other supervision manuals include those by Baltimore & Crutchfield (Reference Baltimore and Crutchfield2003), and Fall & Sutton (Reference Fall and Sutton2004). Some of these manuals emphasize the supervisor more than the trainer, and in turn the topics can be wide-ranging. For instance, Buus et al. (Reference Buus, Cassedy and Gonge2013) described a ‘meta-supervision’ manual that addressed the definition, experiences and the outcomes of supervision. But this manual concerned implementation rather than training, having as its main focus the development of supervisees’ coping strategies in relation to addressing barriers to their effective use of supervision. This study was preparatory to a controlled trial, and provided rich qualitative detail on the practical difficulties in implementing supervision under naturalistic health service conditions. It was also a laudable example of participative action research in shaping up a manual, one that can be viewed as highly complementary to supervisor training manuals and to therapy manuals.
To illustrate a more conventional manual, Reid et al. (Reference Reid, Rotholz, Parsons, Morris, Braswell, Green and Schell2003) described an exceptionally detailed supervisor training curriculum consisting of 26 behavioural modules (e.g. ‘defining behaviour’, ‘prompting’, ‘feedback’). Each module has a summary sheet, equating to one part of a 26-part training manual. Addressing thoroughly the recommendations in Table 1, this sheet specifies the intended learning outcomes, the training methods (a didactic and experiential sequence of some six steps, each with a defined duration), the competencies to be demonstrated, and the support materials. Reid et al. (Reference Reid, Rotholz, Parsons, Morris, Braswell, Green and Schell2003) used direct observation of 12 supervisors to evaluate this training, in relation to the supervisors’ competence in the workshop (during educational role-plays) and subsequently in the clinical environment (e.g. group homes for residents with learning disabilities). Only a minority of these supervisors demonstrated competence at a baseline assessment, but by the end of training all 12 displayed competence, in both environments, which appeared to be due to the training. The supervisors were also asked to evaluate the usefulness of their training and whether they would recommend it to colleagues. Additionally, focus groups were convened 3 months following training, to consider implementation issues (transfer of training). Both evaluation methods yielded very affirmative data (e.g. more positive staff–patient interactions were reported). This manual was subsequently rolled-out across the host state of South Carolina and more than 320 supervisors benefited, 48 of them going on to train as supervisor trainers. This probably makes the Reid et al. (Reference Reid, Rotholz, Parsons, Morris, Braswell, Green and Schell2003) supervisor training manual the most widely disseminated, most precisely detailed, and most systematically evaluated to date. It is certainly an unusually extensive evaluation of the outcomes of supervisor development using written guidance, considering a wide range of learning and generalization objectives, thereby meeting all nine of the desirable characteristics. This is shown in Table 1.
Another state-wide example is from Australia. As part of a strategy to improve mental health nurses’ job satisfaction and quality of care, Hancox et al. (Reference Hancox, Lynch, Happell and Biondo2004) developed an experiential, participative workshop that was provided to 63 nurses over an initial 2-year implementation period. Reaction evaluations were favourable and led to a training manual being developed, which has been updated as the project has continued. However, details of the manual were not provided. A more recent update describes the trials and tribulations of implementing this strategy for supervisor development (Lynch & Happell, Reference Lynch and Happell2008). A second Australian example is the Clinical Supervision Resource Kit (2005), prepared for the ‘alcohol and other drugs’ (AOD) field nationally. This is a substantive package of materials to support the development of supervision within AOD through a one-day workshop and written guidance (to multi-disciplinary supervisors, to their supervisees, and to their managers). The kit includes a full guideline, a demonstration DVD, Powerpoint slides, a sample 1-day training programme, and other written guidance.
The manual developed by Henggeler & Schoenwald (Reference Henggeler and Schoenwald1998) and described by Schoenwald et al. (Reference Schoenwald, Sheidow and Chapman2009) was designed to guide weekly small group supervision in applying multi-systemic therapy in relation to wide-scale dissemination. Also referred to as a ‘supervision protocol’, this document notes the value of supervision, describes the supervision objectives, outlines the recommended structure and processes within a session (including addressing goals and barriers), describes the strategies available to support therapist adherence and to conceptualize cases, together with other supervision and therapy guidelines. This represents an evidence-based, sophisticated, fairly prescriptive approach to supervisor support and development, building on a package of empirically derived materials within a major research programme. Not only does the manual satisfy fully the eight criteria for reporting a complex intervention (Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009), albeit assisted by a second paper (Schoenwald et al. Reference Schoenwald, Sheidow and Chapman2009), but it also includes some exceptional (if not unique) strengths, such as the related adherence checks on supervision and therapy, linked to clinical outcomes. Therefore, this is probably the best available example of a clinical supervision manual.
One of the other uplifting examples of supervisor development has been provided by IAPT, a national programme to extend therapy provision in England (Layard, Reference Layard2005). In terms of training, IAPT supervisors received a 5–7 day workshop, with national guidance on what was required (IAPT Education and Training Group, 2011). This included addressing the supervision competencies (Roth & Pilling, Reference Roth and Pilling2008) and access to training materials. These materials included demonstration video-tapes, supervision checklists, students’ and supervisors’ guides, written support materials (e.g. supervision record sheets) and a reference list (Richards et al. Reference Richards, Chellingsworth, Hope, Turpin and Whyte2010). In addition, these support and development arrangements were supplemented by IT-based case management systems and associated clinical outcome monitoring (Richards & Borglin, Reference Richards and Borglin2011), while during their IAPT training the supervisees had to demonstrate competence in receiving supervision. These impressive features makes the IAPT programme a formidable, state-of-the-art supervision system (see Richards, Reference Richards, Watkins and Milne2014, for a recent summary).
The supervision guidelines developed by Gordon (Reference Gordon2012) and by Milne & Dunkerley (Reference Milne and Dunkerley2010) were incorporated within a 5-day supervisor training workshop, designed for 28 supervisors within the IAPT initiative in England (Newman-Taylor et al. Reference Newman-Taylor, Gordon, Grist and Olding2012). The objectives were to enhance supervision in relation to the full range of competencies defined by Roth & Pilling (Reference Roth and Pilling2008). In addition, these trainers followed the guidance on IAPT training and supervision in Turpin & Wheeler (Reference Turpin and Wheeler2011). Experiential approaches to learning were prominent, and in a Table the authors provided rich detail on the workshop objectives, content and methods. For example, in the module concerned with managing supervision sessions, the objectives were to use supervision contracts, goal setting and record keeping. The methods were small-group educational role-plays on establishing the contract and collaborative goal setting. One of the targeted competencies was: ‘Demonstrate ability to structure supervision’. Although the term was not used, this represents a thoroughgoing training manual. There was also a preliminary, naturalistic evaluation, based on the supervisors’ self-rated competence, assessed before, during and at the end of the workshop. Competence improved significantly over time, across all the domains within the Roth & Pilling (Reference Roth and Pilling2008) framework.
Another IAPT-related project is the impressively systematic programme of supervisor development in Scotland, led by NHS Education for Scotland (NES). This includes a supervision guideline, supervisor training, distance learning modules, and a database (including information on supervisors, educational materials, and updates on developments of clinical supervision; Kyle et al. Reference Kyle, Sloan and Davies2012). The guideline for the clinical supervision offers recommendations for particular therapies, relevant accreditation criteria, and lists specific supervision competencies. These recommendations are derived from the best available evidence, being texts and papers considered authoritative by participating professional groups and the consensus of expert opinion. The supervisor training is guided by a facilitators ‘handbook’, which encourages a flexible approach and includes extra information about the slide show material (Bagnall et al. Reference Bagnall, Sloan, Platz and Murphy2011). Workshop participants received a parallel handbook, including a copy of the slide show, a list of references, and recommendations for further reading.
In another recent national project (Milne, Reference Milne2010), a detailed manual suggested how a 3-day introductory workshop for supervisors could be organized. This took the form of: Powerpoint slide shows (one to introduce each of six modules), a DVD with clips to demonstrate key supervision skills within each module, four supervision guidelines, and learning exercises (set out within the slide shows). These exercises included resource materials, such as the ‘supervision platform’ form (for specifying one's own approach); diagrams and illustrations (e.g. the ‘zone of proximal development’, ethical decision-making); written vignettes describing situations requiring considered supervision within an educational role-play; self-evaluation questionnaires, selected from the literature; and a workshop evaluation form. The four guidelines covered the supervision alliance, the learning contract, facilitating learning, and evaluation (Milne & Dunkerley, Reference Milne and Dunkerley2010). They followed the NICE procedure for building a consensus among experts (NICE, 2003), including input from supervisors, supervisees, service users, trainers and experts, all within an action research project. These guidelines recommended some evidence-based techniques and procedures, and were offered as a tool or resource to aid professional judgements. An acceptability evaluation indicated that all four were satisfactory, though suggestions were made for further improving these guidelines.
Summary and Discussion
The aim of this review was to consider how we might improve CBT supervision through written guidance. Drawing on examples from parallel literatures (therapy and training) to access seams of richer, better developed material indicated that (combined with the reviewed CBT supervision literature) such guidance carried the potential to improve supervision and so help to make supervisor support and development ‘compute’. The main finding appears to be that supervision only improves when there is a suitable innovation effort in place, including an action-research, ‘bottom-up’ ethos and supervision-of-supervision. Paradoxically, from the present sample of studies it was the smaller-scale, ‘local’ implementation projects that appeared to succeed in improving supervision, seemingly because of their attention to innovation factors. By contrast, the larger-scale (national) ‘top-down’ and more rigorous dissemination and implementation projects reviewed appeared to encounter barriers and struggled to improve supervision. Such systemic considerations are reviewed in detail in Milne & Reiser (in press), and are the subject of several papers within a future special issue of this present journal on CBT supervision. But the present review indicates that to date there have been about a dozen supervision guidelines or training manuals that could be described as sound, most having a CBT orientation and associated with generally weak evidence of successful implementation. This actually represents progress since the turn of the century, when a review across four professions indicated to the authors that ‘we have little information to guide us as to the most effective ways of training supervisors’ (Spence et al. Reference Spence, Wilson, Kavanagh, Strong and Worrall2001, p. 135). However, although several studies were impressively large implementation projects, few of these guidelines or training manuals met all eight recommendations outlined in Table 1 above (from Chorpita et al. Reference Chorpita, Daleiden and Weisz2005; Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009), being strongest with respect to the content, method of delivery and duration. The main weaknesses are the absence of data on adherence to the guidance, and on the outcomes (especially on training transfer). This status bears out Milne's (Reference Milne2009) conclusion that ‘the supervision of supervisors is the most deficient area in the whole enterprise of clinical supervision’ (p. 186) and that supervisor training remains in an embryonic condition (Watkins & Wang, Reference Watkins, Wang, Watkins and Milne2014). To benchmark progress using a major analysis of training and transfer (Stokes & Baer, Reference Stokes and Baer1977), while the supervisor development work reviewed above could safely be regarded as well beyond the naïve stage of ‘train and hope’, it could be criticized for lacking methodological sophistication, systemic implementation programming, or robust evidence of effectiveness.
Paradoxically, the 20 or so better-quality research studies (i.e. RCTs, and other controlled evaluations of supervisor training) appear to contain no formal clinical supervision manuals, whereas the more pragmatic, implementation-focused studies include around a dozen sound examples of guidance material. In place of rigour, these manuals valued relevance, based on expert consensus and action research. As a result, valuable implications can be drawn for developing and implementing written guidance most effectively: these examples offer considerable support and guidance to supervisors, trainers, and others. For example, several implementation challenges were explicitly and naturalistically addressed within the cited large-scale studies, primarily using carefully designed training, corrective feedback (from supervisors attending workshops, and from patients’ clinical outcomes), and a collaborative intervention process (e.g. action research). This stands in stark contrast to guideline implementation within healthcare more generally, a classic instance of the gulf that divides research and practice (Addis & Waltz, Reference Addis and Waltz2002). However, it is consistent with the common advice that we should construe written guidance in terms of fitness-for-purpose, or stages of development (Carroll & Nuro, Reference Carroll and Nuro2002); involve local stakeholders, balancing rigour with relevance (Barkham & Parry, Reference Barkham and Parry2008); and concentrate on dissemination research (Hayes, Reference Hayes2002).
As for the weaknesses, the above review indicates that there were significant omissions in relation to the eight recommended characteristics in reporting such complex interventions (Chorpita et al. Reference Chorpita, Daleiden and Weisz2005; Michie et al. Reference Michie, Fixsen, Grimshaw and Eccles2009), as summarized in Table 1. Only one of the located manuals appeared to satisfy all of these criteria (i.e. Henggeler & Schoenwald, Reference Henggeler and Schoenwald1998), with the help of a published research study (Schoenwald et al. Reference Schoenwald, Sheidow and Chapman2009). Other examples, impressive in other respects, were markedly incomplete in reporting the necessary details. For example, the IAPT (2011) account includes long-overdue clarification of the supervision competencies (part of specifying the content of supervisor development), which is absent from Henggeler & Schoenwald (Reference Henggeler and Schoenwald1998). But the IAPT evaluation does not appear to have included learning outcomes or adherence analyses, whereas Henggeler and colleagues added adherence checks for both supervisor and therapist (Schoenwald et al. Reference Schoenwald, Sheidow and Chapman2009). However, the IAPT and Henggeler training manuals are associated with exceptionally rigorous clinical outcome evaluations. The other reviewed manuals are associated with more striking omissions, perhaps due to local priorities (fitness-for-purpose), or to the developmental stage of the related supervisor development programme (e.g. Carroll & Nuro, 2002). In future, the reporting of supervisor training and the preparation of manuals could benefit from a Delphi study that offered guidelines on the details that should be included (Phillips et al. Reference Phillips, Lewis, McEvoy, Galipeau, Glaszion, Hammick, Moher, Tilson and Williams2014). It also appears wise to consider the lessons from past experience, drawing inspiration from across the social sciences (e.g. collaborative working between the developers and users of written guidance; Beauchamp et al. Reference Beauchamp, Drapeau and Dionne2015).
Beyond such reporting guidelines and characteristics lurks the fundamental question as to whether such written material enables replication or transfer, a functional criterion of quality. For example, manuals may vary in terms of such features as their precision, inclusiveness, detail, prescriptiveness, goals, or evaluation criteria. Although some of the reviewed manuals were used successfully in multiple settings with multiple trainers (e.g. Reid et al. Reference Reid, Rotholz, Parsons, Morris, Braswell, Green and Schell2003), it is conceivable that expert trainers were able to compensate for missing information, or that third (unassessed) variables accounted for the success (e.g. the trainers enthusiasm and clinical credibility). To address this issue, researchers could measure trainers’ adherence to manuals (Culloty et al. Reference Culloty, Milne and Sheikh2010), or seek their feedback on the adequacy of the manual as a guide to leading the workshop (Milne, Reference Milne2010).
On this critical note, it should also be acknowledged that the present review was also unsystematic, particularly in terms of the way that the available literature was located and analysed. Similarly, the review was restricted to a small sample of the material available within parallel literatures, and no doubt some valuable work will have been missed. A more systematic & inclusive review will become feasible once there emerges a larger body of written guidance material. A further criticism is the use of analogy. In order to gain a better understanding of the nature of manuals and guidelines, I included parallel literatures extensively. This strategy of reasoning-by-analogy with the therapy or training literatures carries risks, as flawed assumptions may be made and false conclusions drawn. Thouless (Reference Thouless1930) took the view that drawing an analogy was ‘not necessarily dishonest, or a crooked way of thought, although it is a dangerous one, always requiring careful examination’ (p. 140). The conventional examination is to consider the similarity of parallel literatures (e.g. study samples and settings), and Kretz & Krawczyk (Reference Kretz and Krawczyk2015) suggest that we can also judge the merit of an analogy by considering: the ‘depth’ of similarity (is the parallel superficial or extensive?); the ‘distance’ between parallels (are they from the same domain, or from further afield? Are the parallels recent or dated?); and the intended ‘purpose’ (are analogies being used simply to stimulate imagination and interest, or to suggest a causal mechanism? Judged by these criteria, the analogies drawn in the present paper appear to be relatively ‘straight’ ways of thinking: they have the comforting merit of extensive and close similarities (e.g. CBT supervision has been explicitly likened to CBT; Liese & Beck, Reference Liese, Beck and Watkins1997). Also, there has been no attempt to consider or suggest causal parallels. But it is still appropriate to recognize that training and therapy do differ importantly from supervision (e.g. training is often collegial, unlike therapy; therapy is usually optional, unlike supervision; training is usually time-limited, unlike career-long supervision, etc.). Perhaps more significantly, the participants differ in ways that may jeopardize parallels, at least with respect to therapists’ patients. In practice, this can mean that a clinical guideline carries legal ramifications absent from training guidelines, implying the need for far greater adherence. It follows that we should be wary of parallels and analogies, subjecting them to the kind of empirical scrutiny that was not attempted in the present paper.
Conclusions
We can conclude that, by drawing inspiration from the therapy, social sciences, and general healthcare literature, there is good reason to believe that the weaknesses defined above can be tackled and that written guidance material can fulfil more of its promise. In preparing promising written guidance, the ‘guidelines on guidelines’ that emerge from the present review (particularly the literature on dissemination and implementation) and from related reviews (e.g. Milne & Reiser, in press) indicate the wisdom of taking the following steps:
-
• formulate the service system, to identify the contextual barriers and boosters to implementing guidance material successfully (e.g. the contingencies affecting supervisor development in general, such as peer support);
-
• ensure that supervisor development efforts are of sufficiently high quality (e.g. that guidance material is sound, that supervision competencies are acquired through ‘gold-standard’ training, that ‘supervision-of-supervision’ is effective in fostering fidelity);
-
• work collaboratively with participating professional groups and other stakeholders (e.g. to ensure shared ownership and to jointly design support materials, ensuring adequate flexibility, within fidelity limits);
-
• adopt a collaborative, action research approach (e.g. to recognize the importance of joint problem-solving; to evaluate adherence to guidance materials, and the transfer of supervisor training).
Ethical standards
The relevant ethical standards have been complied with fully.
Acknowledgements
I am grateful to Dr Robert Reiser for his support. This work received no financial support.
Declaration of Interest
None.
Learning objectives
By the end of this review, the reader should be able to:
-
(1) Distinguish between a supervision guideline and a supervisor training manual.
-
(2) List the potential benefits and barriers to written guidance within supervision.
-
(3) Summarize how a ‘sound’ written guidance might best be developed.
-
(4) Describe a ‘sound’ example of a supervision guideline or a supervisor training manual.
Comments
No Comments have been published for this article.