Introduction
Clinical audit is defined as ‘a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change’ (NICE, 2002). It is an essential component of good professional practice in all areas of medicine, including psychiatry (Jamtvedt et al. Reference Jamtvedt, Young, Kristofferson, O’Brien and Oxman2007; Ndoro, Reference Ndoro2014), and is a requirement for ensuring professional competence (Chief Medical Officer, 2006; CPsychI, 2011; Medical Council, 2011; RCPsych, 2012; Swanwick, Reference Swanwick2012). Although not everyone agrees with the emphasis placed on clinical audit in training (Jackson, Reference Jackson2012), the Royal College of Psychiatrists cites participation in audit as particularly important for trainees to understand how clinical governance is linked to leadership, good practice and quality improvement (RCPsych, 2009).
Clinical audit has been cited among the best means of promoting effective continuing professional development among postgraduate trainees (Filipe et al. Reference Filipe, Silva, Stulting and Golnik2014). However doctors have identified significant barriers to practicing clinical audit (Firth-Cozens & Storer, Reference Firth-Cozens and Storer1992; Chambers et al. Reference Chambers, Bowyer and Campbell1996; Ertl-Wagner & Steinbrucker, Reference Ertl-Wagner and Steinbrucker2011; Daly et al. Reference Daly, Brennan, Dowling and Plant2012; Daveson et al. Reference Daveson, Simon and Benalia2012; Perrem & O’Neill, Reference Perrem and O’Neill2012). These include: the perception that clinical audit is a pointless, managerially driven exercise; lack of support and supervision from senior doctors, researchers and managers; pressure of workload and lack of protected time; underdeveloped organizational links; lack of funding, resources, training, knowledge and skills; and worry that mistakes will be exposed publicly leading to embarrassment or censure.
When postgraduate trainees in Ireland reach consultant level, they are legally and ethically required to participate in an annual audit of their practice in order to remain on the Specialist Register of the Medical Council. Advice to enhance the practice of clinical audit has been offered by organizations such as the Healthcare Quality Improvement Partnership in the United Kingdom and the World Health Organization (Dixon, Reference Dixon2010; Flottorp et al. Reference Flottorp, Jamtvedt, Gibbs and McKee2010; Burgess, Reference Burgess2011). In the absence of a similar guidance structure in Ireland, doctors are left with little option but to follow the NICE guidelines and those of the UK Care Quality Commission. In our study, we aimed to examine attitudes to clinical audit among postgraduate basic specialist psychiatric trainees in Ireland. We also sought to examine whether demographic factors, training and direct experience of clinical audit influences those attitudes.
Methodology
Instrument
We devised a questionnaire to record the attitudes of postgraduate basic specialist psychiatric trainees to clinical audit, in addition to gender, age, nationality, and details of training and direct experience of clinical audit. Specifically, we adapted and incorporated the 16-item General Practitioner’s Attitudes to Medical Audit questionnaire used in a Staffordshire GP study (Chambers et al. Reference Chambers, Bowyer and Campbell1996). Respondents indicated on a five-point Likert scale how strongly they agreed with each of the 16 attitude statements. A previous study determined that seven of the statements were in favour of audit, seven were against audit, and two were neutral (Chambers et al. Reference Chambers, Bowyer and Campbell1996). Respondents were blind to whether the statements were positive, negative, or neutral.
Respondents scored four points for strongly agreeing with a positive statement or strongly disagreeing with a negative statement, reducing by one point on the Likert scale through to zero points for strongly agreeing with a negative statement or strongly disagreeing with a positive statement. We then added the points of the 14 positive or negative statements to reach a positivity score of between zero and 56. Neutral statements were not counted. Questions that related to demographics, training, and experience were mostly binary.
Sample
We distributed the survey to 81 basic specialist psychiatric trainees in Leinster. We included a letter of information for research participants and a stamped, addressed envelope for survey return. Unique numbers printed on the surveys enabled anonymity while allowing follow-up of unreturned surveys. Trainees who did not respond within 45 days were posted a second copy of the survey. Those who did not respond within a further 45 days received an email reminder. Non-respondents were compared with respondents in terms of gender and whether or not they were working in general adult psychiatry or a psychiatric subspecialty (a proxy for experience).
Statistics and analysis
We ranked empirically the proportions of trainees who agreed or strongly agreed with statements on attitudes to clinical audit, regardless of whether these statements were positive, negative or neutral. We analysed respondents’ agreement with the 16 attitude statements and their positivity scores with reference to respondent age, years of training in psychiatry, years of training in any other specialty and total years of training by calculating Spearman’s rank correlation coefficients (ρ). We analysed the attitude statement responses and the positivity scores with reference to gender, audit training and direct experience of clinical audit using the Mann–Whitney U-test. Finally, we analysed the attitude statement responses and the positivity scores with reference to nationality using the Kruskal–Wallis test. We applied the Bonferroni correction. We used SPSS version 15.0.
Results
General descriptive results
Of the 81 surveys sent out [to 35 (43.2%) males and 46 (56.8%) females], we received 49 responses (response rate 60.5%). One respondent did not give their age, but the mean age of the remaining 48 respondents was 30.73 years (range 25–42). A total of 19 (38.8%) respondents were male; 33 (67.3%) respondents were Irish, while seven (14.3%) came from other European Union (EU) countries and nine (18.4%) came from non-EU countries. Respondents had spent an average of 1.52 years (range 0–3.5 years) in psychiatry, and 1.51 years (range 0–10 years) in other medical specialties. Respondents had spent a total average of 3.03 years (range 0–11.5 years) training in any medical specialty. In general adult psychiatry, there were 30 respondents (40.0% male) and 22 non-respondents (54.5% male). In subspecialty psychiatry, there were 19 respondents (36.8% male) and 10 non-respondents (10% male).
The clinical audit training and experience of respondents are shown in Table 1. Those who had received little or no formal audit training identified ‘teaching as part of my medical degree’, ‘occasional tutorials and lectures’, ‘guidance from senior registrars or consultants’, and ‘my own self-directed learning’. Barriers to closing the audit loop/spiral cited included ‘time constraints’, ‘changed placement/post’, ‘demands of other services’, ‘geographical factors’, and the perception that ‘change was brought about by the initial audit’.
Table 1 Respondents’ clinical audit training and experience
U/A, unanswered (%).
Table 2 shows the overall levels of agreement with the 16 attitude statements in the survey, ranked in descending order of combined ‘strongly agree’ and ‘agree’ scores. The three highest-ranking statements were either positive or neutral. The median positivity score was 30 out of 56 (range 12–40).
Table 2 Attitude statements ranked by combined ‘agree’ and ‘strongly agree’ score
SA, strongly agree; A, agree; U, undecided; D, disagree; SD, strongly disagree; (+), positive statement; (−), negative statement; (0), neutral statement.
Correlations: demographics, training, experience, and attitudes
Using Spearman’s rank correlation coefficients (ρ) to examine if the 16 attitude statements and their positivity scores were predicted by respondents’ age and clinical training, we found only two significant correlations (p=0.01, 0.018). Both of these related to trainees’ number of years’ training. Age did not predict attitude or positivity score. With 68 attempts at correlation, we would have expected three to have occurred by chance with significance set at p<0.05. Our two positive results therefore are unlikely to be significant overall. Moreover, the Bonferroni correction reduces the acceptable p-value to 0.0007, rendering our results insignificant.
Our Mann–Whitney U-test of respondents’ agreement with the 16 attitude statements and their positivity scores in relation to gender and formal training in clinical audit yielded no significant results (68 tests). When we applied the Mann–Whitney U-test to respondents’ agreement with the 16 statements and their positivity scores in relation to their direct experience of clinical audit, there were four significant correlations overall (p=0.043, 0.041, 0.012, 0.007). However, with 102 tests related to direct experience of clinical audit, we would have expected five positive results to occur by chance with significance set at p<0.05. Moreover, the Bonferroni correction reduces the p-value to 0.0005, rendering these results insignificant. Finally, in our Kruskal–Wallis test of respondents’ agreement with the 16 statements and their positivity scores in relation to their nationality (Irish, EU, and non-EU), none of the 17 tests were significant.
Discussion
Our participating trainees were barely more than ‘undecided’ overall when it came to positive attitudes to clinical audit. This is still slightly better than was the case in a Staffordshire GP survey, in which general practitioners tended to display more negative attitudes (Chambers et al. Reference Chambers, Bowyer and Campbell1996). Perhaps the zeitgeist has changed over the course of two decades such that audit is now considered a slightly more important component of modern clinical practice. However, given that our response rate was only 60.5%, it is also possible that those trainees who took the trouble to complete our survey may have had a better attitude to audit than non-respondents.
It is difficult to say what predicts attitudes. Of 255 tests, we would have expected 13 positive results to have arisen purely by chance with significance set at p<0.05. We had only six and it is therefore difficult to attach any overall significance to our findings, especially when we take Bonferroni’s corrections into account.
Age and nationality did not predict attitudes in our study, which is in line with any existing research. In terms of gender, it is noteworthy that 38.8% of our respondents were male. As 43.2% of those to whom the questionnaire was sent were male, there was a slightly lower representation of male trainees among the responses received. This is in line with previous findings that females generally respond better to surveys (Sax et al. Reference Sax, Gilmartin and Bryant2003). Gender did not predict attitudes however, which is at odds with the findings of a study in which that male students were more likely to believe that their research and audit experience should influence their selection into training programmes (Nikkar-Esfahani, Reference Nikkar-Esfahani, Jamjoom and Fitzgerald2012).
We found that years of training in psychiatry did not predict attitudes. This is in line with previous findings (Ertl-Wagner & Steinbrucker, Reference Ertl-Wagner and Steinbrucker2011). Years of training in other medical specialties did not predict attitudes either, nor did the overall time spent training in any specialty. It is noteworthy that there was a higher response rate among sub-specialist trainees, who tend to be more clinically experienced. Our findings are slightly at odds with those of the Staffordshire GP Study, which found that more experienced doctors may have slightly worse attitudes (Chambers et al. Reference Chambers, Bowyer and Campbell1996). It is, of course, conceivable that younger and less-experienced doctors have fewer strong opinions about audit.
Of particular note, training in clinical audit did not predict attitudes. This may simply be because so little training had been received by respondents. This paucity of training received is in line with previous research, in which only 25% of psychiatric trainees got any formal teaching and 78% felt this was inadequate (Joiner et al. Reference Joiner, Hussain, Duddu and Chaudhry2015). Our findings suggest much higher audit participation rates than were the case in many previous studies (Maisonneuve et al. Reference Maisonneuve, Touboul, Bonnelye and Bertrand2008; Daly et al. Reference Daly, Brennan, Dowling and Plant2012; Perrem & O’Neill, Reference Perrem and O’Neill2012). Overall, direct experience of clinical audit did not predict attitudes, which partially correlates with previous research (Chambers et al. Reference Chambers, Bowyer and Campbell1996).
Limitations
Trainees who took the trouble to complete the survey may have had a better attitude to audit than non-respondents. This would have positively influenced the overall results, making attitudes to clinical audit among psychiatric trainees appear better than they actually are.
Our failure to find factors significantly predicting attitudes to clinical audit may be due to an underpowered or excessively narrow sample rather than a real negative finding.
Conclusions
Although basic specialist trainees in psychiatry have moderately positive attitudes to clinical audit (and better than those of UK general practitioners in the mid-1990s), there is still much work for trainers to do in improving these attitudes so that clinical audit continues to develop as an essential component of good medical practice. Given our failure to demonstrate that training and direct experience of clinical audit significantly improve attitudes, it is difficult to determine from these results exactly what needs to be targeted among trainees in order to improve attitudes. Ours is an initial study of this area of training limited by sample size and the narrowness of the group tested. Further study of other specialities, higher trainees, and consultant trainers would further enhance our understanding.
Acknowledgement
The authors would like to acknowledge the trainees who participated in the study.
Financial Support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation with the Helsinki Declaration of 1975, as revised in 2008. The study protocol was approved by the institutional review board of each participating institution including the Saint John of God Hospitaller Ministries Ethics Committee. Written informed consent was obtained from all participating trainees.
