Introduction
Recovery is both a key goal for individuals and a core philosophy for the organisation of mental health services worldwide. In Ireland, public mental health services are frequently complemented by the independent sector, particularly in the areas of counselling and suicide prevention. This study involving Pieta House, an Irish suicide and crisis intervention charity, is the first in Ireland to evaluate personal recovery and the recovery orientation of a service in the independent mental health sector; and the first, to the authors’ knowledge, to evaluate recovery in a suicide intervention service.
While the process of recovery is unique to each person, from analysis of individual narratives of recovery, common themes and factors have been identified. These include hope, functional remission, empowerment, support, meaningful activities, and establishment of a positive identity (Ralph, Reference Ralph2000; Mukolo et al. Reference Mukolo, Heflinger and Baxter2011; Williams et al. Reference Williams, Leamy, Bird, Harding, Larsen, Le Boutillier, Oades and Slade2012). Numerous rating scales and instruments have been developed to measure this process of recovery. Although recovery has been described not as something professionals can do to a person; but something that service users do for themselves (Anthony, Reference Anthony1993); it is recognised that professionals and organisations have a key role to play in facilitating this process (Higgins, Reference Higgins2008).
Developing recovery-oriented services requires changes in service organisation, and how responsibility, information and power are shared (Mental Health Commission, 2005). Key components of a recovery-oriented service include service user involvement in care planning and service planning; peer and advocacy networks; and above all, the instillation of hope and respect (Mental Health Commission, 2005). Internationally, the process of service delivery is recognised as crucial, and is central to mental health policy in the United Kingdom, United States, Australia and other countries (Shanks et al. Reference Shanks, Williams, Leamy, Bird, Le Boutillier and Slade2013). However, translating recovery guidelines into a working clinical model can be a challenge for services, and data on the successful implementation of these guidelines is scant. The present study, in evaluating the recovery orientation of the Pieta House service, may offer insights into these challenges.
Effective suicide intervention strategies are of paramount importance in Ireland, where the incidence of suicide was 11.1 per 100 000 in 2012 (National Suicide Research Foundation), and 11 061 presentations to emergency departments were with deliberate self-harm (DSH) in 2013 (Griffin et al. Reference Griffin, Arensman, Corcoran, Wall, Williamson and Perry2013). From the authors’ experience, emergency and psychiatric staff frequently advise patients of self-referral details for Pieta House, not least when a delay is anticipated between discharge from the ED and accessing the Community Mental Health Team or Psychological service. Data from Pieta House indicate that only 48% of their new referrals had previous contact with medical services (Surgenor, Reference Surgenor2010), indicating that DSH and suicidal ideation may be more prevalent than hospital-based studies can capture, and indicating a need for links between the independent sector and mental health services in suicide prevention.
Pieta House provides up to 15 free sessions of counselling to clients, the majority of whom are engaging in self-harm and/or have suicidal ideation. Therapy aims to address the immediate crisis, challenge suicidal thoughts by identifying and promoting reasons for living, and provide skills to manage future stressors. Clients have reported higher levels of self-esteem and reasons for living, and lower levels of depression after completion of therapy (Surgenor et al. Reference Surgenor, Freeman and O’Connor2015), with reported levels of self-esteem continuing to increase up to 6 months after completion of therapy, suggesting development of resilience (Surgenor, Reference Surgenor2015). These findings are consistent with recovery-based concepts of resilience and instillation of hope; but formal evaluation of recovery in clients of Pieta House had not been investigated before this study.
Aims of study
Our aims were to investigate:
∙ Clients’ self-perception of recovery
∙ Clients’ and therapists’ perception of Pieta House as a recovery-oriented service
∙ Whether an association existed between clients’ self-perception of recovery and their rating of the service
We hypothesised that clients’ self-rated measures of recovery would indicate improvements in self-esteem, hope and resilience following completion of therapy. We also hypothesised that Pieta House would be rated strongly in many aspects of recovery-oriented service, given its person-centred ethos; but that challenges might be noted in ensuring service user involvement in management and governance.
Selection of instruments
There are many instruments designed to assess recovery. Burgess et al. (Reference Burgess, Pirkis, Coombs and Rosen2010) reviewed and evaluated 33 instruments for suitability for use in the Australian mental health system. Their criteria included that the instrument measured domains specific to recovery, was user-friendly, and had been scientifically scrutinised. Among the eight instruments identified as suitable were the Recovery Assessment Scale (individual recovery), and the Recovery Self-Assessment Scale (recovery orientation of the service). In selecting these instruments, we noted that they could be self-administered, and in the case of the RSA, had staff and service user versions.
The Recovery Assessment Scale (RAS)
The RAS has been designed to measure aspects of recovery itself, rather than attitudes to recovery (Burgess et al. Reference Burgess, Pirkis, Coombs and Rosen2010), having been developed from factor analysis of interviews with patients in recovery from mental illness (Giffort et al. Reference Giffort, Schmook, Woody, Vollendorf and Gervain1995). The scale was later refined following independent review to yield a 41-item Likert scale. It has been shown to have adequate test-retest reliability and internal consistency (Corrigan et al. Reference Corrigan, Giffort, Rashid, Leary and Okeke1999), good concurrent validity and test-retest reliability, and can be self-completed by the consumer (Burgess et al. Reference Burgess, Pirkis, Coombs and Rosen2010). RAS total score is associated with empowerment and quality of life, and inversely associated with psychiatric symptoms (Corrigan et al. Reference Corrigan, Salzer, Ralph, Sangster and Keck2004).
The RAS has been shown to have five factors: ‘Personal Confidence and Hope’ (sample item: ‘I am hopeful about my future’); ‘Willingness to ask for Help’ (sample item: ‘I ask for help when I need it’); ‘Goal and Success Orientation’ (sample item: ‘I have goals in life that I want to reach’); ‘Reliance on Others’ (sample item: ‘I have people I can count on’); and ‘No Domination by Symptoms’ (sample item: ‘My symptoms interfere less and less with my life’) (Corrigan et al. Reference Corrigan, Salzer, Ralph, Sangster and Keck2004). These factors correlate with the domains of recovery identified by Ralph (2000), and have been demonstrated to have internal reliability, convergent and concurrent validity (McNaught et al. Reference McNaught, Caputi, Oades and Deane2007).
The RSA-R
The RSA is a 36-item Likert scale designed to measure the recovery orientation of services, which has separate versions for clients and staff (O’Connell et al. Reference O’Connell, Tondora, Croog, Evans and Davidson2005). The revised, 32-item version, RSA-R (O’Connell et al. Reference O’Connell, Tondora, Kidd, Stayner, Hawkins and Davidson2007) was used in our study, as it omits items which presume a knowledge of agency policies, which might not be accessible to clients (O’Connell et al. Reference O’Connell, Tondora, Kidd, Stayner, Hawkins and Davidson2007).
The scale has five factors. Factor One, ‘Life Goals’, describes the extent to which staff help clients with their individual life goals (sample item: ‘Staff help me develop and plan for life goals beyond managing symptoms, e.g., employment, education, physical health, connecting with family and friends’). Factor Two, ‘Involvement’, refers to the input of clients into the organisation, development and running of the service (sample item: ‘I am encouraged to help staff with the development of new groups or services’). Factor Three, ‘Treatment Diversity’, includes peer support, self-help, and non-illness based models of treatment (sample item: ‘Staff offer to help me connect with self-help, peer support, or consumer advocacy groups’). Factor Four, ‘Choice’, refers to client empowerment and includes questions on access to notes, collaboration, and the lack of coercive measures (sample item: ‘Staff listen to me and respect my decisions about my treatment’). Factor Five, ‘Individually Tailored Services’, refers to how service provision can be flexible to the individual’s needs and cultural context (sample item: ‘Staff regularly ask me about my interests and things I would like to do in the community’) (Salyers et al. Reference Salyers, Tsai and Stultz2007). Of the individual items, 22 rate staff behaviour and attitudes, eight reflect the individual, and two the service in general (Krupp, Reference Krupp2013). The scale can be scored as a composite (O’Connell et al. Reference O’Connell, Tondora, Kidd, Stayner, Hawkins and Davidson2007).
Two systematic reviews have noted that at present, there is no ideal instrument to measure either individual recovery (Shanks et al. Reference Shanks, Williams, Leamy, Bird, Le Boutillier and Slade2013), or the recovery orientation of services (Williams et al. Reference Williams, Leamy, Bird, Harding, Larsen, Le Boutillier, Oades and Slade2012). Shanks et al. (Reference Shanks, Williams, Leamy, Bird, Le Boutillier and Slade2013) noted that among instruments for personal recovery, the RAS is most published, but includes relatively more ‘empowerment’ items than other aspects of the CHIME criteria (connectedness, hope and optimism, identity, meaning and purpose, empowerment), and, along with other instruments, lacks data on sensitivity to change. Williams et al. (Reference Williams, Leamy, Bird, Harding, Larsen, Le Boutillier, Oades and Slade2012) noted that of instruments to measure recovery orientation of services, none had fully satisfactory psychometric properties, however the RSA was the most widely used measure, and the only one to have adequate internal consistency.
Methods
All clients aged over 18 who completed therapy at Pieta House, Lucan, Co Dublin, between January and March 2014 were invited to participate. Informed consent was given by 240 clients. Surveys were sent to clients following completion of therapy, either by post or online (surveymonkey.com), depending on the client’s preference. Clients under the age of 18 were excluded due to their legal status as children, which would have placed additional ethical constraints on therapists to collect informed consent. All therapists in Pieta House were invited to complete the ‘staff’ version of the Recovery Self Assessment to evaluate the recovery orientation of the service. All responses were anonymous. The study received ethical approval from the St. Patrick’s University Hospital Research Ethics Committee.
Several approaches were used to increase the response rates. First, the therapists who conducted the initial client assessments and were responsible for securing consent were contacted by the research team to ensure that they were aware of all aspects of the study and could answer any questions that potential participants may have; reminded of the need to invite all clients who matched the criteria to participate in the research; and were asked to remind consenting clients at their final therapy session to expect a questionnaire in their preferred format (online or postal). Prompting was used with reminder text messages sent to those clients who did not complete and return the questionnaire within the timeframe outlined in the cover letter. Following this, those who did not respond were telephoned by the research team, thanked for their willingness to participate, and asked if they had experienced any difficulty in receiving the questionnaire or accessing it online. Any difficulties identified at this stage were resolved by the research team.
Some terms on the RAS were modified to reflect the client group. The RAS was designed for use in mental health populations, whereas in Pieta House, the majority of clients present with suicidal ideation or DSH, and may not have a mental illness or diagnosis. Therefore, references to ‘mental illness’ were changed to ‘suicidal ideation’ or ‘distress’. While this has not previously been done, our study is the first (to the authors’ knowledge) to use recovery-based measures in this population, and accordingly we believe that this approach has face validity. For example, the statement ‘I can identify what triggers the symptoms of my mental illness’ was changed to ‘I can identify what triggers distress or suicidal thoughts’; and ‘I understand how to control the symptoms of my mental illness’ was changed to ‘I understand how to manage very distressing thoughts or suicidal thoughts’.
Results were analysed for frequency analysis using SPSS version 22. Chi-square test was used to detect any significant difference between clients’ and therapists’ responses on the RSA-R.
Results
Surveys were returned by 88 out of 240 clients (response rate 36.7%). Therapists showed a much higher response rate with 49 out of 50 surveys returned (response rate 98%).
Study population
As our data was anonymised, we do not have demographic details of our respondents, but we know that during the period of our study, 409 referrals were made to Pieta House, Lucan. Of the 409 referrals, 36.7% (n=150) were under age 18, 24.9% (n=102) were aged 18–24, 27.9% (n=114) were aged 25–44, 9.2% (n=38) were aged 45–64, and 1.2% (n=5) were aged over 65. Females comprised 60.3% of referrals. The majority of clients, 41.8%, presented with self harm (DSH) and/or suicide attempt) and ongoing suicidal ideation. Suicidal ideation alone was present in 22.9%, and DSH alone present in 7.8%.
Clients’ self-rated recovery (RAS)
The majority of clients rated themselves as agreeing or strongly agreeing with the five factors in the RAS (see Fig. 1). Sixty-five people (73.8%) rated all factors as strongly agree or agree. The highest rated factor was ‘Willingness to Ask for Help’, with 84.5% (n=75) in agreement (36% strongly agree, 48.5% agree), closely followed by ‘Reliance on Others’, with 82.1% (n=72) in agreement (36.7% strongly agree, 45.5% agree), ‘Goal and Success Orientation’ was endorsed by 73.6% (n=65), with 25.9% strongly agreeing and 47.7% agreeing; ‘No Domination by Symptoms’ was acknowledged by 66.6% (n=59), with 18.5% strongly agreeing and 48.1% agreeing; and 61.3% (n=54) agreed with the statements about ‘Personal Confidence and Hope’ (14.6% strongly agreeing, 46.7% agreeing).
Fig. 1 Clients’ Assessment of their own recovery for each of the five factors of the RAS. Bar chart shows percentage of clients strongly agreeing (black) and agreeing (grey) with each factor.
Recovery orientation of service (RSA-R)
Clients’ rating of service
Clients agreed with four of the five factors of the RSA-R (see Fig. 2). The strongest agreement was with factor four, ‘Choice’, with which 90.9% (n=80) agreed (36.4% strongly agree, 54.5% agree). There was also very strong agreement with factor one, ‘Life Goals’, 84.1% (n=74) agreed (34.1% strongly agree, 50% agree); and with factor five, ‘Individually Tailored Services’, 80.6% (n=71) agreed (29.5% strongly agree, 51.1% agree). For factor three, ‘Diversity of Treatment Options’, 65.9% (n=58) agreed (18.2% strongly agree and 47.7% agree).
Fig. 2 Clients’ Assessment of the Service according to the five factors of the RSA-R. Bar chart shows the percentages of those who ‘Agreed’=Strongly Agree+Agree (black); or ‘Did Not Agree’=Unsure+Disagree+Strongly Disagree (grey) with each factor.
The majority of clients either were unsure or disagreed with factor two, ‘Involvement’: 36.4% (n=32) agreed (12.5% strongly agree, 23.9% agree), 40.9% (n=36) were unsure, while 21.6% (n=19) disagreed.
Therapists’ rating of service
Therapists agreed with the same four factors of the RSA-R, in the same order (see Fig. 3). Again, the strongest agreement was with factor four, ‘Choice’, where 100% (n=49) agreed (36.7% strongly agree, 63.3% agree). This was closely followed by factor one, ‘Life Goals’, where 98% (n=48) agreed (24.5% strongly agree, 73.5% agree). For factor five, ‘Individually Tailored Services’, 79.6% (n=39) agreed (6.1% strongly agree, 73.5% agree), followed by factor three, ‘Diversity of Treatment Options’, where 69.3% (n=34) agreed (12.2% strongly agree, 57.1% agree).
Fig. 3 Therapists’ Assessment of the Service according to the five factors of the RSA-R. Bar chart showing the percentages of those who ‘Agreed’=Strongly Agree+Agree (black); or ‘Did Not Agree’=Unsure+Disagree+Strongly Disagree (grey) with each factor.
The majority of therapists also expressed disagreement or being unsure about factor two, ‘Involvement’, with only 30.6% (n=15) agreeing (6.1% strongly agree, 24.5% agree), while 40.8% (n=20) were unsure, and 24.5% disagreed (n=12).
There were no significant differences between therapists’ and clients’ ratings of the service on factors one, two, three or four of the RSA-R on Pearson’s χ 2 testing (p>0.05). There was a statistically significant difference between ratings of clients and those of therapists for factor five, but this did not appear clinically significant as the difference arose between the proportions of those strongly agreeing versus those agreeing with the factor.
Correlation between RAS and RSA-R
Correlation analysis of the relationship between client rating of service (RSA-R) and client rating of recovery (RAS) shows a correlation value of 0.993 (Pearson’s correlation coefficient, p=0.01).
Discussion
This is the first evaluation in Ireland of an independent charity working in the mental health sector, and the first (to the authors’ knowledge) of recovery orientation in a suicide intervention service.
Personal Recovery: Our results indicate that the majority of clients (73.8%), completing therapy at Pieta House expressed overall agreement with the Recovery Assessment Scale. The factors with which most clients agreed related to confidence in being able to access help again if needed (84.5% in agreement), and feeling supported by others (82.1% in agreement), which indicate a positive outlook for averting future risk in these clients. Increased hope and confidence were endorsed by 61.3%, which is a positive finding given that 64.7% of clients referred to Pieta House at this time were experiencing suicidal ideation.
Recovery Orientation: The ratings of clients and therapists did not significantly disagree on any factor; this concordance of view lends weight to their assessment of the service. Pieta House was rated highly by both clients and therapists on most factors of a recovery-oriented service: items relating to client empowerment (factor four), life goals (factor one) and the importance of the client as an individual (factor five) were particularly highly rated, which may reflect the existing philosophy of Pieta House. The lower rating for factor three, ‘Diversity of Treatment Options’, which includes items relating to peer support and self-help organisations, may be due to focusing primarily on short-term, individual crisis interventions. Service user involvement in the management of the service (factor two) emerged from the survey as requiring attention. Responses from both clients and therapists indicated a lack of involvement of service users in service development, management and evaluation. The challenge for Pieta House is how to address this within the structure of the organisation. Planned initiatives include the roll-out of service user committees in three sites across Ireland and the addition of service users to the Board of Trustees clinical sub-committee. These forums will provide an opportunity for greater inclusion and empowerment, and reassure clients that their experiences are central to delivering an effective service and to facilitating recovery.
The authors note that there is a limited international evidence base with which to compare these results at present; and where studies exist, the use of different recovery measures, and the differences both in services surveyed and client groupings, can limit comparison. Our results on the recovery orientation of the service (RSA) compare favourably with those in other reported studies: one study of resident and proprietor perspectives in community housing reported a need for improvement on five out of six RSA factors (Piat et al. Reference Piat, Boyer, Fleury, Lesage, O’Connell and Sabetti2015); and a study of 67 Assertive Community Treatment teams in Canada noted RSA ratings were not uniformly positive or consistent (Kidd et al. Reference Kidd, George, O’Connell, Sylvestre, Kirkpatrick, Browne, Odueyungbo and Davidson2011). Clients’ self-reported measures of recovery (RAS) compare well with those in a study of Wellness Recovery Action Plan intervention, where participants demonstrated improvement over controls in only two subscales of the RAS (Cook et al. Reference Cook, Copeland, Floyd, Jonikas, Hamilton, Razzano, Carter, Hudson, Grey and Boyd2012); and with the recent Refocus trial, which measured recovery in community-based patients with psychosis, and found no difference in recovery between control or intervention groups using the Questionnaire about the Process of Recovery (Slade et al. Reference Slade, Bird, Clarke, Le Boutillier, McCrone, Macpherson, Pesola, Wallace, Williams and Leamy2015).
The high correlation between clients’ assessment of their own recovery, and their assessment of the recovery-orientation of the service, is also notable. This could have several possible explanations. It could be due to a halo effect, or mood-congruent memory bias. It seems unlikely to be due to acquiescence bias, as evidenced by respondents’ disagreement with factor two of the RSA-R. Similarly, the agreement between clients and therapists would mitigate against this being the result of volunteer bias on the part of clients. The authors postulate that this correlation may be due to a positive interaction between the recovery orientation of the service and clients’ recovery outcomes in a causal manner. There is evidence that the expectations and belief which staff have in their clients is reflected in the clients’ outcomes (Tsai & Salyers, Reference Tsai and Salyers2010), and a study of recovery in Assertive Community Treatment teams also found an association between recovery orientation of the service and client outcomes (Kidd et al. Reference Kidd, George, O’Connell, Sylvestre, Kirkpatrick, Browne, Odueyungbo and Davidson2011). As our study was not designed to investigate a causal relationship, this may be an avenue for further research.
Limitations
Our response rate of 36.7% compares unfavourably to previously reported statistics of mean response rates of 72.1% (Sitza & Wood, Reference Sitza and Wood1998) or 65% (Nakash et al. Reference Nakash, Hutton, Jørstad-Stein, Gates and Lamb2006); but is in line with a US average of 32% return rate for hospital patient satisfaction surveys in 2011–2012 (Siegrist, Reference Siegrist2013). The research team made several efforts to maximise the response rate: the use of a multi-modal approach (choice of email or paper survey) has been reported to increase response rates (Fincham, Reference Fincham2008); prompting and reminders (text messages and phone calls), which were found to be the most effective way to improve response rate (by up to 24%) by Nakash et al. (Reference Nakash, Hutton, Jørstad-Stein, Gates and Lamb2006); and our use of a face-to-face recruitment process [found to yield a 76.7% response rate by Sitza & Wood (Reference Sitza and Wood1998)]. The length of our questionnaires (41 and 32 items) may have been a factor, being individually ‘short’ but collectively ‘long’; however length of questionnaires has been found to be relatively less significant in influencing response rate (up to 9%), and there is as yet no clear evidence for optimum questionnaire length (Nakash et al. Reference Nakash, Hutton, Jørstad-Stein, Gates and Lamb2006). The need for clients to complete and return the survey in their own time may have led to less participation by amotivated, disorganised or withdrawn clients, and hence a bias towards endorsement of recovery in the respondents. There is evidence of correlation of patient satisfaction ratings with response rates (Mazor et al. Reference Mazor, Clauser, Field, Yood and Gurwitz2002), and the possibility of a non-response bias in our study findings must be borne in mind.
The lack of data on clients’ psychiatric diagnosis (if any) or symptoms at baseline means it is not possible to determine if there is a correlation between severity of clinical symptoms and perceptions of recovery. There is some evidence for a poor correlation between clinical and personal recovery ratings (Andresen et al. Reference Andresen, Caputi and Oades2010). It is also not known what percentage of clients in the present study are simultaneously attending another mental health service [23% of Pieta House clients in 2010 also attended a mental health team (Surgenor, Reference Surgenor2010)], and whether this might have been a confounding factor in clients’ recovery. Clients under age 18 were excluded from this study, but this age group made up 36.67% of referrals, and their exclusion may limit generalisation of findings to this age group. Our choice of instruments (RAS and RSA-R) was guided by the available peer-reviewed evidence at the time of the study, however we note that there is as yet no universally agreed measurement tool. We further note that these instruments were designed for use with those in recovery from mental illness; rather than clients presenting with suicidal ideation or DSH. However, as discussed above, we believe that the substitution of ‘suicidal thoughts’ or ‘distress’ for ‘mental illness’ on the questionnaires has face validity.
Conclusions
Given the increasing role played by suicide intervention charities and independent counselling services in Ireland, and the significant crossover of service users between the independent sector and public mental health services, there is a need to establish the efficacy of such organisations both in empowering clients personal recovery, and the services’ recovery orientation. The current study provides initial evidence for the role played by Pieta House. The results endorse personal recovery in service users, particularly in relation to confidence in future help-seeking behaviour, and increased feelings of hope. Both clients and therapists rated most aspects of recovery orientation of the service positively, and identified a deficit of client input. By addressing this, the recovery orientation of the service can be strengthened, which may further empower clients and promote recovery.
Acknowledgements
The authors wish to acknowledge and thank those who participated in this study.
Conflicts of Interest
Four of the authors were employed by Pieta House during the study, but had no involvement in the therapy received by clients. Responses of clients and therapists were anonymised to remove potential conflicts of interest. Pieta House staff were not involved in data analysis.
Funding
This study received no grant from any funding agency, commercial, or not-for-profit sector.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation with the Helsinki Declaration of 1975, as revised in 2008. The study protocol was approved by the REC at St. Patrick’s University Hospital. Written informed consent was obtained from all participants.
