Introduction
Over the last 10 years Italian forensic psychiatry has undergone a number of rapid and profound changes, similar to that which occurred 3 decades ago with the closure of psychiatric hospitals (Basaglia, Reference Basaglia1980). According to the Decree of the President of the Council of Ministers (DPCM 1.4.Reference Basaglia2008), health provision in Italian prisons has been devolved from the Prison Administration (Ministry of Justice) to the National Health Service (Ministry of Health). Indeed, a number of settings are involved by the reform process: prisons, forensic psychiatric hospitals (OPG, ‘Ospedali Psichiatrici Giudiziari’) and juvenile correctional facilities. However, due to a number of practical and financial constraints, changes to date have been very slow, while the enforcement of the new legislation continues to be postponed. The Italian Government has set at April 2015 the closing of Italian forensic mental hospitals (Legg 30 maggio 2014, n. 81). It is proposed that offenders sentenced to detention in psychiatric care will be detained in small community residential facilities appointed by the National Health Service. Current forensic hospitals will be replaced by regional forensic units in network with public health services, social services, correctional facilities, courts, public safety bodies aimed to implement individualised caring programs to share with Mental Health Departments. This represents the final step of a legislative pathway started in 2008 and defined with Law Legg 17 febbraio 2012, n. 9, which prescribes the progressive discharge of the inpatient population across the six Italian Forensic Hospitals and their referral to the public community mental health services. One of the most significant limitations with regard to service planning and provision for this group of forensic psychiatric patients is that no recent and reliable information on their characteristics and needs for care is available. It has been argued that this is mainly attributable to the secondary interest on these issues in Italian psychiatric research (Fioritti et al. Reference Fioritti, Ferriani, Rucci, Melega, Venco, Scaramelli and Santarini2001; Cerboneschi et al. Reference Cerboneschi, Bensi, Brandizzi, Dario, Fagioli and Fiori Nastro2009), together with the lack of reliable assessment procedures.
Forensic psychiatric patients have a greater number of unmet needs compared with the general psychiatric population (Harty et al. Reference Harty, Tighe, Leese and Thornicroft2003), with specific recommendations having been made regarding the need for a greater focus in the areas of substance abuse, violent behaviour and co-morbidity with personality disorders with this population (Shaw, Reference Shaw2002). Being able to assess these needs in a standardised way is imperative, both for the delivery of effective treatment interventions and for the development of tailored aftercare packages. However, the methods of assessing the individual needs of this group have, to some extent, been neglected, with the focus instead being much more on security and risk issues. A valid, reliable, standardised assessment tool specifically designed for assessing the health and social needs of forensic patients is the Camberwell Assessment of Need – Forensic version (CANFOR) (Thomas et al. Reference Thomas, Slade, McCrone, Harty, Parrott, Thornicroft and Leese2008). The CANFOR has been found to be a useful tool in helping to identify individually tailored care pathways in the different functional security levels and settings, as well as assisting with evidence-based interventions (Long et al. Reference Long, Webster, Waine, Motala and Hollin2008; Glorney et al. Reference Glorney, Perkins, Adshead, McGauley, Murray, Noak and Sichau2010).
The present study was undertaken to explore convergent validity, inter-rater and test–retest reliability of the Italian version of the CANFOR staff version, a standardised need assessment tool to assist Italian mental health professionals in both planning and providing mental health care to the forensic population in the critical transition process from forensic psychiatric hospitals to forthcoming residential community facilities.
Methods
Participants
The study was carried out within the high security forensic psychiatric hospital (OPG) of Castiglione delle Stiviere (Lombardy Region, North Italy) between February and June 2011. At the time of the sampling, 262 patients were present in the Forensic Hospital, 175 (66.8%) males and 87 (33.2%) females. The average age of the patient population was 42.1 (s.d. 11.9), with males being slightly older than females [42.3 (s.d. 11.5) v. 41.4 (s.d. 12.7), t = 0.576, p = 0.565].
Exclusion criteria for participation included mental retardation (n = 25), drug abuse as the main diagnosis (n = 7) and other psychiatric disorders (neurotic disorders, n = 1).
The participants were assessed by the Italian translation of the CANFOR needs assessment (Thomas et al. Reference Thomas, Harty, Parrot, McCrone, Slade and Thornicroft2003, Reference Thomas, Slade, McCrone, Harty, Parrott, Thornicroft and Leese2008), together with the Italian versions of the Global Assessment of Functioning (GAF; American Psychiatric Association, 1994) and the Brief Psychiatric Rating Scale (BPRS; Ventura et al. Reference Ventura, Lukoff, Nuechterlein, Liberman, Green and Shaner1993; Roncone et al. Reference Roncone, Ventura, Impallomeni, Falloon, Morosini, Chiaravalle and Casacchia1999). Written consent was provided by all participants. Since assessments were conducted within the framework of routine care by patients' treating key-clinicians, no specific ethics approval was requested.
The instrument
Developed from the Camberwell Assessment of Need (CAN) (Phelan et al. Reference Phelan, Slade, Thornicroft, Dunn, Holloway, Wykes, Strathdee, Loftus, McCrone and Hayward1995), the CANFOR has maintained the same format as the CAN, with three new areas of evaluation (treatment, arson and sexual offending) added to the original 22 areas. In this study, we used the full clinical version of the instrument (CANFOR-C), staff-version. The interviewer first evaluates whether a need is present or not, and, if present, whether that need is met or unmet. A need is defined as being present when the interviewer indicates that there have been difficulties in a particular area over the last month. A met need is defined as an area of difficulty for which an appropriate intervention is currently being received from either formal or informal sources. An unmet need is defined as an area of difficulty for which no interventions are currently being provided by local services, or that interventions provided are not perceived as effective. If a need is not considered to be present, it can be scored as no need or, in certain instances, not applicable. The total need score is defined as the sum of the number of met needs and unmet needs.
Translation and back translation
Two of the authors (L.C. and A.L.) first translated the original CANFOR-C from the English; a back translation was then performed by an Italian-English professional translator (C.A.) and checked for consistency by the first author of the original version (S.T.). Finally, a focus group composed by eight mental health professionals working in the high security psychiatric hospital of Castiglione in Italy was asked to discuss the feasibility of using the translated assessment tool in the clinical setting. On the basis of the focus group discussions, only minor changes were introduced. Specifically, the wording of two needs domains were slightly modified: ‘Information on disease, treatment and rights’ was changed to ‘Information on disease, treatment and legal position’, and ‘Drugs’ was renamed ‘Substances of abuse’. Moreover, the term ‘local services’ (used in the items assessing the amount of help being provided) was renamed ‘local forensic/general services’.
Convergent validity
In order to study the convergent validity, the GAF and the BPRS were administered to the same sample of patients. We used the GAF and the BPRS scores, as previous research conducted with both the CAN (see e.g., Issakidis & Teesson, Reference Issakidis and Teesson1999; Lasalvia et al. Reference Lasalvia, Ruggeri, Mazzi and Dall'Agnola2000) and the CANFOR (Thomas et al. Reference Thomas, Harty, Parrot, McCrone, Slade and Thornicroft2003; Segal et al. Reference Segal, Daffern, Thomas and Ferguson2010) found that the number of clinician-rated needs are highly correlated with measures of functional and clinical severity.
Inter-rater and test–retest reliability
Two raters (a psychiatrist, L.C. and a clinical psychologist, E.M.) were used to test inter-rater reliability, with data being collected using one active and one silent rater during the interview. Test–retest reliability was investigated by re-interviewing all participants three weeks after the first interview, led and rated by the same researcher.
Statistical analyses
The sample was randomly selected from a potential sample of 262 patients who were hospitalised in the institution on 30 December 2010 after having stratified by age (18–35 years, 36–50 years, >50 years), marital status (single, married, separated/divorced/widowed) and diagnostic group (non-affective psychosis, personality disorder, affective psychosis). In a test of agreement between two raters using the kappa statistic, a sample size of 50 subjects achieved at least 80% power to detect a true kappa value of 0.90 in a test where the null hypothesis was 0.40 with a significance level of 0.05, regardless of the value of the two categories' frequencies. The power study was performed by PASS 11 (Hintze, Reference Hintze2011; www.ncss.com).
Percentages or means were presented for sociodemographic, offending and clinical characteristics in the whole sample. Each CANFOR item was recoded into two domains: presence of a need (whether met or unmet) and rating of an unmet need. Consistent with Thomas et al. (Reference Thomas, Slade, McCrone, Harty, Parrott, Thornicroft and Leese2008), not applicable responses were considered ‘no need’ for the purposes of analyses. Total needs were calculated as the number of items with the presence of a need (potential range 0–25) and total unmet needs as the number of items scored as an unmet need (potential range 0–25).
The association between total needs and total unmet needs, GAF and BPRS (total and dimensions) was assessed using Kendall's tau-b rank correlation coefficient.
Intra-class correlation coefficients (ICCs) were calculated, using a two-way mixed model defining absolute agreement, to assess both inter-rater and test–retest reliability for total needs and total unmet needs, as recommended by Leese et al. (Reference Leese, White, Schene, Koeter, Ruggeri and Gaite2001).
Inter-rater and test–retest reliability for each CANFOR item (both presence of a need and rating and unmet need) was measured by Cohen's kappa coefficient, which compensates and corrects for the proportion of agreement that might occur by chance. The agreement indicated by the kappa coefficient can be poor (<0.21), fair (0.21–0.40), moderate (0.41–0.60), good (0.61–0.80) and very good (0.81–1.0) (Landis & Koch, Reference Landis and Koch1977). Usually a value of kappa > 0.70 is considered to be adequate agreement. Due to the fact that kappa is affected by the presence of bias between observers or times, we applied a test of symmetry of the off-diagonal cells.
Analyses were carried out using Stata 11.1 for Windows.
Results
The characteristics of the sample are shown in Table 1.
Table 1. Participants' sociodemographic, offending and clinical characteristics (N = 50)
*12 homicide, 2 sexual offence, 13 personal injury, 3 armed robbery, 2 kidnapping, 2 arson.
†6 theft, 4 damage, 6 violation of probation.
Level of needs
Staff rated patients as having an average of 14.2 total needs (s.d. 2.3), with a range between 8 and 20. The average number of unmet needs was 5.6 (s.d. 2.4) with a range of 0–10.
The mean GAF score was 40.12 (s.d. 9.88) and BPRS mean scores were 2.01 (s.d. 0.56) for total, 2.42 (s.d. 0.53) for anxiety/depressive symptoms, 1.63 (s.d. 0.76) for negative symptoms, 2.33 (s.d. 1.07) for positive symptoms, 1.59 (s.d. 0.71) for manic symptoms, and 1.73 (s.d. 0.91) for cognitive symptoms (Table 1).
Convergent validity
As shown in Table 2, a higher number of needs was significantly associated with more severe psychiatric symptoms in the BPRS total mean score and in the dimensions anxiety/depression, manic and cognitive. A higher number of unmet needs was significantly associated with more severe psychiatric symptoms in the BPRS total mean score and in the dimensions positive, manic, anxiety/depression and cognitive. Higher numbers of unmet needs were also significantly associated with lower levels of global functioning (GAF).
Table 2. Association [Kendall's tau-b rank correlation coefficient (95% CI), p-value] between CANFOR scores, GAF and BPRS scores (total and dimensions) (N = 50)
Inter-rater reliability
Intraclass correlation coefficients for total needs score (ICC = 0.74) and unmet needs score (ICC = 0.71) indicated a good level of agreement between raters for both dimensions.
Table 3 shows number and percentage of patients rated by the two assessors as presenting a need (regardless of whether it was met or unmet) and an unmet need for each of the 25 CANFOR items. According to both raters, food, information, daytime activities, psychotic symptoms, company, intimate relationships, sexual expression, psychological distress, money and treatment were the most frequently assessed needs (met or unmet), while social aspects (sexual expression, intimate relationships, company), psychological distress, daytime activities, accommodation and information were the most frequently rated unmet needs.
Table 3. Inter-rater reliability of each CANFOR domain based on interviews with staff. Frequency of patients with needs, as reported by raters (N = 50)
NA: not applicable test because all patients were classified in the same category by raters.
*Kappa affected by the presence of bias between raters (by checking the symmetry of the off diagonal cells).
Cohen's kappa was calculated for each of the 25 CANFOR items in relation to overall agreement about the presence of a need. Agreement in the individual need domains was very good for 11 (44%) domains, good for 5 (20%) domains, moderate for 2 (8%) domains, fair for 3 (12%) domains and poor for 1 (4%) domain, namely information.
Cohen's kappa calculated for ratings of unmet needs showed that agreement was very good for 3 (12%), good for 6 (24%), moderate for 5 (20%), fair for 2 (8%) and poor for 4 (16%) domains, namely information, alcohol, drugs and arson.
Test–retest reliability
Intraclass correlation coefficients for total needs score (ICC = 0.97) and unmet needs score (ICC = 0.92) indicated a very good level of agreement between time 0 and time 1 for both dimensions.
Cohen's kappa calculated for each of the 25 CANFOR items showed a very good agreement in all the domains, with the only exception of safety to self, which showed a good level of agreement (see Table 4).
Table 4. Test–retest reliability of each CANFOR domain based on interviews with staff. Frequency of patients with needs, as assessed at time 0 and time 1 (N = 50)
NA, not applicable test because all patients were classified in the same category at times 0 and 1.
*Kappa affected by the presence of bias between time points (by checking the symmetry of the off diagonal cells).
Cohen's kappa calculated for ratings of unmet needs showed a very good agreement for 12 out of 25 domains (48%), a good agreement for 5 (20%) domains, a moderate agreement for safety to others (4%) and a poor agreement for drugs (4%).
Discussion
Main findings
The Italian version of CANFOR-C, staff-version, has demonstrated adequate psychometric properties. It's convergent validity has been supported by the significant results comparing total and unmet needs scores on the CANFOR-C with BPRS and GAF scores. Unmet needs were significantly associated with BPRS total, positive and manic dimensions, which reflected the main diagnostic features of the participant population. The weaker association found here with respect to the cognitive dimension may be due of sampling procedures that excluded the more severe forms of mental impairment from this study. It should be noted that this study did not address other psychometric properties of the CANFOR, such as intra-rater reliability and sensitivity to change.
Findings against the background literature
Results of the reliability tests are consistent with previous validation studies conducted internationally in the UK (Thomas et al. Reference Thomas, Slade, McCrone, Harty, Parrott, Thornicroft and Leese2008), Spain (Romeva et al. Reference Romeva, Rubio, Güerre, Miravet, Cáceres and Thomas2010) and Portugal (Talina et al. Reference Talina, Thomas, Cardoso, Aguiar, Caldas de Almeida and Xavier2013). The slightly lower agreement found in the inter-rater reliability test compared with other published studies may be due to differences in the professional backgrounds and conceptual frames of reference of the two raters, as similarly noted by Thomas et al. (Reference Thomas, Slade, McCrone, Harty, Parrott, Thornicroft and Leese2008).
We found a lower level of agreement in the more socially orientated need domains of company, intimate relationships, psychological distress and information. It is possible that this finding may reflect the long periods of institutionalisation experienced by many patients in the service, which leads to more challenges with the recognition, expression and interpretation of these needs. Furthermore, 46% of the sample were diagnosed with a personality disorder, which may additionally account for uncertain expression of psychological needs and undesired and/or misunderstood information about their condition and treatment. With respect to the two need domains associated with moderate level of agreement, the different professional backgrounds of the two raters may be a partial explanation with respect to the physical health domain; the high clinical and criminological heterogeneity of the participants may also have contributed to the perceived need for different types and intensities of daytime activities required.
The test–retest reliability studies suggested high levels of agreement with ratings completed 3 weeks apart; these findings are again consistent with previous studies using the CAN tools (McCrone et al. Reference McCrone, Leese, Thornicroft, Schene, Knudsen, Vázquez-Barquero, Lasalvia, Padfield, White and Griffiths2000; Arvidsson, Reference Arvidsson2003). Future research should seek to explore issues around sensitivity to change of the CANFOR tools, especially with clinical populations who have different length of stays and access to different service pathways.
Conclusions
The Italian version of the CANFOR was found to be a valid and reliable instrument to be used for needs assessment across different forensic settings, from correctional institutes to current Ospedali Psichiatrici Giudiziari as well as in the forthcoming new regional forensic units. Mental health services are taking active role in the development of individualised care programs for forensic patients towards discharge as the transition process of reform of forensic psychiatric sector is now progressing across Italy. The introduction of specific assessment tools for this specific population will undoubtedly be an important step towards promoting an evidence-based approach to the development of individually tailored care pathways and service infrastructure. It will be especially important to capture and take into account the diversity and potentially changing health and social needs of this population as they transit into new services and, additionally, to help facilitate active patient involvement in identifying and prioritising care planning and support.
Acknowledgements
The authors acknowledge the assistance and help of Simonetta Desiato in the development of the Italian scoring versions of CANFOR, Claudia Albertalli for her supervision in back-translation, Antonino Calogero, Gianfranco Rivellini and Ilaria Lega for their helpful comments.
Financial Support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Statement of Interest
All the authors declare that they have no competing interests.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
