Introduction
Evidence from literature indicates that the current usual treatment practices fall short of what would be recommended based on the best evidence on treatment efficacy for schizophrenia. This underscores the need for greater efforts to ensure that the treatment research results are translated into practice (Lehman et al. Reference Lehman and Steinwachs1998).
The past decade has witnessed an increasing interest in quality of care measures in mental health, with a focus on the dimensions of accessibility, continuity, appropriateness, efficacy and safety in schizophrenia and in general in mental health disorders. However, there is lack of consensus between clinician and stakeholders as to which measurement should be used and how conformance measurement results should be applied and acted upon (Weinmann et al. Reference Weinmann, Roick, Martin, Willich and Becker2010).
In Italy and in other European countries, the process of deinstitutionalisation and the implementation of a community-based model have not been monitored and evaluated through systematic data collection (Munizza et al. Reference Munizza, Gonella, Pinciaroli, Rucci, Picci and Tibaldi2011).
In Italy, although a national information system for mental health has been enacted with a Ministry Decree in 15 October 2010, to date the development and calculation of mental health indicators have been carried out mainly at the regional level, except for the SIEP Direct's Project (Semisa et al. Reference Semisa, Casacchia, Di Munzio, Neri, Buscaglia, Burti, Pucci, Corlito, Bacigalupi, Parravani, Roncone, Cristofalo, Lora and Ruggeri2008) and the inquiry ‘Comparative analysis of the efficiency, quality and appropriateness of the Italian local health trusts’ promoted by the Parliamentary Commission of Inquiry of the Senate on the effectiveness and efficiency of the national health service (Nuti et al. Reference Nuti, Fantini and Murante2014). The SIEP Direct's Project emphasised a marked variability among Departments of Mental Health (DMHs) in the pharmacological treatment of first-episode psychosis and a low frequency of psychotherapeutic, psychosocial and rehabilitative approaches in people with schizophrenia and the final report of Commission of inquiry underlined large intra- and inter-regional differences in hospitalisation rates of individuals with schizophrenia and psychotic disorders.
This suggests that more research efforts should be addressed at benchmarking the performance of mental health services located in different regions, to identify discrepancies in practice patterns, health outcomes and regional usages of resources that cannot be justified by differences in patient needs.
In a previous paper, our research group (Bollini et al. Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ) examined the structural and organisational determinants of quality of care in patients with schizophrenia in two mental health departments in Italy, with a focus on dosage of antipsychotic drugs. We found that higher doses than recommended could be accounted for by the high patient caseload per psychiatrist, which leads to limited contacts with patients and their families and to an overreliance upon drug treatment.
Building upon this previous work (Bollini et al. Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ), we hypothesised that the department's culture, work climate, motivation, structural characteristics and resources would favour the implementation of a measurement-based quality improvement (MBQI) process with a plan-do-study-act cycle. Specifically, our assumption was that the presence of a multi-professional staff, a good working climate, the staff participation in training programmes, well-defined responsibilities and adequate financial resources is the prerequisite to improve the quality of delivered care. Moreover, we assumed that the MBQI can be successful only when health care professionals are willing to be measured, to select meaningful measures and to be proactive in promoting changes (Weinmann et al. Reference Weinmann, Koesters and Becker2007; Chou et al. Reference Chou, Vaughn, McCoy and Doebbeling2011).
The MBQI process was implemented in two steps. First, recommendations were extracted from international guidelines on schizophrenia, revised and operationalised in 15 indicators (nine concerning drug treatment and six psychosocial treatment) that were calculated for 807 patients with schizophrenia treated in some DMHs of the Piedmont Region, Italy (Bollini et al. Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ). A characteristic element of this experience was the application of eligibility criteria to treatment (the criteria that the patient must have, to be included in the recommendation), compliance (the criteria that must be present to say that the recommendation has been satisfied) and the inclusion of moderators’ factors (factors that may explain the lack of application of a given recommendation). These include, for instance, the presence of severe physical comorbidity, or side effects that would contraindicate the use of recommended drug treatment or explicit refusal to take the drug.
The Evaluation of Treatment Appropriateness in Schizophrenia (ETAS) study was then developed in continuity with this latter project to promote the quality of care in patients with psychotic spectrum disorders (ICD10 F20-F29) in Italy through the involvement of mental health staff in the critical appraisal of existing guidelines and the evaluation of care appropriateness in routine mental health care. This was done through the selection of a number of additional consensus indicators identified starting from guidelines recommendations, the calculation of the indicators using data from a sample of patients with psychotic spectrum disorders attending three Italian DMHs, a review of the results with the operators and the identification of possible measures to address the criticalities emerged.
The primary aim of this paper is to analyse the conformance of usual care patterns for persons with schizophrenia to treatment guidelines. The secondary aim is to examine the organisational and structural reasons underlying the observed variations in care provided to patients with schizophrenia in order to assess their potential role as determinants of the observed appropriateness.
Methods
Setting
The study was carried out at three DMHs located in different areas of Italy. One Department is located in North-Eastern Italy (NE), in an autonomous province with a catchment area of 513 357 inhabitants that comprises a town and the surrounding mountain area. The province is one of the wealthiest in Italy and is a renowned destination for summer and winter tourism in the Dolomites. However, the geographical characteristics of the area cause some problems of accessibility to health care services. Another (NW) is located in the suburban area of a metropolitan town of North-Western Italy, with a catchment area of 202 400 inhabitants and the third (SO) is located in a small town of Southern Italy and has a catchment area of 115 232 inhabitants (20% of the population of the region). The three DMHs can be considered the representative of their geographical area and were chosen because they provided the full involvement of the DMHs staff and management that was required for this study.
Study design
This is an observational study based on primary and secondary data collected on patients and DMHs.
Participants
In each participating DMH, during an index period of 13 days in the years 2009 (NE) or 2010 (NW and SO) a list of patients being treated was prepared using data from the mental health information system or from paper archives. Inclusion criteria were: an ICD-10 F20-F29 diagnosis, age ≤65 years and having at least one contact with the DMH facilities in the previous year. Patients who died or were transferred to another DMH during the previous year were excluded. Patients being in an acute episode were identified during 2 index days among those hospitalised for the first psychotic episode.
For each DMH, a statistician extracted a random sample stratified by the administrative district with a sampling ratio that was approximately 20% in NW and NE and 50% in SO, to obtain a comparable number of patients. This was done using a predefined list of random numbers matched to patients’ alphabetical list. Because patients with psychosis in the three DMHs could seek treatment at any site according to their degree of disability, level of symptoms and family ties, we sampled all of the facilities available in the catchment area for acute inpatient care, long-term residential care, outpatient clinics, rehabilitation centres and day hospitals.
Quality of care indicators
The selection of indicators was conducted by 30 health care professionals (psychiatrists, psychologists, nurses, educators, social workers) of the participating institutions during a 2-day meeting held in Pergine on 20–21 May 2009. Specifically, the health care professionals reviewed the 15 original indicators reported by Bollini et al. (Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ) and nine new indicators from the most recent PORT, NICE, Canadian Psychiatric Association, Royal Australian and New Zealand College of Psychiatrists guidelines identified as relevant by two independent researchers. The health care professionals assigned each indicator a colour according to foreseen problems in their implementation: green (no problem), red (not applicable in the Italian context or having a low level of support from guidelines) and yellow (problematic, in need of adaptation or definition of moderators specific to the local context). Three red indicators were excluded, green indicators were retained and regarding yellow indicators, participants were asked to provide written comments by e-mail after the meeting. After collecting comments, excluding one of the yellow indicators and refining the definition of indicators and their moderators, the final consensual set of 20 quality of care indicators was defined. Ten concerned pharmacological treatment and ten encompassed general care, psychosocial rehabilitation interventions, patient's work potential and collaborative decision-making with the patient's family.
The source guidelines and the criteria making up the indicators are listed in Table 1. Pharmacological indicators covered essential principles of treatment with antipsychotic drugs, namely adequate dosage, length of treatment, monotherapy, use of depot antipsychotics and management of extrapyramidal adverse reactions. Prevention and monitoring of metabolic effects was included among pharmacological treatment indicators based on evidence that second-generation antipsychotics can induce serious metabolic dysregulations, especially in drug-naive, first-episode populations, with olanzapine and clozapine having the highest likelihood to cause these abnormalities. Non-pharmacological indicators encompassed psychiatric interviews, psychosocial interventions, family involvement and vocational interventions. Appropriateness was defined as the percentage of eligible patients receiving an intervention conformant with guidelines. Moderators, i.e., reasons justifying a discrepancy between the interventions actually provided and that recommended by guidelines were recorded.
Table 1. Indicators of conformance to treatment guidelines for patients with schizophrenia and schizoaffective disorders
a The indicators monitoring and management of side effect have two levels of evidence. Level A corresponds to interventions supported by the majority of guidelines for which good scientific evidence suggests that the benefits of the clinical intervention/procedure substantially outweigh the potential risks. Level B corresponds to interventions with weaker levels of evidence and not supported by the majority of guidelines. Level B indicators have been included because they were deemed pertinent and significant for the clinical practice of Italian mental health services. Level A and B interventions are not mutually exclusive.
b McEvoy and colleagues recommended physician appointments in conjunction with non-physicians.
c PORT recommended ongoing assessment of dosage level or the need for maintenance therapy.
d The indicators related to family involvement have two levels of evidence A and B, as defined in (a).
e SOFAS (Social and Occupational Functioning Assessment Scale). Scores range from 0 to 100, with scores ≥55 indicating good social functioning and scores ≤30 indicating poor social functioning.
f McEvoy and colleagues recommended vocational rehabilitation services.
g PORT recommended the availability of a range of vocational rehabilitation services, particularly supported employment.
h Interventions considered for assertive community treatment include the following: help at home for activities of daily living, family interventions, help with finding a job or supported employment, contact or interaction with other agencies, help with prevention and health care, check of compliance with antipsychotic medication, assistance with neighbours and property owner, help with budgeting and counselling.
McEvoy: (McEvoy et al. Reference McEvoy, Scheifler and Frances1999).
PORT: (Lehman et al. Reference Lehman and Steinwachs1998).
NICE: (National Collaborating Centre for Mental Health Commissioned by the National Institute for Health and Clinical Excellence, 2002).
PORT update: (Lehman et al. Reference Lehman, Kreyenbuhl, Buchanan, Dickerson, Dixon, Goldberg, Green-Paden, Tenhula, Boerescu, Tek, Sandson and Steinwachs2004).
NICE 2009: (National Collaborating Centre for Mental Health Commissioned by the National Institute for Health and Clinical Excellence, 2009).
Canadian Psychiatric Association: (Canadian Psychiatric Association, 2005).
Royal Australian and New Zealand College of Psychiatrists (McGorry, Reference McGorry2005).
Marder et al. Reference Marder, Essock, Miller, Buchanan, Casey, Davis, Kane, Lieberman, Schooler, Covell, Stroup, Weissman, Wirshing, Hall, Pogach, Pi-Sunyer, Bigger, Friedman, Kleinberg, Yevich, Davis and Shon2004: (Marder et al. Reference Marder, Essock, Miller, Buchanan, Casey, Davis, Kane, Lieberman, Schooler, Covell, Stroup, Weissman, Wirshing, Hall, Pogach, Pi-Sunyer, Bigger, Friedman, Kleinberg, Yevich, Davis and Shon2004).
ADA/APA, 2004: (American Diabetes Association et al. 2004).
Data sources
Patients’ data were obtained by combining secondary data (patients’ medical records) with primary data on interventions and moderators through an ad hoc survey. The survey was conducted by contact persons including all the DMH professionals who were in charge of the selected patients. They were asked to fill out an ad hoc form including patients’ demographic information, psychiatric history, past and current drug treatments, side effects of drugs, the presence of comorbid physical illnesses, psychosocial interventions, family involvement, the breadth of patient's social network, participation in community life, functioning (measured on a 0–100 scale using the Social and Occupational Functioning Assessment Scale (Morosini et al. Reference Morosini, Magliano, Brambilla, Ugolini and Pioli2000) and severity of psychotic symptoms (suspiciousness, hallucinations, unusual thought content, conceptual disorganisation, rated on a 1–7 scale).
The contact person was supervised by a senior DMH professional who was in charge of the quality of the data collected. The senior DMH professional conducted the descriptive statistical analyses on patients’ characteristics and provided anonymised aggregated data to the main research team for the analysis of indicators.
Data on the structural and organisational characteristics of the DMHs and on budget allocation were retrieved from the administrative databases of the DMHs or of the Local Health Authority and were updated to the last available year.
Patients’ consent to the use of the collected data was not required because data were analysed in anonymised form, in compliance with the Italian ‘Code of conduct and professional practice applying to processing of personal data for statistical and scientific purposes’ enforced by this Authority http://www.garanteprivacy.it/web/guest/home/docweb/-/docweb-display/docweb/1115480 (Published in the Official Journal no. 190 of August 14, 2004).
The study protocol was approved by the institutional review boards of each of the three local authorities whose DMHs participated in the study.
Statistical analysis
The characteristics of patients were compared among the DMHs using analysis of variance (ANOVA) F or χ 2 test, where appropriate. Following significant tests, post hoc pairwise comparisons with Bonferroni corrections were conducted at p < 0.016 to adjust for multiple comparisons.
For each indicator, the number and percentage of appropriate interventions and of inappropriate interventions with and without moderators were calculated. Percentages of appropriate care, inappropriate care with moderators and inappropriate care without moderators were compared among DMH using the χ 2 test. Multinomial logistic regression was used to adjust these comparisons for patients’ gender, age and functioning.
Results
Characteristics of the DMHs
Structural and organisational characteristics of the DMHs are provided in Table 2. The three DMHs differed in many respects: NE had the lowest mean number of community mental health centre (CMHC) opening hours (60.9) and of patients with schizophrenia per full-time equivalent operator (3.6), the largest treated prevalence (240.2) and the highest number of general hospital psychiatric ward (GHPW) beds per 100 000 inhabitants (9.2). NW had fewer beds in GHPW and residential facilities but longer hospitalisations. The number of patients with schizophrenia per operator was similar between NW and SO, but the treated prevalence was lower in NW.
Table 2. Characteristics of the three departments of mental health
CMHC, community mental health centre; GHPW, general hospital psychiatric ward; FTE, full-time equivalent; NA, not applicable.
Patients’ characteristics
The study sample included 458 patients with schizophrenia recruited from three DMHs. Patients’ characteristics broken down by DMH are provided in Table 3. Patients were predominantly male, living with their original family, with a secondary school diploma, not working, with a limited social network and with an established and long-term relationship with mental health services (mean time in contact with mental health services: 14.0 years). Significant differences were found in the case mix of patients in the three DMHs on living arrangement (p = 0.003), working status (p = 0.001) and functioning (p < 0.001).
Table 3. Characteristics of the study samples in the three departments of mental health
NE, North-East; SO, South; NW, North-West.
ANOVA was used for continuous variables (age, illness duration, time in contact with MHS, functioning); χ 2 test for categorical variables (gender, education, living arrangement, working status, alcohol use, substance use, social network, participation in community life).
Appropriateness indicators
Table 4 provides, for each indicator and DMH, the number of eligible patients, the percentage of patients with appropriate care and with inappropriate care, and the percentage of patients with inappropriate care who had moderators to explain the inappropriateness. Four indicators (NP3A, NP6.2, NP6.3 and NP6.4) could not be computed because no patient met criteria to be included in the numerator of the indicator. This means that, although guidelines recommend family psychotherapy, cognitive behavioural therapy and art therapy, and regular attendance to support groups and self-mutual help, the use of these treatments was virtually absent.
Table 4. Number and percentages of patients receiving appropriate or inappropriate care in the three departments of mental health
P, pharmacological; NP, non-pharmacological; NE, North-East; SO, South; NW, North-West; NA, not applicable.
The χ 2 test compares all the percentages among DMHs. Denominators for the calculation of indicators are eligible patients.
In the overall sample, the level of conformance of pharmacological treatment to treatment guidelines exhibited a wide variability. During the maintenance phase, the antipsychotic dosage appeared conformant to guidelines in the large majority of patients (92%), virtually all, if patients with moderators are included. The prescription of depot to patients with non-compliance was appropriate in 27.6% of patients, with a range from 8.3% in NW to 34.6% in SO. Monitoring of extrapyramidal effects was appropriate in more than 60% of patients, while prevention and monitoring of metabolic effects was appropriate only in 30.5%. Regarding monotherapy, inappropriateness without moderators (i.e., unnecessary polytherapy) was found in 26.0%.
As to non-pharmacological indicators, conformance to guidelines for psychiatric interviews and family involvement was generally high. However, psychosocial rehabilitation was inappropriate in a large number of patients, although most of these cases were accounted for by the presence of moderators. Similarly, involvement in vocational rehabilitation training was appropriate in 38%, but an additional 50% had moderators preventing from work. Notably, assessment of patients’ work potential was poor (59.3% inappropriate without moderators).
Comparison of indicators among DMHs
Given the constraints related to the inclusion criteria and the lack of eligible subjects, only 16 indicators out of the 20 initially identified could be compared among the three DMHs to identify possible heterogeneity in the quality of care provided. For the other four indicators (P1, P2, P3 and NP6.1), the limited number of eligible subjects for one or more DMHs precluded a meaningful statistical analysis. Significant differences were found for indicators ‘Prevention and monitoring of metabolic effects’, ‘Psychiatric interviews’, ‘Psychosocial rehabilitation’, ‘Family involvement’ and ‘Work’ (Table 4). Specifically, appropriateness for monitoring metabolic side effects ranged from 6.5 to 70.0%, for psychosocial rehabilitation ranged from 10.6 to 22.4%, for psychiatric interviews ranged from 61.6 to 89.0% and for vocational training programmes from 28.7 to 44.7%. Compared with the other DMHs, higher level of inappropriateness were found at NE for indicators P8B (Prevention and monitoring of metabolic side effects), NP1 (Psychiatric interviews), NP3B (Collaborative decision making with the patient's family) and at NW for indicators NP2 (Psycho-social rehabilitation) and NP5 (Work). For three of these indicators (‘Prevention and monitoring of metabolic effects’, ‘Psychiatric interviews’ and ‘Work’) differences persisted after adjustment for patients’ gender, age and functioning in multinomial logistic regression.
Lastly, we calculated the percentage of patients receiving appropriate integrated care (at least one appropriate non-pharmacological intervention and one pharmacological intervention). The three DMHs exhibited different levels of integrated care, with a predominance of drug treatment alone at NE and SO and a higher proportion of patients receiving both pharmacological and psychosocial treatment at NW. Specifically, at NE 56.1% received drug treatment alone, 38.3% the combination of drug treatment and psychosocial interventions, and 5.7% other interventions. At SO the percentages were 69.3, 27.7 and 3 and at NW 47.3, 49.5 and 3.2%, respectively (χ 2 test = 11.9, p = 0.018).
Discussion
This study examined the results of the implementation of a set of consensual quality of care indicators based on guidelines for treatment and management of schizophrenia. Our approach implied an active involvement of the staff in a quality improvement process, consistent with existing MBQI models in Italy focused on care processes (Nuti et al. Reference Nuti, Vola, Bonini and Vainieri2016) and with institutional accreditation system processes that have been implemented since 1990 when the Italian national health system changed to a corporate model (Rossi et al. Reference Rossi, Agnetti, Bosio, De Luca, Erlicher, Morganti, Neri, Re, Semisa and Fioritti2014).
Our findings concerning pharmacological indicators showed that drug treatment during the maintenance phase and monitoring of extrapyramidal side effects were appropriate in the large majority of patients and across all DMHs. In the previous study conducted by our group using the same indicators (Bollini et al. Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ), dosage appropriateness during the maintenance was lower, ranging from 25 to 34%.
Regarding monotherapy, the percentage of patient receiving inappropriate treatment (i.e., polytherapy) in the present study (26%) was higher than a recommended cut-off of 10% (Weinmann et al. Reference Weinmann, Roick, Martin, Willich and Becker2010), but lower than the percentage of polytherapy (40.6%) reported in an Italian study conducted in 2007 (Santone et al. Reference Santone, Bellantuono, Rucci, Picardi, Preti and de Girolamo2011).
Prescription of a depot antipsychotic in patients with non-compliance was appropriate in 8 to 35% of patients. Results concerning the SIEP Direct's Project (Semisa et al. Reference Semisa, Casacchia, Di Munzio, Neri, Buscaglia, Burti, Pucci, Corlito, Bacigalupi, Parravani, Roncone, Cristofalo, Lora and Ruggeri2008) indicate that a depot antipsychotic was administered to more than 75% of patients with poor treatment compliance in 14/19 centres surveyed.
The indicator ‘monitoring of metabolic side effects’ showed a high variability among DMHs. One possible explanation is that when the study was conducted there was no concern about metabolic effects of second-generation antipsychotics. In fact, until the publication of the Lancet editorial ‘No mental health without physical health’ in 2011 (Tiihonen et al. Reference Tiihonen, Lönnqvist, Wahlbeck, Klaukka, Niskanen, Tanskanen and Haukka2011), monitoring of physical health was not deemed relevant and was not included among the qualifying goals of the DMHs. So, variability among DMHs was related to the local sensitivity to this problem. Our findings are consistent with the results of SIEP Direct's Project (Semisa et al. Reference Semisa, Casacchia, Di Munzio, Neri, Buscaglia, Burti, Pucci, Corlito, Bacigalupi, Parravani, Roncone, Cristofalo, Lora and Ruggeri2008), indicating a low attention on physical health across Italian DMHs. Still, because health care setting for persons with mental disorders encompasses other settings outside of mental health (such as primary care), we argue that both primary care and mental health providers might be involved in the management of physical conditions co-occurring with mental disorders.
The appropriateness in the use of clozapine varied from 28.6 to 77.8%, but the small sample size of eligible patients prevents from drawing conclusions.
Non-pharmacological indicators showed a large discrepancy in appropriateness among DMHs, with a higher focus on psychosocial rehabilitation, family involvement and assessment of work potential in NW, while the higher appropriateness achieved at NE for the Work indicator was related to the availability of financial resources provided by the autonomous province.
Concerning organisational determinants, we noted that inappropriateness in the monitoring of side effects, psychiatric interviews and family involvement was high in NE. However, in this DMH patients’ work or enrolment in vocational rehabilitation training was appropriate in 44.7% (reflecting a high reliance on community resources for patient's employment) and attendance in training courses was wider than in NW or SO. On the contrary, results concerning treatment appropriateness in NW indicate a high attention on psychosocial treatments and metabolic complications of antipsychotic treatment. This is consistent with the large percentage of patients receiving psychosocial interventions in addition to pharmacotherapy.
In SO, the small number of beds coupled with the high number of CMHC opening hours and the high level of appropriateness in psychiatric interviews indicates that the model in place relies mostly on community resources for treatment, although vocational rehabilitation and assessment of work potential is appropriate only for a minority of patients.
The predominance of pharmacological treatment alone in our sample (where the mean time in contact with mental health services is 14 years) raises concerns about the distance between the current available care in the Italian Mental Health Departments and the suggested ‘best practice’ of integrated care both for the short- and long-term patients (Lenroot et al. Reference Lenroot, Bustillo, Lauriello and Keith2003; Van Os & Kapur, Reference van Os and Kapur2009).
Our findings should be interpreted keeping in mind strengths and limitations. One strength is the availability of data from different sources, not limited to patients’ clinical charts and the involvement of staff members in the survey. Another strength is that indicators are compared among DMHs after adjusting for demographic characteristics, which reduces a possible bias related to case mix imbalances. Limitations include the small number of patients eligible for the calculation of some indicators, that did not allow to obtain reliable evidence regarding specific aspects of care, the heterogeneous sample, that did not allow us to aggregate our quality indicators into one single adherence index and the cross-sectional study design, that does not allow us to determine whether appropriate care is associated with better outcomes. However,Weinmann et al. (Reference Weinmann, Koesters and Becker2007) suggested that there is insufficient high-quality evidence to draw firm conclusions about the effects on outcomes of the implementation of specific psychiatric guidelines. To address this last limitation, we designed a second (ongoing) study on a sample of first-onset schizophrenia patients to assess the impact of appropriateness in the early stages of treatment on medium to long-term outcomes. Moreover, our methodology relies strongly on information provided by key clinical informant(s); therefore, it is not generalisable to contexts in which such ad hoc surveys are not feasible. We were in fact aware from our previous experience with medical chart review (Bollini et al. Reference Bollini, Pampallona, Nieddu, Bianco, Tibaldi and Munizza2008a , Reference Bollini, Pampallona, Tibaldi, Bianco, Nieddu and Munizza b ) that some information is reliably recorded (including diagnosis, current pharmacological treatment or episodes of non-compliance), while other information (i.e., psychosocial interventions at the community level, family meetings, reasons for changing medication, presence of side effects) is usually limited to brief and haphazard notes, making it impossible to define conformance and eligibility criteria on the basis of medical records alone. We tried to overcome this limitation by entrusting the collection of data to the professionals in charge of the patients, who complemented data extraction from medical records with his/her first-hand knowledge of patients’ history or by interviewing other staff members involved in the care of the patient. In order to be used in daily routine, this data collection methodology would require the adoption of electronic medical records in which information on functioning, side effects of drugs, refusal to take drugs and family involvement is collected.
In summary, the variability shown by indicators measured in DMHs located in three distant Italian regions confirms the notion that each region has its own treatment model (Ferrannini et al. Reference Ferrannini, Ghio, Gibertoni, Lora, Tibaldi, Neri and Piazza2014), in which adherence to guidelines for pharmacological treatment and psychosocial treatments may vary depending on available resources and clinical attitude for integrated care.
Thus, benchmarking among DMHs in the same region and across regions becomes more and more important to ensure horizontal equity of treatment in patients with severe mental illness. It should be noted that in Italy, where the National Health System is based on a corporate model, an institutional accreditation system is in force and no professional accreditation system has been implemented based on performance. Kilbourne et al. (Reference Kilbourne, Keyser and Pincus2010) argued that performance incentives such as pay-for-performance should be given at the group and not individual provider level, and should be rewarded based on incremental changes rather than attaining absolute benchmarks. This would reduce costs of performance incentives and maximise the potential for addressing the system-level deficiencies in care.
In the present study, feedback on local results was provided to the staff of each DMH and the Heads of the three DMHs had full autonomy with respect to the implementation of any corrective strategies or incentives.
In conclusion, we argue that MBQI cannot rely on routinely collected data only, but requires the active involvement of health care professionals for the identification of areas for improvement and promoting subsequent actions (Chou et al. Reference Chou, Vaughn, McCoy and Doebbeling2011). The use of consensus indicators adapted to the local context, in a bottom-up perspective, is a key ingredient to undertake a quality improvement initiative and favour the harmonisation of practices.
Acknowledgement
The staff of the participating DMHs is gratefully acknowledged for data collection.
Financial Support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Conflict of Interest
None.
