Introduction
Evidence indicates that the primary health issue for young people is their mental health (Kessler et al. Reference Kessler, Avenevoli, Costello, Georgiades, Green, Gruber, He, Koretz, McLaughlin, Petukhova, Sampson, Zaslavsky and Merikangas2012; Merikangas et al. Reference Merikangas, He, Burstein, Swanson, Avenevoli, Cui, Benjet, Georgiades and Swendsen2010; United Nations, 2013). According to the World Health Organization, mental health is a state of well-being in which the individual realises his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community (World Health Organisation, 2001). From this definition, a strengths-based approach to assessment in research and clinical practice has emerged, which incorporates positive indicators such as psychological well-being (Slade, Reference Slade2010). Psychological well-being is typically understood as a combination of positive affect and optimal functioning in one’s personal and social life; that is, it incorporates hedonic and eudemonic perspectives (Ryan & Deci, Reference Ryan and Deci2001). In the essence of hedonism, well-being is rooted in feelings and emotions, happiness and pleasure in life (i.e. emotional well-being). On the other hand, the eudemonic dimension is comprised of meaning, self-realisation, and positive functioning both individually and socially (i.e. social and psychological well-being).
One of the few measures to adequately assess this broader picture of well-being is the Mental Health Continuum-Short Form (MHC-SF; Keyes, Reference Keyes2002). It examines the experiences of emotional, social and psychological subjective well-being over the past month, producing continuous well-being scores and categorical cut-offs of flourishing, languishing, or moderate mental health. The MHC-SF has been validated cross-culturally with adults (Keyes et al. Reference Keyes, Wissing, Potgieter, Temane, Kruger and van Rooy2008; Lamers et al. Reference Lamers, Westerhof, Bohlmeijer, ten Klooster and Keyes2011; Lupano Perugini et al. Reference Lupano Perugini, de la Iglesia, Castro Solano and Keyes2017), young adults (Dore et al. Reference Dore, O’Loughlin, Sabiston and Fournier2017; Joshanloo, Reference Joshanloo2016; Joshanloo et al. Reference Joshanloo, Wissing, Khumalo and Lamers2013), and adolescents (Guo et al. Reference Guo, Tomson, Guo, Li, Keller and Söderqvist2015; Keyes, Reference Keyes2006; Keyes, Reference Keyes2009; Lim, Reference Lim2014). While there is support for use of the MHC-SF, the majority of research has been conducted with adults in general community, school and university settings. In addition, the measure has only more recently been used to assess changes in well-being following various interventions (McGaffin et al. Reference McGaffin, Deane, Kelly and Ciarrochi2015; Schotanus-Dijkstra et al. Reference Schotanus-Dijkstra, Drossaert, Pieterse, Boon, Walburg and Bohlmeijer2017; Turner & Deane, Reference Turner and Deane2016). As such, there is a gap in the research regarding use of the MHC-SF in youth mental health services, with young people aged 12–25 years. The American Psychological Association (APA) recommends that the validity and reliability of measures should be established for populations of interest through research. As such, this research study seeks to provide evidence to support the use of the MHC-SF in an area where the psychometric properties of the measure have not yet been investigated (APA, 2017).
The current research sought to examine the psychometric properties of the MHC-SF in Jigsaw, a primary care youth mental health service in the Republic of Ireland that works in collaboration with existing services in local areas (O’Keeffe et al. Reference O’Keeffe, O’Reilly, O’Brien, Buckley and Illback2015; O’Reilly et al. Reference O’Reilly, Illback, Peiper, O’Keeffe and Clayton2015). There are currently 13 Jigsaw services in Irish communities providing evidence-informed, therapeutic support to young people aged 12–25 years experiencing mild to moderate mental health difficulties. The MHC-SF was used in as part of routine outcome measurement in Jigsaw in 2017 after the need to introduce a strengths-based outcome measure in services was identified. Although the MHC-SF was designed to allow for categorisation of mental health on a parallel with the mental ill-health continuum, as Jigsaw adopts a biopsychosocial approach to mental health, this aspect was not considered as part of the current study.
Method
Participants
A sample of 229 young people (female n=143, male n=85, unknown n=1) aged 12-24 years (M=15.87, SD=2.51) who completed the MHC-SF prior to commencing their first intervention session in Jigsaw participated in the study. Ninety-three young people (female n=54, male n=39) aged 12–24 years (M=15.86, SD=2.56) completed the MHC-SF between June 2016 and January 2017. A total of 136 young people (Female n=89, Male n=46, unknown=1) aged 12–23 years (M=15.88, SD=2.49) completed the measure between May and October 2017.
Procedure
The procedure of administration of the MHC-SF was unchanged for both time points. Young people were invited to complete the MHC-SF by a staff member at the beginning of their first brief intervention session in Jigsaw. A brief intervention in Jigsaw consists of evidence-informed, therapeutic in-person contact between a mental health professional and a young person that typically lasts between one to six sessions. Written consent was obtained from the young person and their parent/guardian if they were under 18 years of age. The average number of sessions per young person for both time periods in the current research was 5.6 sessions.
Measure
The MHC-SF is comprised of 14 items measuring emotional (items 1–3), social (items 4–8), and psychological well-being (items 9–14). Respondents rate their feelings over the previous month on a six-point Likert scale ranging from 0 to 5 (never to every day), meaning the range in continuous scores is 0–70 (emotional well-being 0–15; social well-being 0–25; psychological well-being 0–30). Higher scores indicate higher levels of well-being.
Statistical analysis
Data were analysed using SPSS 24 and Amos 23. Missing data were minimal, and Little’s MCAR test determined that the data were missing completely at random (x 2 (161)=172.11, p=0.260). Thus the expectation maximisation (EM) technique was used to address the missing data. Cronbach’s alpha was conducted to examine the internal consistency of the MHC-SF and its subscales, with values greater than 0.7 deemed acceptable.
The structural validity of the MHC-SF was examined using Confirmatory Factor Analysis (CFA) with Amos 23 (Arbuckle, 2014). The primary aim was to examine the original three-factor structure of emotional, social, and psychological well-being (Keyes, Reference Keyes2002). In order to confirm whether the three-factor structure had the best fit in the current research, theoretical and applied models from existing research (e.g. Guo et al. Reference Guo, Tomson, Guo, Li, Keller and Söderqvist2015; Lamers et al. Reference Lamers, Westerhof, Bohlmeijer, ten Klooster and Keyes2011) were examined: a two-factor structure of hedonic (emotional) and eudemonic (social and psychological) well-being as components of subjective well-being (Ryan & Deci, Reference Ryan and Deci2001), and a single factor structure of overall subjective well-being (Keyes, Reference Keyes2006b; Machado & Bandeira, Reference Machado and Bandeira2015).
Factor loadings, correlations between factors, and a range of fit indices were explored to determine model fit. Fit indices included the Tucker–Lewis Index (TLI) and Incremental Fit Index (IFI), as they are relatively unaffected by sample size (Hu & Bentler, Reference Hu and Bentler1995), where values above 0.90 and ideally 0.95 indicate good model fit (Hu & Bentler, Reference Hu and Bentler1999). Standardized root mean square residual (SRMR), with values of<0.08 indicating good model fit (Hu & Bentler, Reference Hu and Bentler1999), and root mean square error of approximation (RMSEA), with values<0.06 (Hu & Bentler, Reference Hu and Bentler1999) indicating good model fit, were also investigated.
Results
Internal consistency
Cronbach’s alpha values for the total MHC-SF scale (α=0.913), emotional (α=0.833), social (α=0.834), and psychological (α=0.817) subscales demonstrated very good internal consistency. Removal of any item did not improve the total or subscale values.
Structural validity
The three-factor structure of emotional, social, and psychological well-being demonstrated best model fit in the current sample. On first review, modification indices indicated that the addition of covariances between e4-e5 and e5-e6, all from the social well-being factor, would improve model fit. Covariances were thus added and the subsequent fit indices are displayed in Table 1. Fit indices for the two- and one-factor models indicated noticeably poorer model fit when compared with the three-factor model fit indices (Table 1), thus indicating best model fit for the three-factor model.
Table 1 CFA fit indices for MHC-SF
Factor loadings for the three-factor model are displayed in Table 2. While most loadings were acceptable, some items displayed lower factor loadings, e.g. items 4, 7, 8, 10, and 11, all falling between 0.50 and 0.60. High correlations between latent variables were also observed (emotional-social=0.807; emotional-psychological=0.806; social-psychological=0.889), indicating that emotional, social and psychological well-being are highly inter-related.
Table 2 Factor loadings for the three-factor model for MHC-SF
Discussion
This pilot study is the first to examine the MHC-SF in a mental health service with a sample of young people aged 12–25 years. Overall findings demonstrated that the original three-factor structure of emotional, social, and psychological well-being (Keyes, Reference Keyes2002) was supported for the MHC-SF in this population and context, as was internal consistency. Lower factor loadings on some items may be due to a number of reasons. It is possible that certain items were not suitable for the context of the current research, that is an Irish sample of young people (e.g. Item 11 Positive Relations with Others “That you had warm and trusting relationships with others”). Alternatively items may not have been fully understood by young people in the current sample, e.g. Item 7 Social Acceptance “That people are basically good”. Investigation of rephrased items would be of interest.
An initial strength of this study is the novel evidence it provides regarding the suitability of the MHC-SF for an Irish primary care youth mental health context. The sample size for CFA is also adequate (MacCallum et al. Reference MacCallum, Widaman, Zhang and Hong1999). Sample size recommendations for CFA based on Monte Carlo methods are samples greater than or equal to 200 for theoretical models (Myers et al. Reference Myers, Ahn and Jin2011). Use of Cronbach’s alpha to measure internal consistency in the current study allowed for comparisons with existing research on the MHC-SF with populations of young people (Guo et al. Reference Guo, Tomson, Guo, Li, Keller and Söderqvist2015). Although it is best used to examine a specific subscale rather than a scale overall, Cronbach’s alpha is used in conjunction with factor analysis and with a relatively brief measure in the current study, as recommended by Taber (Reference Taber2017).
In terms of limitations, the lack of spread of age and smaller sample sizes for age/gender subgroups restricted further multi-group analysis. Future research would benefit from adjusting the MHC-SF in light of lower factor loadings on some items, to assess the fit of a reduced measure in a sample such as that of the current research. A further limitation of note is that the measure is not generalisable to the Irish youth population but rather to young people engaging with primary care mental health services, such as Jigsaw. However, it nonetheless provides novel evidence in this area. Future research may also consider examining convergent and discriminant validity and examining the clinical utility of the MHC-SF in the primary care youth mental health context.
Conclusions
The current findings provide initial support for the use of the MHC-SF in a primary care youth mental health service, a previously uninvestigated context. Internal consistency and a three-factor structure were supported, providing further cross-cultural validation of the MHC-SF. Nonetheless, in light of lower factor loadings and the novelty of the research, further investigation of the clinical utility of the MHC-SF in this context is needed.
Acknowledgements
The authors wish to thank Professor Corey Keyes for allowing the MHC-SF to be used in Jigsaw, the young people and their parents who took part in this research, and the Jigsaw staff who facilitated data collection.
Financial Support
The work of the authors was supported by government funding provided by the Irish Health Service Executive.
Conflict of Interest
The authors are currently, or were at the time of conducting this research, employed by Jigsaw: The National Centre for Youth Mental Health. The authors have no other conflict of interest to declare.
Ethical Standards
Ethical approval was received from the Jigsaw ethics committee. This review conforms to legal and ethical standards in the Republic of Ireland. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committee on human experimentation with the Helsinki Declaration of 1975, as revised in 2008.
