Introduction
Comorbidity among mental disorders is common (e.g. Kessler et al. Reference Kessler, McGonagle, Zhao, Nelson, Hughes and Eshleman1994; Bijl et al. Reference Bijl, Ravelli and van Zessen1998; Teesson et al. Reference Teesson, Slade and Mills2009; Hasin & Kilcoyne, Reference Hasin and Kilcoyne2012) and has been hypothesized to reflect the latent general structure of psychopathology (Sher & Trull, Reference Sher and Trull1996; Mineka et al. Reference Mineka, Watson and Clark1998; Widiger & Clark, Reference Widiger and Clark2000; Carragher et al. Reference Carragher, Krueger, Eaton and Slade2015). Much scientific work has focused on gaining a better understanding of this ‘meta structure’ of mental disorders (e.g. Mineka et al. Reference Mineka, Watson and Clark1998; Krueger & Markon, Reference Krueger and Markon2006; Carragher et al. Reference Carragher, Krueger, Eaton and Slade2015; Eaton et al. Reference Eaton, Rodriguez-Seijas, Carragher and Krueger2015; Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017) and has shown that the structure of common mental disorders can be explained by two broad underlying domains: internalizing and externalizing (Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Krueger, Reference Krueger1999; Krueger & Markon, Reference Krueger and Markon2006). Depressive disorders and anxiety disorders load on the internalizing factor and conduct disorder (CD), substance-related disorders and antisocial problems load on the externalizing factor.
The existence of separate, but correlated, internalizing, and externalizing domains has been confirmed repeatedly in empirical studies. However, the structure is likely to be more complex and multiple underlying subdomains are likely to exist. For instance, the internalizing domain has been consistently shown to have at least two lower order subfactors that explain the more specific clustering of, respectively, ‘fear’ (panic disorder, agoraphobia, specific phobia, social phobia, and obsessive–compulsive disorder ) and ‘distress’ disorders [major depressive episode (MDE), dysthymia, generalized anxiety disorder (GAD), and post-traumatic stress disorder (PTSD) e.g. Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Krueger, Reference Krueger1999; Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Watson, Reference Watson2005; Krueger & Markon, Reference Krueger and Markon2006; Slade & Watson, Reference Slade and Watson2006; Eaton et al. Reference Eaton, Keyes, Krueger, Noordhof, Skodol and Markon2013a; Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017]. For externalizing disorders, subfactors have been found that explain additional clustering of disorders that are either characterized by ‘norm-violations’ (CD, substance-use disorders) or by ‘oppositional behavior’ [oppositional-defiant disorder (ODD), attention-deficit hyperactivity disorder (ADHD); Farmer et al. Reference Farmer, Seeley, Kosty and Lewinson2009]. It has also been suggested that there may be aggression-related and substance-related subfactors (Krueger et al. Reference Krueger, Markon, Patrick, Benning and Kramer2007). Overall, the findings on the subfactor structure of the externalizing domain have so far been less consistent than for the internalizing domain (e.g. Krueger et al. Reference Krueger, Markon, Patrick and Iacono2005; Markon & Krueger, Reference Markon and Krueger2005; Vrieze et al. Reference Vrieze, Perlman, Krueger and Iacono2012).
Despite ongoing discussions about the best lower order structure, the higher order, two-factor model has been found to be very robust and is considered to reflect the natural structure of common mental disorders (Krueger, Reference Krueger1999; Watson, Reference Watson2005; Krueger & Markon, Reference Krueger and Markon2006; Kotov et al. Reference Kotov, Chang, Fochtmann, Mojtabai, Carlson and Sedler2011a, Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmermanb; Carragher et al. Reference Carragher, Krueger, Eaton and Slade2015; Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). Various studies have supported this idea. For instance, variations on the internalizing and externalizing domains have been found to be linked to variations on distinct genetic risk factors (Kendler et al. Reference Kendler, Aggen, Knudsen, Røysamb, Neale and Reichborn-Kjennerud2011; Lahey et al. Reference Lahey, Krueger, Rathouz, Waldman and Zald2017). In addition, the internalizing and externalizing domains have been shown to account for a large part of comorbidity patterns that are observed over a patients’ lifetime: the association of disorders with subsequent onset was found to be stronger within each domain than between domains (Kessler et al. Reference Kessler, Cox, Green, Ormel, McLaughlin and Merikangas2011a, Reference Kessler, Ormel, Petukhova, McLaughlin, Green and Russob, Reference Kessler, Petukhova and Zaslavskyc). However, disorders from each of the separate domains have also been shown to predict each other over time (Lahey et al. Reference Lahey, Krueger, Rathouz, Waldman and Zald2017). Structural studies that investigated psychopathology together with personality disorders have shown that the joint latent structure can be described with a finite number of dimensions, including internalizing and externalizing dimensions (Markon, Reference Markon2010; Røysamb et al. Reference Røysamb, Kendler, Tambs, Orstavik, Neale and Aggen2010; Kendler et al. Reference Kendler, Aggen, Knudsen, Røysamb, Neale and Reichborn-Kjennerud2011; Kotov et al. Reference Kotov, Chang, Fochtmann, Mojtabai, Carlson and Sedler2011a, Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmermanb; South & Jarnecke, Reference South and Jarnecke2017). The two-factor structure has been shown to be structurally invariant across ethnic groups in the US (Eaton et al. Reference Eaton, Krueger, Markon, Keyes, Skodol and Wall2013b), gender (Hicks et al. Reference Hicks, Blonigen, Kramer, Krueger, Patrick and Iacono2007; Kramer et al. Reference Kramer, Krueger and Hicks2007; Eaton et al. Reference Eaton, Keyes, Krueger, Balsis, Skodol and Markon2012) and over time (e.g. Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Eaton et al. Reference Eaton, Krueger and Oltmanns2011).
Despite the many insights that have been gained from the above described research, important research questions still remain unsatisfactorily answered. The current study aims to address the following of these questions: (1) how stable is the cross-national structure of common mental disorders and (2) where do uncommon mental disorders fit into the structure?
The first question deals with what has been referred to as ‘structural validity’. Establishing the structural validity of the model of mental disorders is very important as this stability across countries determines to what extent factors and (mean) factor scores can be validly compared across countries when conducting mental health research. Therefore, more insight must be gained into cross-national measurement invariance of the latent structure of common mental disorders. Currently, there is little available research on this. Although the same general latent structure has been replicated in different countries (e.g. Australia: Slade & Watson, Reference Slade and Watson2006; the Netherlands: Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Norway: Røysamb et al. Reference Røysamb, Kendler, Tambs, Orstavik, Neale and Aggen2010), most of this work is from Western countries. Krueger et al. (Reference Krueger, Chentsova-Dutton, Markon, Goldberg and Ormel2003) conducted the only cross-national study of the latent structure of mental disorders with data from the WHO Collaborative Study of Psychological Problems in General Health Care, which were collected in 15 countries around the world (Brazil, Chile, China, France, Germany, Greece, India, Italy, Japan, the Netherlands, New Zealand, Nigeria, Turkey, the UK, and the US). They showed that two factors (internalizing v. alcohol problems) described the structure best in the cross-national and most of the country datasets. Formal evaluations of measurement invariance showed that the model had configural invariance and metric invariance (invariant factor loadings across countries) across countries, indicating that the factor-loading configuration was invariant but that factor means and residual variances could vary across countries. Unfortunately, this study included only one externalizing disorder (‘hazardous use of alcohol’), prohibiting a thorough investigation of the structural stability of the full externalizing domain. In the current cross-national study, the stability of a model based on a broader range of disorders could be investigated.
The optimal placement of many less common mental disorders into the two-factor model has remained unclear, partly due to the limited availability of datasets that include these less common disorders. Fortunately, more recent, large epidemiological datasets have enabled researchers to extend the original two-factor model by including additional disorders. These studies have shown that psychotic experiences (PEs) have been found to load on a separate factor (‘thought disorders’ or ‘psychosis’; Markon, Reference Markon2010; Kotov et al. Reference Kotov, Chang, Fochtmann, Mojtabai, Carlson and Sedler2011a, Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmermanb; Keyes et al. Reference Keyes, Eaton, Krueger, Skodol, Wall and Grant2013; Wright et al. Reference Wright, Krueger, Hobbs, Markon, Eaton and Slade2013; Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). Bipolar disorders have been found to load on this ‘thought disorders’ factor as well (Kotov et al. Reference Kotov, Chang, Fochtmann, Mojtabai, Carlson and Sedler2011a, Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmermanb), but others have found a ‘bipolar/mania’ subfactor of the internalizing domain (Forbush & Watson, Reference Forbush and Watson2013; Kotov et al. Reference Kotov, Perlman, Gámez and Watson2015) or have found bipolar disorder to cross-load on the fear and distress subfactors (Eaton et al. Reference Eaton, Keyes, Krueger, Noordhof, Skodol and Markon2013a), making it unclear whether bipolar disorder belongs with the internalizing disorders, the thought disorders or both (Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). Eating disorders have been found to group with the internalizing disorders, more specifically, as a subfactor of the internalizing domain (Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark and Keel2010; Forbush & Watson, Reference Forbush and Watson2013). Taken together, research on uncommon disorder placement has been comparatively scarce and the results rather inconsistent. This could reflect the multifactorial nature of the studied uncommon disorders, but could also be explained by methodological differences (e.g. the kind and number of included symptoms/disorders) which brings us to a more nuanced notion of common and uncommon disorders: some of the common disorders have rather low prevalence (e.g. panic disorder) while some of the uncommon disorders are in fact more prevalent (e.g. eating disorder). This has a lot to do with methodological differences in their grouping but also in exclusion of subthreshold variants of a disorder (e.g. PEs v. schizophrenia). Although not perfect, we will use the term uncommon disorders in the present study to describe eating disorders, bipolar disorder and PEs. Both from an etiological and clinical perspective, it would be very useful to find out much more about the placement of these disorders into the latent structure, as this could provide very interesting clues about the nature and the extent of overlap between vulnerabilities for common and uncommon, but very severe, disorders (Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). To gain a more systematic and complete insight into this, more research is needed using samples, in which all common but also (several) uncommon disorders were systematically assessed. Such datasets were available for the current study.
Given the above described need for further research, the current study aimed to investigate (1) the latent structure of common mental disorders and its measurement invariance across countries, and (2) the placement of uncommon disorders into the latent structure. Data came from the World Mental Health (WMH) Surveys, which were conducted in several countries around the world and included a comprehensive assessment of both common internalizing and externalizing disorders [e.g. ADHD, CD, ODD, and intermittent explosive disorder (IED), substance abuse and dependence] and less common disorders. More specifically, a range of models, based on the previous literature was tested to evaluate the optimal placement of (1) eating disorders, (2) bipolar disorder and (3) PEs in the model.
Methods
WMH Surveys
Data came from the WMH Surveys. The WHO Composite International Diagnostic Interview (CIDI) version 3.0 was administered in 29 WMH surveys across the world (Table 1). Most surveys used stratified multistage clustered area probability household sampling with no substitution for non-participants. Data collection took place between 2001 and 2012, and response rates ranged from 45.9 to 97.2%, with an average of 69.5%. Classification of country income categories was based on the World Bank criteria at the time of each survey (The World Bank, 2009). All WMH surveys were conducted face-to-face by lay interviewers who had received standardized training. Standardized translation, back-translation, harmonization and quality control procedures were applied in all of the participating survey sites (Pennell et al. Reference Pennell, Mneimneh, Bowers, Chardoul, Wells, Viana, Kessler and Üstün2008). Informed consent was obtained according to protocols endorsed by local Institutional Review Boards.
Table 1. WMH sample characteristics by World Bank income categoriesa
a The World Bank (2012) Data. Accessed 12 May 2012 at: http://data.worldbank.org/country. Some of the WMH countries have moved into new income categories since the surveys were conducted. The income groupings above reflect the status of each country at the time of data collection. The current income category of each country is available at the preceding URL.
b NSMH (The Colombian National Study of Mental Health); EMSMP (La Encuesta Mundial de Salud Mental en el Peru); MMHHS (Medellín Mental Health Household Study); M-NCS (The Mexico National Comorbidity Survey); RMHS (Romania Mental Health Survey); NISHS (Northern Ireland Study of Health and Stress); EZOP (Epidemiology of Mental Disorders and Access to Care Survey); PEGASUS-Murcia (Psychiatric Enquiry to General Population in Southeast Spain-Murcia); NCS-R (The US National Comorbidity Survey Replication).
c Most WMH surveys are based on stratified multistage clustered area probability household samples in which samples of areas equivalent to counties or municipalities in the US were selected in the first stage followed by one or more subsequent stages of geographic sampling (e.g. towns within counties, blocks within towns, households within blocks) to arrive at a sample of households, in each of which a listing of household members was created and one or two people were selected from this listing to be interviewed. No substitution was allowed when the originally sampled household resident could not be interviewed. Several WMH surveys (Poland, Spain-Murcia) used country resident or universal health-care registries to select respondents without listing households. Four of the 10 surveys are based on nationally representative samples.
d The response rate is calculated as the ratio of the number of households in which an interview was completed to the number of households originally sampled, excluding from the denominator households known not to be eligible either because of being vacant at the time of initial contact or because the residents were unable to speak the designated languages of the survey. The weighted average response rate is 69.9%.
e Colombia moved from the “lower and lower middle income” to the “upper middle income” category between 2003 (when the Colombian National Study of Mental Health was conducted) and 2010 (when the Medellin Mental Health Household Study was conducted), hence Colombia's appearance in both income categories. For more information, please see footnote a.
The CIDI was divided into two parts, with part I assessing core mental disorders and part II additional disorders and correlates. Part I was completed by all subjects and part II was administered to all subjects meeting criteria for any of the part I disorders and a probability subsample of the other subjects. To adjust for differential sampling, all responses in the part II subsample were weighted by the inverse of their probability of selection into the part II sample.
Mental disorders
Analyses were conducted with lifetime CIDI/DSM-IV diagnoses (present/absent), without hierarchy rules. The following common mental disorders were included: MDE, dysthymia, panic disorder, agoraphobia, social phobia, specific phobia, GAD, PTSD, ADHD, ODD, alcohol abuse, drug abuse, CD, and IED. The prevalence rates of these disorders in the different countries have been described previously (see Supplementary 1 for disorder-specific references). Alcohol and drug abuse were combined into a single substance-abuse variable (with or without substance dependence). Bipolar-i, bipolar-ii, and subthreshold bipolar disorder were assessed with the CIDI and were combined into a single bipolar disorder variable (present/absent). In the eating disorders dataset, bulimia and binge-eating disorder were assessed in all part II subjects in Romania, Brazil, and Poland and in a random part of part-II subjects in the other countries. combined into a single eating-disorder variable (anorexia nervosa was not included due to very low prevalence). The assessed PEs included hallucinations (visual and auditory) and delusions [insertion/withdrawal of thoughts, mind control by some strange force, ideas of reference (e.g. telepathy), plot to harm you/people following you]. In this dataset, PEs were assessed in all part II subjects in Brazil and Romania, and in a random part of the part II sample in the other countries. The six assessed PEs were used as individual input variables in the confirmatory factor analyses (CFAs).
Samples
To enable inclusion of as much disorders as possible, all current analyses were run in subjects in the part II sample within the age range of 18–44 years because most externalizing disorders (e.g. ADHD) were only assessed in these subjects to limit recall bias. Bipolar disorder, eating disorders and PEs were only assessed in subsamples. Therefore, models including these respective disorders were estimated in the subsets of countries, in which they were assessed.
The subsample that was used to investigate the structure of all common mental disorders included 10 samples from nine countries (Brazil, Colombia, Colombia-Medellin, Mexico, Murcia, North-Ireland, Peru, Poland, Romania, US; n = 15 499). This dataset was also used to investigate the optimal placement of bipolar disorder. The additional placement of IED into the model was investigated in a subsample where IED was assessed (Brazil, Colombia, North-Ireland, Peru, Poland, Romania, US; n = 12 162). The placement of eating disorders was investigated in a subsample (n = 10 585) that included all subjects that were assessed for eating disorders with the CIDI. In Romania, Brazil, and Poland, eating disorders were assessed in all part II subjects. In the other countries, eating disorders were assessed in a random part of part-II subjects. The placement of PEs was investigated in a subsample of six countries, in which they were assessed in addition to common mental disorders and bipolar disorder (Brazil, Colombia, Mexico, Peru, Romania, US; n = 5478). In each of these countries, PEs were assessed in a random subsample and only those who had completed the psychosis section were included.
Analyses
Hypothesized models from the literature were fit to the WMH surveys data using CFAs. Multigroup-CFA (MG-CFA) was used to investigate structural invariance across countries. All CFAs were conducted with Mplus 7 (Muthén & Muthén, Reference Muthén and Muthén1998–2012), using a mean and variance adjusted weighted least squares estimator. All CFAs were run using the Mplus procedures for complex designs and included WMH surveys design variables (clusters and strata) to account for effects of the complex design (Asparouhov, Reference Asparouhov2005) and weights to adjust for differential probabilities of selection and discrepancies with census data (see Heeringa et al. Reference Heeringa, Wells, Hubbard, Kessler and Üstun2008 for more info on sampling weights and design variables in WMH). The comparative fit index (CFI) and the root mean square error of approximation (RMSEA) were used to evaluate model fit, with a CFI ⩾ 0.95 and an RMSEA ⩽ 0.06 indicating good fit to the data (Hu & Bentler, Reference Hu and Bentler1999).
In the MG-CFAs, the model with the best cross-national fit was estimated with increasing levels of invariance across countries: configural invariance (similar patterns of factor loadings and item thresholds; loadings and thresholds may differ across countries), partial invariance (constrained factor loadings across countries, freely estimated item thresholds in each country), and scalar invariance (factor loadings and item thresholds constrained across countries). The difference in CFI between models with different levels of invariance (ΔCFI) was used to compare models with different levels of measurement invariance. A difference of ⩾0.01 has previously been suggested to indicate a meaningful difference when comparing two groups (Cheung & Rensvold, Reference Cheung and Rensvold2002).
Results
Bivariate associations
Table 2 shows the tetrachoric correlations between the lifetime disorders. The highest correlations among common disorders were observed between MDE and dysthymia (0.82), ODD and CD (0.71), MDE and GAD (0.65), GAD and PTSD (0.65), ADHD and ODD (0.65), and dysthymia and GAD (0.61). In addition, high correlations were observed between several of the PEs. The lowest correlations among common disorders were observed for agoraphobia with CD (0.24) and agoraphobia with substance abuse (0.19), eating disorder with CD (0.21), and specific phobia with substance abuse. In addition, lower correlations were observed between many of the PEs and other disorders.
Table 2. Tetrachoric correlations of lifetime common mental disorders, bipolar disorder, eating disorder, and psychotic experiences
All correlation coefficients estimated with Mplus 7.0.
MDE, major depressive episode; Dysth, dysthymia; PTSD, post-traumatic stress disorder; GAD, generalized anxiety disorder; PD, panic disorder; Ago, agoraphobia; SAD, social anxiety disorder; Sp.Phob, specific phobia; ADHD, attention-deficit hyperactivity disorder; CD, conduct disorder; ODD, oppositional-defiant disorder; Substance, substance abuse disorder (with/without dependence); PE, psychotic experience; Vis. Hal., having a visual hallucination; Aud.Hal, having an auditory hallucination; Thought, believing that thoughts are extracted from or inserted into your head ; Mind cont., feeling that your mind was taken over by a strange force; Telepathy, believing that a strange force tries to communicate with you (e.g. through special signs from the radio or TV); Delusion: believing that there is a plot going on against you or that people follow you.
a Estimated in sample including Colombia, Peru, Romania, Brazil, Northern Ireland, Poland, Murcia, Medellin, Mexico, and the United States (n = 15 499).
b Estimated in subsample including Colombia, Peru, Romania, Brazil, Northern Ireland, Poland, and the United States (n = 12 162).
c Estimated in sample including the complete data or a random subsample that got the eating disorders assessment from Colombia, Peru, Romania, Brazil, Northern Ireland, Poland, Murcia, Medellin, Mexico, and the United States (n = 10 585).
d Estimated in subsample including complete data or a random subsample that got the psychotic experiences assessment from Colombia, Peru, Romania, Brazil, Mexico, and the United States (n = 5478).
e IED was not investigated together with eating disorders bipolar disorder and PEs; eating disorders were not investigated together with IED, bipolar disorders, and PEs.
Structural model of the common mental disorders
A simple two-factor model and a higher order model with distress and fear subfactors were estimated (Fig. 1a; Table 3). Of these models, the higher order model showed the best fit to the data (CFI = 0.985; RMSEA = 0.017), in line with a large body of previous work (Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Krueger, Reference Krueger1999; Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Watson, Reference Watson2005; Krueger & Markon, Reference Krueger and Markon2006; Slade & Watson, Reference Slade and Watson2006; Eaton et al. Reference Eaton, Keyes, Krueger, Noordhof, Skodol and Markon2013a; Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). In this model, all disorders showed considerable standardized loadings on their respective factors, the fear and distress factors showed considerable loadings on the second-order internalizing factor and the correlation between the second-order internalizing and externalizing factors was 0.61. Similar results were found when the analyses were run in a subset of countries that also assessed IED.
Fig. 1. Factor loadings (unstandardized) of different structural models in the WMH surveys. One loading per factor was fixed to 1 for model identification. INT, internalizing; EXT, externalizing; PSY, psychotic experiences; PD, panic disorder; Ago, agoraphobia; SAD, social anxiety disorder; Sp.Phob, specific phobia; MDE, major depressive episode; Dysth, dysthymia; PTSD, post-traumatic stress disorder; GAD, generalized anxiety disorder; ADHD, attention-deficit hyperactivity disorder; CD, conduct disorder; ODD, oppositional-defiant disorder; Sub, substance abuse disorder (with/without dependence); Eat, eating disorder; Bipol, bipolar disorder; Visual, having visual hallucination; Audit, having auditory hallucination; Thought, having ideas of thought extraction or thought insertion; Mind, feeling that your mind was taken over by an external force; Reference, feeling that a strange force tries to communicate with you; Plot, belief in an unjust plot that is going on to harm you.
Table 3. Confirmatory factor analyses of the structure of common and uncommon mental disorders
a Including abuse and dependence.
b Model without cross-loadings and with freely estimated correlation(s) between factors.
c Model with freely estimated correlation between the higher-order factors.
d Eating disorders were assessed in random subsamples in 7 of the 10 included countries.
e Psychotic experiences were assessed in random subsamples in 4 of the 6 samples.
Structural validity
The invariance of the model of common mental disorders (minus IED) across countries was tested with MG-CFA using country as the group variable. When fitted to the individual samples, good fit of the model was observed in Brazil (CFI = 0.969; RMSEA = 0.017), Colombia (CFI = 0.975; RMSEA = 0.015), Colombia-Medellin (CFA = 0.976; RMSEA = 0.022), Mexico (CFI = 0.982; RMSEA = 0.016), and the US (CFI = 0.98; RMSEA = 0.022). In the remaining countries (North Ireland, Murcia, Poland, Romania, and Peru), model fitting was complicated by correlations close to 0 (or negative) in the disorder correlation matrix (mostly due to low frequencies of one or more disorders), which led to parameter estimates that were hard to interpret. Subsequent measurement-invariance analyses were therefore conducted with the first five samples from four countries. The results (Table 4) showed that the model with configural invariance across the five samples fit the data well (CFI = 0.978; RMSEA = 0.019). Constraining the first-order and second-order factor loadings led to a small decrease in fit (CFI = 0.976; RMSEA = 0.018). Constraining all first- and second-order factor loadings and all item thresholds to be the same across samples (scalar invariance) led to a larger decrease in fit (CFI = 0.962; RMSEA = 0.021; ΔCFI = 0.016). The decrease in CFI from the configural model to the model with constrained factor loadings (ΔCFI = 0.002) indicated that partial invariance did hold across the five samples. The decrease in CFI from the configural model to the scalar model (ΔCFI = 0.016) indicated that the model did not have scalar invariance across the five samples.
Table 4. Analyses of measurement invariance across five samples of the higher order structural model of common mental disorders
CFI, comparative fit index; RMSEA, root mean squared error of approximation.
All models testing invariance across 10 samples.
All models estimated with the θ parameterization in Mplus 7 (Muthén & Muthén, Reference Muthén and Muthén1998–2012).
Structural model of common disorders and bipolar disorders
The placement of bipolar disorder in the higher order model was investigated next (Table 3). Different ways of grouping bipolar disorder with the internalizing factor all led to better fit than grouping it with the externalizing domain. Models with bipolar disorder loading separately on the second-order internalizing factor alongside the distress and fear subfactors [in line with Forbush & Watson (Reference Forbush and Watson2013) and Kotov et al.(Reference Kotov, Perlman, Gámez and Watson2015)] showed slightly better fit (CFI = 0.982; RMSEA = 0.017) than a configuration with bipolar disorder loading on either the fear or distress subfactors, or bipolar disorder cross-loading on the fear and distress subfactors [as found by Eaton et al. (Reference Eaton, Keyes, Krueger, Noordhof, Skodol and Markon2013a, Reference Eaton, Krueger, Markon, Keyes, Skodol and Wallb); CFI = 0.980; RMSEA = 0.018], but the difference in model fit was very small, suggesting that bipolar disorder can be grouped with the internalizing domain.
Structural model of common disorders and eating disorders
The placement of eating disorders in the higher order model was investigated next. In line with previous work (Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark and Keel2010; Forbush & Watson, Reference Forbush and Watson2013), placing eating disorders as a subdomain under the internalizing domain led to better fit than grouping eating disorders with the externalizing domain. However, unlike previously found, a model with eating disorders loading directly on the internalizing factor (i.e. as a separate internalizing subdomain) alongside the distress and fear subfactors was not the best-fitting solution. A model with eating disorders loading on the distress and/or on the fear subfactor(s) showed better fit. However, fit was very similar across the latter models (all RMSEA = 0.015) making it hard to draw definite conclusions about the best subfactor grouping of eating disorders (see Fig. 1b for an illustration of the model with eating disorders loading on the distress subfactor).
Structural model of common disorders and PEs
A model with all PEs loading on a separate factor alongside the internalizing and externalizing factors fit the data well (Table 3; CFI = 0.972; RMSEA = 0.015) and a higher order model with a PEs factor, and distress and fear subfactors for the internalizing factor showed even better fit (CFI = 0.984; RMSEA = 0.011).
Structural model of common disorders, bipolar disorder, and PEs
A model with all common disorders, bipolar disorder, and PEs was estimated next. IED and eating disorders were not included in these analyses because they were only assessed in smaller subsamples. Several higher order models were compared, evaluating whether bipolar disorder could be better grouped with the internalizing disorders, with the PEs, or with both (see Table 3). The results showed that a model with bipolar disorders set to load directly on the internalizing factor (see Fig. 1c) fit the data better than a model with bipolar disorder loading on the same factor as the PEs. Setting bipolar disorder to cross-load on both factors did not improve fit much (same RMSEA) compared with the first model. In this model, bipolar disorders showed a very small standardized loading on the PEs factor (−0.19), which differed strongly in terms of direction and magnitude from the other PEs loadings (0.76–0.91).
Discussion
We addressed several unresolved issues regarding the structure of mental disorders using the largest cross-national dataset to date encompassing a relatively wide selection of both internalizing and externalizing disorders, and both common and uncommon disorders. Overall, we found that a higher order model with an internalizing and externalizing factor and fear and distress subfactors, described the structure of common mental disorders well. Investigations of measurement invariance of this structure across five selected countries showed that the factor loadings can be assumed to be stable. Investigations of the placement of uncommon disorders into the model showed that eating disorders and bipolar disorder could be grouped under the internalizing factor and that PEs were best modeled as a separate domain within the model. When bipolar disorder and PEs were included together in a single model, bipolar disorder was still best grouped with the internalizing domain (but not with a specific subfactor).
The results confirmed the higher order structure of common mental disorders that has been observed in many previous studies (i.e. Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Krueger, Reference Krueger1999; Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Watson, Reference Watson2005; Krueger & Markon, Reference Krueger and Markon2006; Slade & Watson, Reference Slade and Watson2006; Miller et al. Reference Miller, Fogler, Wolf, Kaloupek and Keane2008; Eaton et al. Reference Eaton, Krueger, Markon, Keyes, Skodol and Wall2013b). Importantly, the current findings about measurement invariance showed that the structure was relatively stable across five different countries. These findings align with the only previous cross-national study that also found evidence for metric cross-national measurement invariance (Krueger et al. Reference Krueger, Chentsova-Dutton, Markon, Goldberg and Ormel2003). The current results extend on these previous findings by showing that a higher order model that encompasses a full range of externalizing disorders, instead of only substance-related problems, fits the data well and shows metric invariance across countries.
When investigated together with the common mental disorders, bipolar disorder was found to be best grouped with the internalizing factor, in line with previous research showing bipolar/mania to be an internalizing subfactor (e.g. Forbush & Watson, Reference Forbush and Watson2013; Kotov et al. Reference Kotov, Perlman, Gámez and Watson2015). Other authors found bipolar disorder to cross-load on the distress and fear subfactors when investigated in the context of other internalizing disorders (Eaton et al. Reference Eaton, Keyes, Krueger, Noordhof, Skodol and Markon2013a, Reference Eaton, Krueger, Markon, Keyes, Skodol and Wallb), but such model configurations were not found to fit better in the current study. These results indicate that, in the context of the included common mental disorders, bipolar disorder shows a clear general association with the internalizing domain. Previously, bipolar disorders have also been shown to be partly related to a ‘thought problems’ dimension together with, e.g. psychosis (Kotov et al. Reference Kotov, Krueger, Watson, Achenbach, Althoff and Bagby2017). Therefore, this was also evaluated by investigating a model that included both bipolar disorder and PEs. These analyses showed that grouping bipolar disorder strictly with the PEs led to a decrease in fit and letting bipolar disorder cross-load on the internalizing and PEs factor did hardly improve fit. This could be explained by the fact that the internalizing and PEs factors were already strongly correlated, making the added cross-loading redundant. In addition, the fact that six PEs from a single interview module (see below for further discussion) were used and just a single bipolar-disorder indicator, could have made the setup of the analysis suboptimal to evaluate the existence of a broad thought-disorder factor. Finally, an alternative explanation may be that there is a significant heterogeneity in our measure of bipolar disorder. As it consists of different subtypes and represents different episodes, it is possible that some subtypes or episodes may be more strongly related to either internalizing disorders or thought problems.
As previously observed (Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark and Keel2010; Forbush & Watson, Reference Forbush and Watson2013), models grouping eating disorders with the internalizing disorders fit best to the data. Setting eating disorders to load on the fear, distress or both subfactors led to similar model fit, but led to better fit than setting eating disorders to load directly on the internalizing domain. These findings suggest that, when investigated in the context of common mental disorders, eating disorders show a general association with the internalizing domain and its subdomains, although its exact placement remained somewhat unclear. These somewhat ambiguous findings could partly be related to the fact that only one pooled eating-disorder indicator was used in the current model. It could be that a separate eating disorder (sub)factor could be identified if more individual indicators were used, also including anorexia nervosa.
In line with previous findings that PEs reflect a separate mental-health domain (i.e. ‘thought problems’) alongside the internalizing and externalizing domains (e.g. Markon, Reference Markon2010; Wright et al. Reference Wright, Krueger, Hobbs, Markon, Eaton and Slade2013), the current results showed that a model with PEs loading on their own separate factor fit the data well. Interestingly, this factor showed considerable correlation with the internalizing factor, indicating that PEs are related to other mental disorders, in line with previous work showing the cross-diagnostic importance of PEs (Rössler et al. Reference Rössler, Hengartner, Ajdacic-Gross, Haker and Angst2011; Fusar-Poli et al. Reference Fusar-Poli, Bonoldi, Yung, Borgwardt, Kempton and Valmaggia2012; Werbeloff et al. Reference Werbeloff, Drukker, Dohrenwend, Levav, Yoffe and van Os2012).
The current results align with previous work in the WMH surveys. Kessler et al. (Reference Kessler, Cox, Green, Ormel, McLaughlin and Merikangas2011a, Reference Kessler, Ormel, Petukhova, McLaughlin, Green and Russob) investigated the role of the internalizing and externalizing domains in the development of comorbidity patterns within persons over time and showed that these were explained to a strong degree by within-domain clustering. The current results confirmed that the 2-factor structure of mental disorders and expanded on these results by providing more insight into the finer-grained structure and placement of mental disorders into a higher order factor model.
An important implication of the higher order model is that variations in mental health/disorders could be explained by causal influences that occur at different levels with different degrees of disorder specificity. Some influences can be disorder-specific, some occur at the level of the subfactor and are shared with a limited set of other disorders (i.e. other fear or distress disorders), and some occur at the level of the higher order domains (i.e. internalizing or externalizing) and are shared with all other disorders within the domain.
It is important to note that there are also influences that are very non-specific and influence all disorders, irrespective of their grouping with first- or second-order factors (Lahey et al. Reference Lahey, Krueger, Rathouz, Waldman and Zald2017). Indeed, the internalizing and externalizing domains have been repeatedly shown to be robustly correlated, which could indicate the existence of an overarching general factor of psychopathology that accounts for the shared variance of all disorders (‘p-factor’; Caspi et al. Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington and Israel2017; Lahey et al. Reference Lahey, Krueger, Rathouz, Waldman and Zald2017). To investigate this, bifactor models can be used that incorporate both a general factor and domain-specific group factors (e.g. Lahey et al. Reference Lahey, Krueger, Rathouz, Waldman and Zald2017). Although it has been pointed out that bifactor models can become complex and very hard to interpret (Eid et al. Reference Eid, Geiser, Koch and Heenein press; Koch et al. Reference Koch, Holtmann, Bohn and Eidin press), their use in structural research of mental disorders is an interesting topic for further investigation.
There are some limitations that should be kept in mind when interpreting the findings of this study. First, the analyses were limited to those WMH surveys with data on all disorders and to subjects in the age range of 18–45 years, which may limit the generalizability of the findings to other populations. Second, the models were fitted on cross-national datasets but prevalence rates could differ across surveys. Still, the investigations of measurement invariance indicated that at least partial invariance across countries could be assumed, although only part of the countries could be included in these analyses as some countries showed very low disorder frequencies. Investigations of measurement invariance across larger sets of countries will probably only be possible with smaller models that include a very limited range of disorders. This could be done in future research. Third, PEs, rather than psychotic disorders and schizophrenia were assessed and included in the analyses. On the one hand, this could seem somewhat inconsistent, as all other modeled mental problems were included as formal disorders. On the other hand, PEs are more common than full-blown psychotic disorders, which provides more possibilities to model the full psychotic spectrum, irrespective of diagnosis. However, it is possible that some of the observed clustering of PEs in the analyses can be explained by the fact that they were assessed differently from the other diagnoses. Finally, the general indices of fit indicated good fit for all tested models. As such all model comparisons were among models that (objectively) fit the data well and were often based on very small differences in fit.
The current results show that the structure of mental disorders is best represented by a higher order factor model with some degree of cross-national stability. In addition, eating disorders were shown to group with the internalizing domain, bipolar disorder was shown to group most strongly with the internalizing domain and PEs were shown to group with their own domain that was correlated with the other domains.
Acknowledgements
This work was supported by a VICI grant (no: 91812607) received by Peter de Jonge from the Netherlands Organization for Scientific research (NWO-ZonMW). The World Health Organization World Mental Health (WMH) Survey Initiative is supported by the United States National Institute of Mental Health (NIMH; R01 MH070884), the John D. and Catherine T. MacArthur Foundation, the Pfizer Foundation, the United States Public Health Service (R13-MH066849, R01-MH069864, and R01 DA016558), the Fogarty International Center (FIRCA R03-TW006481), the Pan American Health Organization, Eli Lilly and Company, Ortho-McNeil Pharmaceutical Inc., GlaxoSmithKline, and Bristol-Myers Squibb. The authors thank the staff of the WMH Data Collection and Data Analysis Coordination Centers for assistance with instrumentation, fieldwork, and consultation on data analysis. The São Paulo Megacity Mental Health Survey is supported by the State of São Paulo Research Foundation (FAPESP) Thematic Project Grant 03/00204-3. The Colombian National Study of Mental Health (NSMH) is supported by the Ministry of Social Protection. The Mental Health Study Medellín – Colombia was carried out and supported jointly by the Center for Excellence on Research in Mental Health (CES University) and the Secretary of Health of Medellín. The Mexican National Comorbidity Survey (MNCS) is supported by the National Institute of Psychiatry Ramon de la Fuente (INPRFMDIES 4280) and by the National Council on Science and Technology (CONACyT-G30544- H), with supplemental support from the PanAmerican Health Organization (PAHO). The Northern Ireland Study of Mental Health was funded by the Health & Social Care Research & Development Division of the Public Health Agency. The Peruvian World Mental Health Study was funded by the National Institute of Health of the Ministry of Health of Peru. The Polish project Epidemiology of Mental Health and Access to Care – EZOP Project (PL 0256) was supported by Iceland, Liechtenstein, and Norway through funding from the EEA Financial Mechanism and the Norwegian Financial Mechanism. EZOP project was co-financed by the Polish Ministry of Health. The Romania WMH study projects “Policies in Mental Health Area” and “National Study regarding Mental Health and Services Use” were carried out by National School of Public Health & Health Services Management (former National Institute for Research & Development in Health, present National School of Public Health Management & Professional Development, Bucharest), with technical support of Metro Media Transilvania, the National Institute of Statistics – National Centre for Training in Statistics, SC. Cheyenne Services SRL, Statistics Netherlands and were funded by Ministry of Public Health (former Ministry of Health) with supplemental support of Eli Lilly Romania SRL. The Psychiatric Enquiry to General Population in Southeast Spain – Murcia (PEGASUS-Murcia) Project has been financed by the Regional Health Authorities of Murcia (Servicio Murciano de Salud and Consejería de Sanidad y Política Social) and Fundación para la Formación e Investigación Sanitarias (FFIS) of Murcia. The US National Comorbidity Survey Replication (NCS-R) is supported by the National Institute of Mental Health (NIMH; U01-MH60220) with supplemental support from the National Institute of Drug Abuse (NIDA), the Substance Abuse and Mental Health Services Administration (SAMHSA), the Robert Wood Johnson Foundation (RWJF; Grant 044708), and the John W. Alden Trust. The Brazilian Council for Scientific and Technological Development supports Dr Laura Helena Andrade (CNPq Grant # 307784/2016–9). None of the funders had any role in the design, analysis, interpretation of results, or preparation of this paper. The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of the World Health Organization, other sponsoring organizations, agencies, or governments.
Declaration of interest
In the past 3 years, Dr Kessler received support for his epidemiological studies from Sanofi Aventis, was a consultant for Johnson & Johnson Wellness and Prevention, Sage Pharmaceuticals, Shire, Takeda; and served on an advisory board for the Johnson & Johnson Services Inc. Lake Nona Life Project. Kessler is a co-owner of DataStat, Inc., a market research firm that carries out healthcare research.
