Introduction
Mental illness is a serious public health concern affecting nearly 140 million individuals in the USA during their lifetimes (Regier et al. Reference Regier, Kaelber, Rae, Farmer, Knauper, Kessler and Norquist1998). Mental disorders are associated with divorce, lost work time/productivity, social stigma, caregiver burden and suicide (Broadhead et al. Reference Broadhead, Blazer, George and Tse1990; Henriksson et al. Reference Henriksson, Aro, Marttunen, Heikkinen, Isometsä, Kuoppasalmi and Lönnqvist1993; Lesage et al. Reference Lesage, Boyer, Grunberg, Vanier, Morissette, Ménard-Buteau and Lover1994; Kessler & Frank, Reference Kessler and Frank1997; Kessler et al. Reference Kessler, Walters and Forthofer1998; Crisp et al. Reference Crisp, Gelder, Rix, Meltzer and Rowlands2000; Ohaeri, Reference Ohaeri2003; Corrigan, Reference Corrigan2004). The negative effects of mental illness are even greater for individuals who suffer from more than one psychiatric disorder, such that as the number of mental health diagnoses increases, educational attainment declines and teenage pregnancy, divorce rates and functional impairment increase (Kessler et al. Reference Kessler, Foster, Saunders and Stang1995, Reference Kessler, Walters and Forthofer1998; Skodol et al. Reference Skodol, Oldham, Hyler, Stein, Hollander, Gallaher and Lopez1995). Moreover, it appears that suffering from more than one mental illness is the norm rather than the exception (Kessler et al. Reference Kessler, Chiu, Demler and Walters2005). Understanding the nature and structure of these co-occurrences is, therefore, a major public health priority.
To address problems associated with diagnostic co-morbidity, researchers have modeled the underlying dimensions of psychopathology statistically. The results of these studies have consistently demonstrated that, at the higher-order level, a latent internalizing factor accounts for co-morbidity patterns among unipolar mood and anxiety disorders, whereas a latent externalizing factor underlies the covariation of substance-use disorders and adult antisocial behaviors. At the second-order level, internalizing disorders split into distress disorders (which include major depression, dysthymia, generalized anxiety disorder and post-traumatic stress disorder), fear disorders (which include panic disorder, the phobias and obsessive–compulsive disorder) and eating disorders (Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark, Keel, Legrand and Watson2010) (for a review, see Eaton et al. Reference Eaton, South, Krueger, Millon, Krueger and Simonsen2010).
This quantitative phenotypic structure is robust, as indicated by its invariance across cultures (Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Kessler & Üstün, Reference Kessler and Üstün2004; Slade & Watson, Reference Slade and Watson2006), sex (Eaton et al. Reference Eaton, Keyes, Krueger, Balsis, Skodol, Markon, Grant and Hasin2012) and time (Krueger et al. Reference Krueger, Caspi, Moffitt and Silva1998; Vollebergh et al. Reference Vollebergh, Iedema, Bijl, de Graaf, Smit and Ormel2001; Measelle et al. Reference Measelle, Stice and Hogansen2006; Eaton et al. Reference Eaton, Krueger and Oltmanns2011). Genetic covariance structure modeling of twin data has shown that the phenotypic structure of mental disorders is largely due to additive genetic factors (Kendler et al. Reference Kendler, Prescott, Myers and Neale2003; Hettema et al. Reference Hettema, Neale, Myers, Prescott and Kendler2006). However, uncommon mental disorders typically have been excluded from these structural analyses (for exceptions, see Slade & Watson, Reference Slade and Watson2006; Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark, Keel, Legrand and Watson2010; Markon, Reference Markon2010). This reflects the fact that disorders with low base rates are difficult to incorporate within structural analyses with sufficient statistical precision. However, recent large-scale epidemiological data provide an opportunity to model the covariance among common and uncommon mental disorders, as well as to extend this structure to a broader range of Axis II psychopathology. Here we present a revised structural model of mental disorders using data from the Collaborative Psychiatric Epidemiology Surveys, an integration of three nationally representative surveys. The purpose of the study was to replicate the placement of eating disorders within the structure and to extend previous structural analyses of psychopathology by examining the location of impulse control disorders, bipolar disorders, and dimensionally based personality disorders within the internalizing–externalizing framework.
Impulse control disorders
A large body of research supports the inclusion of certain impulse control disorders within the externalizing spectrum. Substance-use disorders co-occur with pathological gambling and pyromania at a high rate, with lifetime co-morbidity estimates ranging from 76.3% to 80.4% in nationally representative samples (Kessler et al. Reference Kessler, Hwang, LaBrie, Petukhova, Sampson, Winters and Schaffer2008; Blanco et al. Reference Blanco, Alegria, Petry, Grant, Simpson, Liu, Grant and Hasin2010). Externalizing and impulse control disorders also share similar neuropsychological dysfunction, gender ratios, treatment response and etiologic influences (Slutske et al. Reference Slutske, Eisen, True, Lyons, Goldberg and Tsuang2000, Reference Slutske, Eisen, Xian, True, Lyons, Goldberg and Tsuang2001; Blanco et al. Reference Blanco, Myers and Kendler2012). These similarities have led to suggestions to reclassify pathological gambling, intermittent explosive and compulsive shopping disorders as behavioral addictions (Black et al. Reference Black, Shaw and Blum2010; Grant et al. Reference Grant, Potenza, Weinstein and Gorelick2010) and have led the Diagnostic and Statistical Manual of Mental Disorders, version 5 (DSM-5) Work Group to propose moving pathological gambling disorder to the ‘Substance use and addictive disorders’ section (APA, 2011).
Nevertheless, compulsive shopping disorder demonstrates substantially higher co-morbidity with major depression than externalizing disorders (Black et al. Reference Black, Repertinger, Gaffney and Gabel1998) and intermittent explosive disorder demonstrates consistent, moderate correlations with both internalizing and externalizing disorders (Fincham et al. Reference Fincham, Grimsrud, Corrigall, Williams, Seedat, Stein and Myer2009; Kessler et al. Reference Kessler, Ormel, Petukhova, McLaughlin, Green, Russo, Stein, Zaslavsky, Aguilar-Gaxiola, Alonso, Andrade, Benjet, de Girolamo, de Graaf, Demyttenaere, Fayyad, Haro, Hu, Karam, Lee, Lepine, Matchsinger, Mihaescu-Pintia, Posada-Villa, Sagar and Üstün2011; Yoshimasu & Kawakami, Reference Yoshimasu and Kawakami2011), suggesting potentially complex associations between impulse control disorders and other forms of psychopathology.
Bipolar disorders
Wolfe et al. (Reference Wolf, Schubert, Patterson, Grande, Brocco and Pendleton1988) conducted an exploratory factor analysis of several mental disorders and found that mania and depression formed an affective disorders factor that was independent from externalizing and schizophrenia factors. More recent studies found high correlations between bipolar disorder, attention-deficit hyperactivity disorder and oppositional defiant disorder, in addition to strong, positive correlations between bipolar disorder, depression and dysthymia (Krishnan, Reference Krishnan2005; Merikangas et al. Reference Merikangas, Akiskal, Angst, Greenberg, Hirschfeld, Petukhova and Kessler2007, Reference Merikangas, Jin, He, Kessler, Lee, Sampson, Viana, Andrade, Hu, Karam, Ladea, Medina-Mora, Ono, Posada-Villa, Sagar, Wells and Zarkov2011) . Consistent with recent bivariate analyses of co-morbidity, Kessler et al. (Reference Kessler, Ormel, Petukhova, McLaughlin, Green, Russo, Stein, Zaslavsky, Aguilar-Gaxiola, Alonso, Andrade, Benjet, de Girolamo, de Graaf, Demyttenaere, Fayyad, Haro, Hu, Karam, Lee, Lepine, Matchsinger, Mihaescu-Pintia, Posada-Villa, Sagar and Üstün2011) found that bipolar I and II disorders loaded most strongly on an internalizing factor, yet had cross-loadings on externalizing.
Markon (Reference Markon2010) found that mania did not load substantially on any factor, nor did it form an independent factor. These divergent findings may be explained by between-study differences in the disorders included in (or omitted from) analyses or, as suggested by Markon (Reference Markon2010), could be due to an under-representation of mania symptoms in his analyses. The present study will contribute to a better understanding of where bipolar disorders fit within structural models of mental disorder by examining an expanded set of mental disorders and multiple markers of bipolar disorders.
Personality disorders
The internal structure of personality disorders is a topic that has received considerable attention, particularly because the DSM-5 Workgroup has proposed significant changes in the way these disorders are diagnosed. Currently the personality disorders are divided into three subgroups, labeled cluster A (paranoid, schizoid, and schizotypal personality disorders), cluster B (antisocial, borderline, histrionic, and narcissistic personality disorders) and cluster C (avoidant, dependent, and obsessive–compulsive personality disorders). However, statistical analyses of the 10 DSM-defined personality disorders do not support a three-factor model. Krueger et al. (Reference Krueger, Eaton, Clark, Watson, Markon, Derringer, Skodol and Livesley2011) reviewed the literature on the empirical structure of personality pathology and found evidence for six dimensions that have replicated across several samples and self-report instruments. These dimensions were characterized by propensities toward negative emotionality/emotional instability, introversion, antagonism, disinhibition, schizotypy/psychoticism, and compulsivity.
Studies suggest that two broad classes or superordinate ‘meta-traits’, representing internalizing and externalizing, have emerged from analyses of self-report measures of pathological personality traits or the DSM-IV personality disorder symptoms (De Clercq et al. Reference De Clercq, De Fruyt, Van Leeuwen and Mervielde2006;Kushner et al. Reference Kushner, Quilty, Tackett and Bagby2011). De Clercq et al. (Reference De Clercq, De Fruyt, Van Leeuwen and Mervielde2006) found that internalizing was comprised of neuroticism (which included dependent and submissive traits), affective instability, introversion, and compulsivity, whereas the externalizing factor subsumed disagreeableness. Kushner et al. (Reference Kushner, Quilty, Tackett and Bagby2011) found similar results, with the exception that compulsivity loaded on externalizing.
Present study
Based on our review of co-morbidity patterns, we hypothesize that personality disorder dimensions (except antagonism/disinhibition) and bipolar disorders will load most strongly with internalizing, whereas pathological gambling disorder, intermittent explosive disorder and personality disorder dimensions characterized by antagonism/disinhibition will be indicators of externalizing.
Method
Participants
Participants were individuals from the Collaborative Psychiatric Epidemiological Surveys (CPES), an integration of three nationally representative multi-stage area probability samples: the National Comorbidity Survey – Replication (n = 5692), the National Survey of American Life (n = 6082) and the National Latino and Asian American Study (n = 4649). The CPES-weighted sample is similar to the 2000 United States Census in terms of education, marital status and geographic distribution. The sample consisted of male (41.36%) and female (58.64%) adults who were a mean age of 42.61 (s.d. = 16.16) years. Participants reported the following ethnic/racial backgrounds: Vietnamese (3.17%), Filipino (3.09%), Chinese (3.65%), ‘other’ Asian (3.35%), Cuban (3.51%), Puerto Rican (3.01%), Mexican (7.39%), ‘other’ Hispanic (5.96%), Afro-Caribbean (8.99%), African American (25.87%), non-Latino white (30.88%) and ‘other’ (1.13%). Due to missing data, the final sample included 16 233 participants. Further details about the study sample and recruitment strategies can be found in Heeringa et al. (Reference Heeringa, Wagner, Torres, Duan, Adams and Berglund2004) .
Measures
The World Health Organization 2004 version of the Composite International Diagnostic Interview (Kessler & Üstün, Reference Kessler and Üstün2004) assessed the lifetime presence of DSM-IV diagnoses. Although a diagnosis of bipolar II disorder requires that an individual has never had a manic episode, we did not apply this hierarchical exclusion rule to allow for the possibility that bipolar disorders might form their own latent class of disorder. Obsessive–compulsive disorder was not included because it was not assessed in the National Latino and Asian American Study, was assessed using the DSM-IV Short Form Module in The National Survey of American Life, and was underestimated in the National Comorbidity Survey – Replication due to issues with skip logic. Non-affective psychosis was ‘not sufficiently robust to be included in the publically-released dataset’ (Collaborative Psychiatric Epidemiology Surveys, 2011).
Due to low base rates of anorexia nervosa (n = 31) and pathological gambling disorder (n = 34), we examined subthreshold forms of these disorders. Subthreshold anorexia nervosa (n = 177) was diagnosed if an individual met all criteria for anorexia nervosa with the exception of criterion D (amenorrhea). Studies suggest that the amenorrhea criterion does not add diagnostic specificity to the identification of anorexia nervosa nor is it associated with differences in personality traits, patterns of co-morbidity, or demographics (Garfinkel et al. Reference Garfinkel, Lin, Goering, Spegg, Goldbloom, Kennedy, Kaplan and Woodside1996; Cachelin & Maher, Reference Cachelin and Maher1998; Watson & Andersen, Reference Watson and Andersen2003; Mitchell et al. Reference Mitchell, Cook-Myers and Wonderlich2005; Thomas et al. Reference Thomas, Vartanian and Brownell2009). Full-threshold pathological gambling disorder requires the presence of five out of ten symptoms that reflect ‘persistent and maladaptive gambling behavior’. Subthreshold pathological gambling disorder (n = 125) was diagnosed if individuals met any one of these symptoms. Studies examining differences between full- and subthreshold pathological gamblers have found comparable patterns of co-morbidity (Kessler et al. Reference Kessler, Hwang, LaBrie, Petukhova, Sampson, Winters and Schaffer2008), but lower levels of novelty seeking in subthreshold gamblers (Cunningham-Williams et al. Reference Cunningham-Williams, Cottler, Compton and Spitznagel1998).
The CPES dataset included 34 screening questions from the International Personality Disorder Examination (IPDE) (Loranger, Reference Loranger1999). These items were selected by the CPES investigators in order to carry out a small clinical reappraisal study of personality disorders (see Collaborative Psychiatric Epidemiology Surveys, 2011). These items were used to examine symptoms of cluster A, B and C personality disorders. Cluster A traits were poorly represented in the CPES dataset; thus we were unable to model a factor corresponding to the schizotypy domain. Participant responses were coded as true or false. Two items were recoded due to skip logic in which individuals who responded false to ‘I have never been arrested’ were coded as missing to ‘I have done things that could get a person arrested’. These variables were merged so that participants met this symptom if they had ever been arrested or engaged in activities for which they could have been arrested. This resulted in a revised total of 33 items.
Statistical analyses
We began by deriving a baseline model using more established variables, which was then used to test how less established variables fit into the internalizing–externalizing scheme. To accomplish this, data analysis proceeded in three steps. In the first step, an exploratory principal factor analysis (EFA) of the personality disorder screening items was carried out to form scales that were entered into subsequent structural analyses. In step two, a joint EFA was carried out for personality disorder scales and Axis I psychiatric diagnoses. The results from EFAs were used to develop a baseline confirmatory factor analysis (CFA) model.
EFAs and CFAs were estimated using Mplus version 6 (Muthén & Muthén, Reference Muthén and Muthén2007), whereas Stata/SE 12.0 (StataCorp LP, USA) and SPSS 19 (IBM, USA) were used for other analyses. EFAs and CFAs were estimated using robust weighted least squares, which is appropriate for analysing categorical or ordinal variables (Brown, Reference Brown and Brown2006b). EFAs used oblique rotation, given that factors typically have correlated significantly with one another in previous structural analyses of psychopathology (e.g. Slade & Watson, Reference Slade and Watson2006; Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark, Keel, Legrand and Watson2010). CFA models were scaled by fixing the variance of each latent factor to 1.0. To account for the study's complex design, standard errors and χ2 tests of model fit were computed by taking into account the stratification and non-independence due to cluster sampling and the unequal probabilities of selection.
To determine the optimal number of factors to extract in EFAs, we carried out parallel analysis (Hayton et al. Reference Hayton, Allen and Scarpello2004), which is a Monte Carlo-based simulation that generated 100 random datasets with the same number of participants and variables as the actual dataset. This analysis allowed for a determination of when factors could be considered trivial and when the extraction of additional factors was no longer warranted.
CFA model fit was evaluated using criteria outlined by Hu & Bentler (Reference Hu and Bentler1999) . The Bayesian Information Criterion (BIC) (Raftery, Reference Raftery1995) was used to compare nested and non-nested CFA models. Due to the number of CFA models examined, factor loadings for observed indicators were considered significant at p < 0.001 or less.
Results
Creation of personality disorder scales
We carried out an initial EFA on all 33 personality disorder screening items. Results of parallel analysis indicated that no more than four meaningful factors could be extracted from the data. Nine items with low primary loadings (<|0.40|) and two items with high cross-loadings (>|0.30|) were dropped from the model and the EFA was re-run (see Table 1 footnote for a list of omitted items). The resulting one- through four-factor solutions were interpretable, meaning that each solution had a clear, meaningful factor structure, with at least two marker items per factor. Table 2 shows model selection statistics, including χ2, root mean square error of approximation (RMSEA) and root mean square residual (RMSR) values. Overall, the three- and four-factor solutions demonstrated a better fit to the data than the one- and two-factor solutions. However, RMSEA and RMSR values favored a four-factor model. Although the third factor was defined by only two markers, this solution appeared meaningful and its inclusion will allow for an examination of where obsessive–compulsive personality traits fit within the full psychopathology structure. Promax-rotated factor correlations ranged from 0.14 for antisocial behaviors and rigidity to 0.58 for borderline and pathological introversion.
Table 1. EFA of IPDE screening questionsa
EFA, Exploratory factor analysis; IPDE, International Personality Disorder Examination.
a n = 10695. Unstandardized oblique factor loadings are reported because Mplus does not provide standardized factor loadings for exploratory factor analyses. Scales were computed by summing the responses to the markers of each factor. Factor 1 was labeled ‘borderline’, factor 2 was labeled ‘antisocial behaviors’, factor 3 was labeled ‘rigidity’ and factor 4 was labeled ‘pathological introversion’. The following IPDE items were omitted due to primary loadings <|0.40|: ‘Show my feelings for everyone to see’, ‘Giving into urges gets me in trouble’, ‘Feel bad when hurt or upset someone’, ‘Lose temper and get in physical fights’, ‘Hard to stay out of trouble’, ‘At times, fail to meet financial obligations’, ‘Others make fun behind my back’, ‘Held grudges for years’, and ‘Convinced conspiracy behind many things in world’. ‘I'm very moody’ and ‘Feelings always changing’ were omitted due to cross-loadings >|0.30|.
b Factor loadings ⩾|0.35|.
Table 2. EFA model selection statisticsa z
EFA, Exploratory factor analysis; df, degrees of freedom; RMSEA, root mean square error of approximation; RMSR, root mean square residual; IPDE, International Personality Disorder Examination.
a Models 1–4 , for the IPDE, were significant at p < 0.001. Models 1–6 , for Axis I and II indicators, were significant at p < 0.001, whereas model 7 was significant at p < 0.01. For EFA models, Mplus provides only the RMSEA and RMSR. Acceptable values of RMSEA are indicated by 0.06 or less, whereas RMSR values of 0.08 or less are suggestive of good fit.
Items that loaded above |0.40| on a given factor from the four-factor solution were summed to create a personality disorder scale for that factor (see Table 1). We labeled these scales borderline, antisocial behaviors, rigidity, and pathological introversion. Polychoric correlations for the personality scales ranged from 0.12 for antisocial behaviors and rigidity to 0.45 for borderline and introversion. Coefficient α's for scales were 0.59 (rigidity), 0.62 (pathological introversion), 0.70 (antisocial behaviors) and 0.73 (borderline). Given that coefficient α is both a function of the inter-item correlation and the number of variables, the lower than recommended coefficient α's for rigidity and pathological introversion largely reflect the low number of items for these scales.
EFA of Axis I and II indicators
Eigenvalues generated by parallel analysis were only slightly greater than eigenvalues from the actual data in the seven-factor solution (difference in eigenvalues = 0.086). Thus, we examined the one- through seven-factor solutions for all Axis I and II indicators. Fit indices (i.e. RMSEA, RMSR, the overall model χ2) favored a seven-factor model, compared with the one- through six-factor models (see Table 2). This solution had clear externalizing, fear, substance use, bipolar, distress, eating disorders, and dysphoria factors (see Table 3). Although dysphoria had only one strong marker item and two lower loading items, we favored a rich factor model (with more factors), when interpretable. The dysphoria factor included personality scales characterized by high negative affect and low positive affect, which is relevant to the detachment, negative affectivity, and compulsivity trait domains currently under consideration by the DSM-5 Task Force. The inclusion of this factor will, therefore, provide information about the location of several proposed trait domains within the internalizing–externalizing framework.
Table 3. EFA of Axis I and II indicators
EFA, Exploratory factor analysis.
a Unstandardized oblique factor loadings are reported because Mplus does not provide standardized factor loadings for exploratory factor analyses. Factor loadings above 1.00 are due to reporting unstandardized loadings. Because factor loadings greater than 1.00 can be associated with Heywood cases (negative residual variances associated with model over-fitting), we re-ran this analysis using Stata 12.0, which provides standardized factor loadings, and did not find factor loadings greater than 1.00 (for the unrotated and promax rotated solutions). We also did not find negative residual variances that were significantly different from zero for any variables in the original Mplus analysis. We chose to report unstandardized factor loadings from Mplus (versus standardized factor loadings from Stata) because Stata does not allow for complex survey design estimation procedures for exploratory factor analysis.
b Factor loadings ⩾|0.35|.
CFA
We examined the fit of various CFA models, using the results of the seven-factor EFA and theory to guide model development. The initial CFA model included two latent factors: internalizing and externalizing (model 1). Externalizing variables included antisocial behaviors and alcohol use, attention-deficit hyperactivity, conduct, intermittent explosive, pathological gambling and substance-use disorders. Next, we examined the superordinate structure of internalizing by modeling endogenous latent factors, including distress (major depression, generalized anxiety disorder, dysthymia and post-traumatic stress disorder), fear (social phobia, specific phobia, agoraphobia and panic disorder), eating disorders (anorexia nervosa, bulimia nervosa and binge eating disorder), bipolar disorders (bipolar I and bipolar II disorder) and dysphoria (pathological introversion, borderline and rigidity) (model 2, baseline model). Finally, we collapsed the distress and fear factors into a single factor, given that some previous research suggests that the distinction between distress and fear does not always result in improved model fit (model 3) (Kessler et al. Reference Kessler, Ormel, Petukhova, McLaughlin, Green, Russo, Stein, Zaslavsky, Aguilar-Gaxiola, Alonso, Andrade, Benjet, de Girolamo, de Graaf, Demyttenaere, Fayyad, Haro, Hu, Karam, Lee, Lepine, Matchsinger, Mihaescu-Pintia, Posada-Villa, Sagar and Üstün2011; Kotov et al. Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmerman2011). For each of the above-mentioned models, intermittent explosive and pathological gambling disorders were regressed on externalizing, due to their affinity towards externalizing in EFA analyses. Model 2 was the best fitting, based on a comparison of fit indices and BIC values (see Table 4). Model 2 was selected as the ‘baseline’ model with which to compare subsequent CFA models.
Table 4. Model comparisons for confirmatory factor analysesa
df, Degrees of freedom; CFI, comparative fit index; TLI, Tucker–Lewis index; BIC, Bayesian Information Criterion; n.a., not applicable; RMSEA, root mean square error of approximation.
a The following values reflected acceptable levels of fit: (1) CFI and TLI ⩾0.95 and (2) RMSEA ⩽0.06. BIC ▵ was calculated by subtracting the current model from a comparison model and taking the absolute value of the difference.
b BIC ▵ reflects comparison between model 1 versus model 2.
c BIC ▵ reflects comparison between model 2 versus model 3.
d BIC ▵ reflects comparison between current model versus baseline model 2.
Personality disorder scales
We compared four models with the baseline model: (1) borderline was regressed on distress, pathological introversion and rigidity were regressed on fear, and antisocial behaviors was an indicator of externalizing (model 4); (2) model 5 was identical to the baseline model, except that borderline was regressed on externalizing; (3) personality scales represented their own latent factor (model 6); and (4) personality scales were regressed on externalizing (model 7). Based on a comparison of BIC values, the baseline model was the best fitting (see Table 4).
Bipolar disorders
Three models were compared with the baseline model, in which bipolar disorders: (1) were indicators of distress within internalizing (model 8), (2) were indicators of externalizing (model 9), and (3) represented their own latent factor (model 10). The baseline model and model 10 fit better than models 8 and 9. However, the difference in BIC values for the baseline model and model 10 were small (see Table 4). Due to the high correlation between the latent internalizing and bipolar factors (r = 0.72), we modeled bipolar disorders as a part of the internalizing spectrum. Thus, the baseline model remained the best fitting.
Impulse control disorders
We examined whether impulse control disorders were best characterized as their own latent factor (model 11) or as a subfactor within internalizing (model 12). BIC values indicated that the baseline model fit better than model 12. BIC ▵ was small for the baseline model versus model 11 (see Table 4) and the correlation between the impulse control and externalizing factors was too high for them to be considered distinct factors (r = 0.96). We, therefore, retained the baseline model.
Eating disorders
Because previous research theorized that anorexia nervosa is an indicator of the fear subfactor within internalizing – whereas bulimia and binge eating disorder are indicators of externalizing (Forbush et al. Reference Forbush, South, Krueger, Iacono, Clark, Keel, Legrand and Watson2010) – we examined this possibility in model 13. In the other models, eating disorders represented their own latent factor (model 14) or were indicators of externalizing (model 15). The best-fitting model was the baseline model.
Cross-loadings
First, we allowed impulse control disorders to cross-load on internalizing (model 16). Second, in separate models, we allowed the following variables to cross-load on externalizing: (1) bipolar disorders (model 17), (2) scales comprising the dysphoria factor (model 18), and (3) eating disorders (model 19). Results indicated that intermittent explosive disorder (λ = 0.230, z = 5.93, p < 0.001), but not pathological gambling disorder (λ = −0.157, z = − 2.23, p = 0.026), loaded significantly on internalizing. Intermittent explosive disorder loaded most strongly on fear (λ = 0.255, z = 6.60, p < 0.001), compared with distress (λ = 0.232, z = 5.94, p < 0.001) and dysphoria (λ = 0.242, z = 7.06, p < 0.001). Bipolar I disorder (λ = 0.230, z = 5.61, p < 0.001) and the borderline personality scale (λ = 0.179, z = 9.97, p < 0.001) – but not bipolar II disorder (λ = −0.054, z = 1.12, p = 0.262), the rigidity scale (λ = −0.044, z = − 1.72, p = 0.086), or the pathological introversion scale (λ = −0.063, z = 2.08, p = 0.038) – loaded significantly on externalizing. None of the eating disorders had significant cross-loadings on externalizing (λ's ranged from −0.067 to 0.046, z's ranged from −0.99 to 0.69, p's ranged from 0.321 to 0.494).
Final best-fitting model
In the best-fitting model (model 20), eating disorders, bipolar disorders, and personality disorder scales characterized by dysphoria, were subfactors within internalizing, whereas impulse control disorders and antisocial behaviors were indicators of externalizing. Intermittent explosive disorder had significant cross-loadings on internalizing and borderline traits had significant cross-loadings on externalizing. The cross-loading of bipolar I disorder on externalizing was not retained in the final sample because subsample analyses revealed that bipolar I did not demonstrate a significant cross-loading on externalizing for black or Hispanic participants. Otherwise, subsample analyses did not suggest meaningful ethnic/racial differences for the final model. The final best-fitting model is shown in Fig. 1.
Fig. 1. Completely standardized path diagram of the best-fitting structural equation model of common and uncommon disorders within the diagnostic taxonomy. MDD, major depression; GAD, generalized anxiety disorder; PTSD, post-traumatic stress disorder; DYS, dysthymia; SPP, specific phobia; SOCP, social phobia; PAN, panic disorder; AGOR, agoraphobia with or without panic; BN, bulimia nervosa; BED, binge eating disorder; AN, subthreshold anorexia nervosa; INTR, pathological introversion scale; RIGD, rigidity scale; BORD, borderline scale; BPII, bipolar II disorder; BPI, bipolar I disorder; IED, intermittent explosive disorder; PG, subthreshold pathological gambling disorder; DRG, drug-use disorders; ALCH, alcohol-use disorders; CD, conduct disorder; ASPD, antisocial behaviors personality scale; ADHD, attention-deficit hyperactivity disorder. Factor loadings for bipolar I and II were set to equality due to empirical under-identification, which we believe resulted from the high correlation (r = 0.895) between these indicators. Setting highly correlated indicators to equality is preferred over using the sum or mean of the two items, because the former approach allows for the estimation of measurement error (Bollen, Reference Bollen and Bollen1989; Brown, Reference Brown and Brown2006a). a Cross-loadings.
Discussion
The goal of the present study was to replicate the placement of eating disorders within the internalizing–externalizing model of psychopathology and to expand structural models of mental disorders by incorporating additional uncommon forms of mental illness. Results indicated that eating and bipolar disorders formed subfactors within internalizing, impulse control disorders were indicators of externalizing, and factor analytically-derived personality disorder scales split between internalizing and externalizing factors.
The location of eating disorders within this structure replicates the findings of Forbush et al. (Reference Forbush, South, Krueger, Iacono, Clark, Keel, Legrand and Watson2010). These results indicate that future research seeking to examine general and specific risk factors for eating pathology should focus primarily on commonalities with (and differences from) mood and anxiety disorders, rather than substance-use, impulse control or antisocial personality disorders.
The present study extends recent structural analyses of Axis I and II disorders (Markon, Reference Markon2010; Kotov et al. Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmerman2011) in several important ways. First, this was the first study to examine where certain impulse control disorders fit within a major taxonomic model of mental disorders. Results indicated that subthreshold pathological gambling and intermittent explosive disorder are markers of externalizing, which lends support to proposals to group these disorders with addictive disorders (for a review, see Grant et al. Reference Grant, Potenza, Weinstein and Gorelick2010). An interesting result was that intermittent explosive disorder had significant ties to fear-based internalizing psychopathology. These results suggest that under conditions of heightened negative affect, certain individuals may alternate between anxious avoidance (or suppression of angry emotions) and angry approach behaviors. Future work is needed to examine whether common physiological and cognitive processes underlie co-occurrences between intermittent explosive and anxiety disorders; such information has the potential to aid in the treatment of individuals suffering from reoccurring episodes of severe anger expression.
Second, this study helps to clarify the placement of bipolar disorders within the diagnostic taxonomy. Our findings suggest that bipolar disorders are clear markers of internalizing. One limitation of our study, however, is that we did not examine other potential indicators of thought disorders. Thus, we were not able to test the possibility that bipolar disorders fall within the thought disorder spectrum (see Kotov et al. Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmerman2011).
Finally, this was the first study to examine the location of factor analytically-derived DSM personality disorder trait scales within the internalizing–externalizing structure. The results of the best-fitting four-factor model of personality disorder symptoms were strikingly consistent with prior structural analyses of Axis II traits (O'Connor, Reference O'Connor2002) and are similar to the proposed personality trait domains that are under consideration by the DSM-5 Task Force (APA, 2011). One exception, however, was that we did not find a schizotypy or psychoticism factor. The lack of a schizotypy factor is due to the fact that cluster A personality disorder symptoms were not well represented in the present study. Nevertheless, these results are the first to indicate how the proposed DSM-5 personality trait domains fit within syndrome-based models of the full psychopathology structure and support the notion that personality disorders need not be placed on a distinct axis from Axis I disorders.
Certain limitations may have an impact on the interpretation of our results. The loadings of subthreshold pathological gambling disorder and rigidity on their latent factors were low. This is likely due to the lower severity of sub- versus full-threshold pathological gambling disorder, as well as the fact that there were few indicators of the rigidity scale. In addition, although the cross-loadings for intermittent explosive disorder and the borderline scale were significant, they were relatively low. These low cross-loadings, combined with the results of EFA, suggest that these disorders are weak markers of the exogenous latent factors upon which they cross-loaded. These results are nevertheless encouraging, as they indicate that the model delineated in this study possesses good to strong discriminant validity, a feature that is not characteristic of the current diagnostic system.
Although the present study is one of the most comprehensive analyses of the structure of psychopathology to date, a final limitation is that some forms of psychopathology were omitted from the current dataset. For example, as noted earlier, the inclusion of thought disorders would have been useful for conceptualizing bipolar disorders, given that Kotov et al. (Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmerman2011) found that manic episodes combined with psychosis to define a factor that was independent from internalizing and externalizing. Nevertheless, Kotov et al. (Reference Kotov, Ruggero, Krueger, Watson, Yuan and Zimmerman2011) did not examine full-threshold bipolar I disorder and did not include bipolar II disorder, and may have had limited ability to model associations between bipolar and internalizing disorders. Future epidemiologic studies are, therefore, needed to examine the full range of psychopathology.
Despite its limitations, this study has significant strengths. A unique advantage of this study, compared with prior nationally representative studies, is the use of a racially and ethnically diverse sample. In addition, whereas other studies of the structure of psychopathology have had limited power to examine rare forms of psychopathology, the CPES's large sample size provided an opportunity to incorporate several new disorders within a taxonomic model of mental illness. Finally, the inclusion of personality trait dimensions provided information relevant to proposals to change the way personality disorders are diagnosed in DSM-5, prior to the implementation of the new diagnostic system.
The current study provides a useful context for understanding core psychopathological processes. Examining shared features between disorders is crucially important, as the emergence of a single disorder in isolation over one's lifetime is relatively rare. It is likely that individuals manifest multiple mental disorders due to common vulnerabilities that exert etiologic influence over a class of psychopathology. A greater understanding of the core processes that cut across internalizing and externalizing disorders will be useful for identifying who is at risk for the development of certain broad classes of psychopathology.
Nevertheless, there are important differences between subclasses of disorders and between disorders within subclasses. A greater understanding of features specific to these classes and disorders will help target disparate aspects of psychopathology. For this to be useful, however, symptom dimensions for disorders themselves must have good phenotypes. This is a problem for many areas of mental illness, as disorders are often heterogeneous and show different patterns depending on the particular set of symptoms a person experiences. Future work is needed to refine the lower-order structure of psychopathology, as well as examine how the lower-order structure relates to the higher-order structure identified in this paper.
In conclusion, this was the first large-scale nationally representative study that has included uncommon mental disorders with sufficient power to examine their fit within a structural model of mental disorders. Given that genetic factors have been found to be largely responsible for co-morbidity patterns among common psychiatric illnesses (Kendler et al. Reference Kendler, Prescott, Myers and Neale2003), our results may help inform future research seeking to identify shared and disorder-specific sources of genetic risk for uncommon mental disorders.
Acknowledgements
The authors thank Danica Dunbar and Kathryn Patterson for their assistance with manuscript editing. This research was supported by a grant from the National Institute of Mental Health (no. F31MH081345; principal investigator: K.F.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.
Declaration of Interest
None.
