Introduction
Cognitive deficits are well recognized in schizophrenia (Saykin et al. Reference Saykin, Gur, Gur, Mozley, Mozley, Resnick, Kester and Stafiniak1991; Harvey & Keefe, Reference Harvey and Keefe1997; Heinrichs & Zakzanis, Reference Heinrichs and Zakzanis1998; Aleman et al. Reference Aleman, Hijman, de Haan and Kahn1999; Bokat & Goldberg, Reference Bokat and Goldberg2003; Harvey et al. Reference Harvey, Green, McGurk and Meltzer2003, Reference Harvey, Siu and Romano2004; Henry & Crawford, Reference Henry and Crawford2005; Lee & Park, Reference Lee and Park2005; Keefe et al. Reference Keefe, Perkins, Gu, Zipursky, Christensen and Lieberman2006a , Reference Keefe, Poe, Walker and Harvey b ; Keshavan et al. Reference Keshavan, Tandon, Boutros and Nasrallah2008; Szöke et al. Reference Szöke, Trandafir, Dupont, Méary, Schürhoff and Leboyer2008; Mesholam-Gately et al. Reference Mesholam-Gately, Giuliano, Goff, Faraone and Seidman2009) and are hypothesized to be a more direct expression of underlying biological abnormalities than formal diagnosis (Harvey & Keefe, Reference Harvey and Keefe1997; Heinrichs & Zakzanis, Reference Heinrichs and Zakzanis1998). More refined neuropsychological measures could facilitate better understanding of the illness, with regards to how neurobiological susceptibility progresses into a disease phenotype (Gottesman & Gould, Reference Gottesman and Gould2003; Heinrichs, Reference Heinrichs2005; Cannon & Keller, Reference Cannon and Keller2006; Keshavan et al. Reference Keshavan, Tandon, Boutros and Nasrallah2008; Prasad & Keshavan, Reference Prasad and Keshavan2008).
Factor analytic approaches (Holdnack et al. Reference Holdnack, Xiaobin, Larrabee, Millis and Salthouse2011) can be employed to determine the cognitive architecture of schizophrenia derived from neuropsychological tests (Wechsler, Reference Wechsler1945, Reference Wechsler1955; Keefe et al. Reference Keefe, Bilder, Harvey, Davis, Palmer, Gold, Meltzer, Green, Miller and Canive2006c ; Kern et al. Reference Kern, Nuechterlein, Green, Baade, Fenton, Gold, Keefe, Mesholam-Gately, Mintz, Seidman, Stover and Marder2008; Nuechterlein et al. Reference Nuechterlein, Green, Kern, Baade, Barch, Cohen, Essock, Fenton, Frese, Gold, Goldberg, Heaton, Keefe, Kraemer, Mesholam-Gately, Seidman, Stover, Weinberger, Young, Zalcman and Marder2008). Speed of processing, attention/vigilance, working memory, verbal learning and memory, visual learning and memory and reasoning/problem solving have been identified via factorial approaches as areas of abnormality in schizophrenia (Nuechterlein et al. Reference Nuechterlein, Green, Kern, Baade, Barch, Cohen, Essock, Fenton, Frese, Gold, Goldberg, Heaton, Keefe, Kraemer, Mesholam-Gately, Seidman, Stover, Weinberger, Young, Zalcman and Marder2008). A review of several confirmatory factor analysis (CFA) studies highlighted several intriguing trends (online Supplementary Table S1):
-
(1) Subtle differences in cognitive architecture have been reported; separation of extracted factors may be biased by test-specific features (e.g. tests that measure reaction time may form strong covariance with other tests that are reaction time based; see Podsakoff et al. Reference Podsakoff, MacKenzie, Lee and Podsakoff2003), leading to factor structures that may not entirely reflect cognitive substructure.
-
(2) The correlated and hierarchical factor models have been commonly tested. The former, a first-order CFA model, indicates associated but separate cognitive domains; the latter, a second-order CFA model, assumes that a single general cognitive factor g subserves all cognitive domains. The use of either model influences interpretation of the derived cognitive architecture.
-
(3) Most studies suggest that cognitive substructures in schizophrenia and control samples (mostly healthy participants) are qualitatively homologous (see online Supplementary Table S1). Only one study in the review (Dickinson et al. Reference Dickinson, Ragland, Calkins, Gold and Gur2006) reported non-invariance in the cognitive structure of healthy controls and schizophrenia. Further investigation is necessary. Homology in cognitive structures appears to run contrary to the broad and profound cognitive impairments observed in schizophrenia.
The present study aims to: (i) define the latent cognitive architecture in schizophrenia cases and healthy controls; (ii) examine covariances across neuropsychological tests, to broadly compare latent factors against published evidence; (iii) refine and reduce the latent architecture to fit the data without methodological variance; and (iv) establish a final model that best discriminates cases and controls. The secondary objectives of the current study are: (i) establish agreement between factor analytic g and conventional methods of g calculation (or composite g) – averaging standardized test scores with pooled standard deviations; (ii) establish discriminability of factor analysis-derived g and composite g; and (iii) establish convergent validity between g estimands and education as a candidate reference.
Method
Participants
A total of 1012 healthy participants (controls) and 707 schizophrenia cases were recruited as part of the Singapore Translational and Clinical Research in Psychosis (project title: Elucidating the Genetic Architecture of Neurocognitive Endophenotypes in Schizophrenia; grant no. NMRC/TCR/003/2008). Healthy controls were recruited from the community while schizophrenia cases were recruited from rehabilitation centers, community care centers across the country, out-patient clinics and in-patient wards, under purview of the Institute of Mental Health, Singapore. Data collection was completed in approximately 3 years. Inclusion criteria were: Chinese ethnicity, to ensure a genetically homogeneous sample; and completion of a minimum of 6 years of primary school education. Additional exclusion criteria precluded all participants with significant history of substance abuse, clinically significant neurological disease or injury, color blindness, and healthy participants with first-degree relatives suffering from schizophrenia or other psychotic disorders. Schizophrenia cases fulfilled Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) diagnostic criteria for schizophrenia based on the Structured Clinical Interview for DSM-IV Axis I disorders (First et al. Reference First, Spitzer, Gibbon and Williams2002). All participants consented to participate in research procedures prior to data collection. Consent procedures adhered to the guidelines specified by the National Healthcare Group Domain Specific Review Board's (domain A, NHG DSRB) requirements for human subject research.
Procedures
Neuropsychological and clinical evaluations were carried out by psychometricians (trained by R.S.E.K., S.L.C., M.K. and A.R.). Wechsler Abbreviated Scale of Intelligence (WASI) matrix reasoning (Wechsler, Reference Wechsler1999), Continuous Performance Tests-Identical Pairs (CPT-IP; Cornblatt et al. Reference Cornblatt, Risch, Faris, Friedman and Erlenmeyer-Kimling1988), Wisconsin Card Sorting Test, 64-card version (WCST-64; Heaton, Reference Heaton1993), Benton Judgment of Line Orientation Test (Benton et al. Reference Benton, Sivan, Hamsher, Varney and Spreen1994) and the Brief Assessment of Cognition in Schizophrenia (BACS; Keefe et al. Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour2004, Reference Keefe, Harvey, Goldberg, Gold, Walker, Kennel and Hawkins2008), consisting of verbal memory, digit sequencing, token motor task, semantic fluency, symbol coding and Tower of London, were administered to all participants. A total of 32 subtests were obtained from the current battery of 10 neuropsychological tests (e.g. within sematic fluency, there were three subtests – animals, fruits and vegetables; see Fig. 1 for list of all subtests). Data including age, gender, education, duration of illness, medications and Positive and Negative Syndrome Scale ratings were also collected.
Fig. 1. Cognitive profiles of schizophrenia subjects standardized against healthy controls. Adjusted scores were adjusted for age and gender; Blom inverse rank transformation was applied to normalize scores. Values are Z-scores, with standard errors represented by vertical bars. BACS, Brief Assessment of Cognition in Schizophrenia; WASI, Wechsler Abbreviated Scale of Intelligence; CPT-IP, Continuous Performance Tests-Identical Pairs; WCST-64, Wisconsin Card Sorting Test, 64-card version.
Data analysis
Data preparation and analysis software
All neuropsychological subtests were corrected for age and gender, normalized via Blom inverse rank transformation (Blom, Reference Blom1958) and standardized against healthy controls. IBM SPSS version 20 (IBM, USA) and AMOS™ version 18 (Amos Development Corp., USA; Albright & Hun, Reference Albright and Hun2009) were used for analyses (see online Supplementary Fig. S5 for analysis flowchart).
Examining covariance across test performances
Data-driven EFA procedures were employed to examine patterns of covariances across neuropsychological test performances. This was to investigate if neuropsychological tests were indeed approximating known cognitive domains. No assumptions were made regarding the covariance structure of neuropsychological tests. All subtests from the neuropsychological battery were entered in an EFA model, using principal components extraction and varimax rotation. Subtests with the highest factor loadings and communalities × factor loading indices were selected for further CFA.
Addressing methodological variances in factor analysis
Methodological variances have been flagged as a challenge in the schizophrenia literature to factor analysis of neuropsychological tests (Podsakoff et al. Reference Podsakoff, MacKenzie, Lee and Podsakoff2003; Genderson et al. Reference Genderson, Dickinson, Diaz-Asper, Egan, Weinberger and Goldberg2007; Dickinson & Gold, Reference Dickinson and Gold2008; Dickinson & Harvey, Reference Dickinson and Harvey2009). Within the 32 neuropsychological subtests, 12 were CPT-IP subtests, three were semantic fluency subtests, and five were WCST-64 subtests (see online Supplementary Table S2 for a list of subtests). These subtests from the same tests are expected to demonstrate strong covariance. In a separate analysis, each block of subtests from CPT-IP, WCST-64 and semantic fluency were reduced via principal component analysis (PCA). Derived regression factors scores were then used for subsequent EFAs and CFAs.
Evaluation of model types
Correlated and hierarchical models were tested in cases and controls, respectively. Subtests that were identified in earlier EFA were entered in the models. Both unrefined and refined models were tested.
Model fitting for derived factors
CFA model fit was evaluated. CFA fit indices included the normed fit index (NFI; Bentler & Bonnett, Reference Bentler and Bonett1980), relative fit index (RFI; Bollen, Reference Bollen1986), incremental fit index (Bollen, Reference Bollen1989), non-normed fit index/Tucker–Lewis index (Bollen, Reference Bollen1989), comparative fit index (Bentler, Reference Bentler1990) and root mean square of approximation (RMSEA; Browne & Cudeck, Reference Browne, Cudeck, Bollen and Long1993). A good model fit is indicated by a RMSEA <0.05 and >0.9 for the other indices (Bentler & Bonnett, Reference Bentler and Bonett1980; Bollen, Reference Bollen1986, Reference Bollen1989; Bentler, Reference Bentler1990; Browne & Cudeck, Reference Browne, Cudeck, Bollen and Long1993). CFA factor scores were generated via full information maximum likelihood (FIML) imputations for subsequent investigations. CFA factor scores were generated via FIML imputations for subsequent investigations.
Evaluation of model discrimination of case–control status
True factor scores for latent models were entered as predictors in a logistic regression model. Case–control status was entered as a dependent variable. Discriminability of factor analytic estimation of g and composite g was also evaluated.
Agreement of composite g and factor-derived g
Use of g composites from test batteries as a cognitive phenotype has been previously discussed in the literature (Dickinson et al. Reference Dickinson, Schaefer, Weinberger and Harvey2013; Donohoe et al. Reference Donohoe, Deary, Glahn, Malhotra and Burdick2013). The conventional method of deriving g or general cognitive composite from a neuropsychological battery has been to average tests scores by pooled standard deviations. The sum scores approach is putatively desirable when computation is considered exploratory, wherein each item is weighted equally (DiStefano et al. Reference DiStefano, Min and Mîndrilă2009; Hair & Anderson, Reference Hair and Anderson2010), whereas factor analytic-derived g takes into account factor weights and correlations specific to the sample with which the analysis is performed (DiStefano et al. Reference DiStefano, Min and Mîndrilă2009). However, it is not known if factor-derived g and conventional g calculation approaches are equivalent. Post-hoc Bland–Altman analysis (Bland & Altman, Reference Bland and Altman1986) was performed to examine level of agreements between factor analysis-derived latent factors and composite g.
Evaluation of convergent validity of factor-derived g
Convergent validity of factor-derived g and composite g was examined alongside education attainment. Education is a candidate measure for establishing convergent validity for g due to previously known associations with cognitive performances. Bivariate correlation was performed for education attainment and factor analysis-derived g and composite score-derived g. Education attainment was estimated by adjusted years depending on the stage of education that the participants were in (for more in-depth discussion, see Lam et al. Reference Lam, Eng, Rapisarda, Subramaniam, Kraus, Keefe and Collinson2012).
Results
Sample description
Sample characteristics are reported in Table 1. Scaled and adjusted neuropsychological profiles of schizophrenia cases are presented in Fig. 1. Cognitive scores in schizophrenia cases ranged between 1 and 2 standard deviations from controls.
Table 1. Sample demographic characteristics
Data are given as mean (standard deviation) unless otherwise indicated.
df, Degrees of freedom; PANSS, Positive and Negative Syndrome Scale.
Examining covariance across test performance
BACS, CPT-IP and WCST-64 subtests, WASI matrix reasoning and Benton Judgment of Line Orientation Test, 32 subtests in total, were subjected to exploratory factor analysis (EFA). Subtests with factor communalities <0.4 and equal to 1 were removed. These may reflect low reliability or collinearity. This resulted in the exclusion of BACS token motor task, WCST-64 correct responses, BACS semantic fluency (total), CPT-IP average hits, CPT-IP average d′, CPT-IP average random errors and CPT-IP average false alarms. EFA was repeated on the remaining 24 items. The CPT-IP two-digit subtest was excluded from subsequent analysis as it was deemed non-specific to factor solutions (see online Supplementary Table S2).
Six factors were extracted from EFA in both samples. Subtests were selected based on breakpoints on factor loadings, factor loadings × communalities indices and judgment of item relevance for each modeled factor for subsequent CFA model fitting (online Supplementary Table S2 and Fig. S1). Subtests selected were: (i) factor 1: CPT-IP three- and four-digit hits; (ii) factor 2: WASI matrix reasoning and BACS Tower of London; (iii) factor 3: WCST-64 perseverative errors, completed categories and first category scores; (iv) factor 4: BACS semantic fluency, animals, fruits and vegetables; (v) factor 5: CPT-IP three- and four-digit random errors; and (vi) factor 6: CPT-IP three- and four-digit false alarms.
Addressing methodological variances in factor analysis
BACS semantic fluency animals, fruits and vegetables; WCST-64 perseverative errors, completed categories, final category responses, first category responses, correct responses; and CPT-IP two-, three-, four-digits, and average hits, false alarms, random errors and d′ were subjected to reduction procedures for each set of subtests. PCA extraction was employed to establish component scores for the entire case–control sample. Four reduced components were obtained for CPT-IP, and one reduced component each for BACS semantic fluency and WCST-64 subtests. The first CPT-IP reduced component corresponded to CPT-IP hits and d′, the second CPT-IP reduced component corresponded to CPT-IP random errors, and the third and fourth CPT-IP reduced components corresponded to three- and four-digit false alarms and two-digit false alarms, respectively (online Supplementary Table S3).
Five subtests from the BACS battery (verbal memory, digit sequencing, token motor task, symbol coding, and Tower of London), the Benton Judgment of Line Orientation Test and WASI matrix reasoning, four reduced CPT-IP factors, and one reduced factor each from the BACS semantic fluency tests, and the WCST-64 were entered into separate case–control EFA models. Four- and three-factor solutions were obtained in healthy controls and schizophrenia cases, respectively (online Supplementary Table S4). Patterns of item loadings suggested that factor loadings differed in schizophrenia. Most cognitive subtests loaded on one single factor in schizophrenia cases, indicating high congruity of cognitive performances across tests. Due to interpretation challenges in attempting to further deconstruct a single-factor solution in schizophrenia, EFA solutions obtained from controls were referenced for test selection in subsequent CFA modeling in schizophrenia cases as well. Using similar methods as in earlier EFA procedures, the following items were selected for subsequent CFA: (i) solution 1: Benton Judgment of Line Orientation Test and WASI matrix reasoning; (ii) solution 2: BAC semantic fluency – reduced component and BACS verbal memory; (iii) solution 3: CPT-IP three- and four-digit hits – reduced component and CPT-IP three- and four-digit false alarms – reduced component; (iv) solution 4: BACS symbol coding and BACS token motor task (online Supplementary Table S4 and Fig. S2). The term ‘solution’ is used for disambiguating ‘factors’ from the six-factor EFA.
Evaluation of model types
Derived from previous EFA procedures, six, four and three correlated and hierarchical CFA factor models were built. Models were applied to both schizophrenia and healthy controls separately; results of CFA are reported in Tables 2 and 3.
Table 2. Model fitting of six-, four- and three-factor CFA models
CFA, Confirmatory factor analysis; HC, healthy controls; CFM, correlated factor model; SCZ, schizophrenia cases; HFM, hierarchical factor model; CPT-IP, Continuous Performance Tests-Identical Pairs; BACS, Brief Assessment of Cognition in Schizophrenia; WASI, Wechsler Abbreviated Scale of Intelligence; WCST-64, Wisconsin Card Sorting Test, 64-card version.
a Earlier factor analysis indicated Tower of London and WCST-64 loaded on the same factor. These were removed in favour of selecting higher loading tests. Nevertheless, this suggested that that was significant overlapping variances. Hence the label was retained as executive function.
Table 3. Fit indices for CFA model fitting
CFA, Confirmatory factor analysis; NFI, normed fit index; RFI, relative fit index; IFI, incremental fit index; TLI, Tucker–Lewis index; CFI, comparative fit index; RMSEA, root mean square of approximation; CI, confidence interval; HC, healthy controls; SCZ, schizophrenia cases.
CFA model fitting for derived factors
Six-factor correlated and hierarchical models appeared to fit both case and control data at a reasonable level. The four-factor correlated model was found to fit cognitive data in the healthy controls, but did not converge in schizophrenia. The structural model for schizophrenia was minimally re-specified; the CPT-IP false alarm reduced component was removed, while the CPT-IP hits reduced component was modeled with the BACS token motor task and BACS symbol coding, obtaining a three-factor correlated model for schizophrenia. After re-specification, the three correlated and hierarchical factor models were found to fit cognitive data in schizophrenia. To facilitate subsequent sample comparisons, both three-factor models were tested on the entire sample of cases and controls. Good model fit was found when the case models were applied to all the subjects (see model fit indices; Table 3). Future work is required to validate the executive function latent factor conceptualized by matrix reasoning and line orientation. These tasks are not typically utilized for assessment of executive function. However, the nomenclature for the latent variable was assigned as such due to the initial loading of these tasks with the WCST-64 and Tower of London, suggesting that variances in these tasks were not separable from executive function. Matrix reasoning and line orientation appeared to have higher loadings in the initial EFA and thus were selected to be indicators of the latent variable executive function.
Evaluation of model discrimination of case–control status
Negative predictive values (NPV) and positive predictive values (PPV) were reported for each model tested. Three-factor models (three correlated factors: NPV: 89.1%; PPV: 80.6%; three hierarchical factors: NPV: 89.1%; PPV: 80.6%) appeared to discriminate cases and controls equally well. However six-factor models were poorer in classifying cases and controls (six correlated factors: NPV: 88.4%; PPV: 73.6%; six hierarchical factors: NPV: 86.6%; PPV: 70.9%). The g estimate from the six-factor model appeared to be the poorest classifier (six hierarchical factor g: NPV: 83.3%; PPV: 64.5%; composite g: NPV: 87.2%; PPV: 72.6%; three hierarchical factor g: NPV: 88.7%; PPV: 76.5%). To further evaluate three-factor models, forward stepwise logistic regression was conducted. Executive function [odds ratio (OR) = 3.13, p = 1.0 × 10−5, 95% confidence interval (CI) = 1.89–5.18] and speed/vigilance (OR = 0.002, p = 1.93 × 10−69, 95% CI = 0.001–0.004) remained significant predictors of case–control status (online Supplementary Fig. S3).
Agreement of composite g and factor-derived g
To clarify the utility of g estimands, post-hoc Bland–Altman (Bland & Altman, Reference Bland and Altman1986) evaluation of g estimands from the six- and three-factor hierarchical models and composite g were tested (online Supplementary Fig. S4). Bland–Altman plots revealed that though composite scores showed tighter associations with the three-factor g and greater agreement compared with the six-factor g, there appeared to be systematic differences between the composite score- and factor-derived g.
Evaluation of convergent validity of factor-derived g
Bivariate correlations revealed moderate education correlations with g in both cases [r (three-factor g) = 0.528; r (six-factor g) = 0.413; r (composite g) = 0.489] and controls [r (three-factor g) = 0.328; r (six-factor g) = 0.202; r (composite g) = 0.284].
Discussion
The current study comprises one of the largest single-site schizophrenia samples cognitively profiled. The primary objectives were to elucidate the latent cognitive structure that underlies neuropsychological performance in schizophrenia, establish if cognitive structures were similar between schizophrenia and healthy samples, and to examine the discriminant properties of various latent cognitive factors. Secondary objectives were to examine properties of factor analytic-derived g and conventionally derived g composite scores from a neuropsychological battery – to establish a viable cognitive phenotype that is adequately robust for subsequent studies (Dickinson et al. Reference Dickinson, Schaefer, Weinberger and Harvey2013; Donohoe et al. Reference Donohoe, Deary, Glahn, Malhotra and Burdick2013).
Sample description
Case–control differences across cognitive tests up to two standard deviations below the mean performances of healthy controls were similar to those in previously published work (Saykin et al. Reference Saykin, Gur, Gur, Mozley, Mozley, Resnick, Kester and Stafiniak1991; Harvey & Keefe, Reference Harvey and Keefe1997; Heinrichs & Zakzanis, Reference Heinrichs and Zakzanis1998; Aleman et al. Reference Aleman, Hijman, de Haan and Kahn1999; Bokat & Goldberg, Reference Bokat and Goldberg2003; Harvey et al. Reference Harvey, Green, McGurk and Meltzer2003, Reference Harvey, Siu and Romano2004; Henry & Crawford, Reference Henry and Crawford2005; Lee & Park, Reference Lee and Park2005; Keefe et al. Reference Keefe, Perkins, Gu, Zipursky, Christensen and Lieberman2006a , Reference Keefe, Poe, Walker and Harvey b ; Keshavan et al. Reference Keshavan, Tandon, Boutros and Nasrallah2008; Szöke et al. Reference Szöke, Trandafir, Dupont, Méary, Schürhoff and Leboyer2008; Mesholam-Gately et al. Reference Mesholam-Gately, Giuliano, Goff, Faraone and Seidman2009).
Examining covariance across test performance
Six factors were obtained in preliminary EFA in cases and controls, respectively, which appear to correspond to test-specific factors: F1 (CPT-IP hits); F2 (executive function/spatial reasoning); F3 (WCST-64); F4 (semantic fluency); F5 (CPT-IP commission errors); and F6 (CPT-IP false alarms). Initial factors appeared to be broadly consistent with previously reported cognitive factors in the literature (online Supplementary Table S1). However, methodological variance may distort the initial cognitive architecture. Data-driven EFA methods extracted broadly similar cognitive domains as in previous reports (Genderson et al. Reference Genderson, Dickinson, Diaz-Asper, Egan, Weinberger and Goldberg2007; Wang et al. Reference Wang, Vassos, Deng, Ma, Hu, Murray, Collier and Li2010). However, a model driven by the method-specific variances might not be useful in detecting neurobiological abnormalities or intervention effects.
Addressing methodological variance
Further refinement of neuropsychological test measures resulted in derivation of four- and three-factor models in controls and schizophrenia subjects, respectively. The three-factor model corresponded to S1 (executive function), S2 (fluency/memory) and S4 (speed/vigilance). While speed and vigilance did not appear separable in schizophrenia subjects, they did in healthy individuals. Factors appeared to represent commonly accepted domains of cognition and associated test modalities (see online Supplementary Table S1; Fioravanti et al. Reference Fioravanti, Carlone, Vitale, Cinti and Clare2005; Dickinson & Gold, Reference Dickinson and Gold2008).
CFA model fitting for derived factors
Nine models were tested. Fit indices across models were reasonable. Models that were reduced and refined were comparable with larger, more complex models that included method variances. Results may suggest several cognitive substrata of neuropsychological architecture, such that the initial extracted architecture could represent how test batteries are organized and administered, or method variance. Further refinement of the factor structure via CFA uncovered a simpler but more parsimonious architecture that represents underlying cognitive processes responsible for test performance.
While fit indices were generally acceptable for the nine models that were tested, there appears to be a trend that first-order correlated factors fit better than hierarchical models estimating latent g. An interpretation of this phenomenon may be related to first-order correlated models being more flexible in accounting for variances in the data, while hierarchical models tend to constrain the estimation of a second-order latent factor g. However, further exploration of the data is necessary to investigate if factor structures are similar after adjusting for g, and if g plays a direct role in test performance that is not otherwise mediated by latent cognitive domains (Gignac, Reference Gignac2008).
Cognitive factor analytic studies seek to identify separable cognitive domains subserved by specific neural processes. Factors derived from traditional neuropsychological tests are posited to be sufficiently independent to permit assay of discrete neural systems (Egan et al. Reference Egan, Goldberg, Gscheidle, Weirich, Rawlings, Hyde, Bigelow and Weinberger2001). On the other hand, the multidimensional nature of neuropsychological tests suggests caution when interpreting their associations with specific neural substrates (Keefe, Reference Keefe1995; MacDonald & Carter, Reference MacDonald and Carter2002). These differing viewpoints were carefully considered in the course of the current study. We demonstrated evidence of subtle discordance in cognitive architecture between schizophrenia subjects and healthy controls. Though subsequent CFA model re-specification of the three-factor model was less restrictive and was able to fit the full sample, differences observed in cognitive architectures between cases and controls raises the question of whether a fully dimensional cognitive approach is sufficiently descriptive, or a mixture model approach (e.g. McLachlan & Peel, Reference McLachlan and Peel2004) of specific cognitive subtypes with dimensional severity of deficits could further illuminate cognitive processes yet uncovered in schizophrenia. Though Dickinson & Gold (Reference Dickinson and Gold2008) pointed out that inter-correlations amongst cognitive domains are high and may reduce orthogonality, the resultant cognitive architecture may represent the subtle facets of cognition that may nevertheless be valuable to future biological research in schizophrenia.
Evaluation of model discrimination of case–control status
The reduced and refined three-factor model best discriminated cases and controls. Results support the notion that the factor structure more proximal to cognitive domains is probably more sensitive in separating cases and controls.
Follow-up logistic regression indicated that executive function and speed/vigilance demonstrated superior discriminant properties in our sample. Executive function deficits are among the most marked in schizophrenia (Fioravanti et al. Reference Fioravanti, Carlone, Vitale, Cinti and Clare2005; Snitz et al. Reference Snitz, MacDonald and Carter2006; Reichenberg & Harvey, Reference Reichenberg and Harvey2007) and are observed in unmedicated or first-episode schizophrenia (Ho et al. Reference Ho, Alicata, Ward, Moser, O'Leary, Arndt and Andreasen2003; Daban et al. Reference Daban, Amado, Bourdel, Loo, Olié, Poirier and Krebs2005; Reilly et al. Reference Reilly, Harris, Khine, Keshavan and Sweeney2008; Mesholam-Gately et al. Reference Mesholam-Gately, Giuliano, Goff, Faraone and Seidman2009), before illness onset (Lencz et al. Reference Lencz, Smith, McLaughlin, Auther, Nakayama, Hovey and Cornblatt2006), stabilized patients (Townsend et al. Reference Townsend, Malla and Norman2001) and unaffected relatives (Kuha et al. Reference Kuha, Tuulio-Henriksson, Eerola, Perälä, Suvisaari, Partonen and Lönnqvist2007; Birkett et al. Reference Birkett, Sigmundsson, Sharma, Toulopoulou, Griffiths, Reveley and Murray2008; Breton et al. Reference Breton, Planté, Legauffre, Morel, Adès, Gorwood, Ramoz and Dubertret2011). Executive function was found to be a candidate endophenotype in Han Chinese schizophrenia subjects (Hu et al. Reference Hu, Chen, Li, Zheng, Wang, Guo, Wu and Zhao2011). However, while pending further replication of the present results, reasonable consideration has to be given to the heterogeneous definitions of executive function (Raffard & Bayard, Reference Raffard and Bayard2012).
Speed of processing and vigilance were reported as separable factors in previous literature; this was not supported by our data (the four-factor CFA model did not converge in schizophrenia subjects; see online Supplementary Table S1; Nuechterlein et al. Reference Nuechterlein, Barch, Gold, Goldberg, Green and Heaton2004). We argue that speed of processing is a necessary aspect of the vigilance task. Early work in vigilance supports the view that processing speed is a major dimension that is part of the vigilance taxonomy (Parasuraman & Davis, Reference Parasuraman, Davies and Mackie1977; Fisk & Schneider, Reference Fisk and Schneider1981). Speed of processing had been postulated to be among the most impaired cognitive domain in schizophrenia, and to mediate and account for considerable/sizeable variance in disturbances in other cognitive domains (Rodríguez-Sánchez et al. Reference Rodríguez-Sánchez, Crespo-Facorro, González-Blanch, Perez-Iglesias and Vázquez-Barquero2007; Knowles et al. Reference Knowles, Weiser, David, Dickinson, Glahn, Gold, Davidson and Reichenberg2012; Ojeda et al. Reference Ojeda, Peña, Schretlen, Sánchez, Aretouli, Elizagárate, Ezcurra and Gutiérrez2012). This cognitive domain had been associated with illness risk (Niendam et al. Reference Niendam, Bearden, Rosso, Sanchez, Hadley, Nuechterlein and Cannon2003; Keefe et al. Reference Keefe, Perkins, Gu, Zipursky, Christensen and Lieberman2006a , Reference Keefe, Poe, Walker and Harvey b ; Glahn et al. Reference Glahn, Almasy, Blangero, Burk, Estrada, Peralta, Meyenberg, Castro, Barrett, Nicolini, Raventós and Escamilla2007; Reichenberg et al. Reference Reichenberg, Caspi, Harrington, Houts, Keefe, Murray, Poulton and Moffitt2010), illness severity (Dickinson et al. Reference Dickinson, Ramsey and Gold2007), functional disability (Milev et al. Reference Milev, Ho, Arndt and Andreasen2005; Brekke et al. Reference Brekke, Hoe, Long and Green2007; Bowie et al. Reference Bowie, Leung, Reichenberg, McClure, Patterson, Heaton and Harvey2008; Ojeda et al. Reference Ojeda, Pena, Sánchez, Elizagárate and Ezcurra2008; Harvey et al. Reference Harvey, Keefe, Patterson, Heaton and Bowie2009) and articulated as a candidate endophenotype in schizophrenia (Appels et al. Reference Appels, Sitskoorn, Westers, Lems and Kahn2003; Wang et al. Reference Wang, Chan, Sun, Yao, Deng, Sun, Liu, Sham, Ma, Meng, Murray, Collier and Li2007; Glahn et al. Reference Glahn, Almasy, Blangero, Burk, Estrada, Peralta, Meyenberg, Castro, Barrett, Nicolini, Raventós and Escamilla2007; Wang et al. Reference Wang, Vassos, Deng, Ma, Hu, Murray, Collier and Li2010). The emergence of the speed/vigilance factor, being separable in healthy controls but not in schizophrenia cases, suggests that it may be a promising proximal candidate for research on treatment outcome and neurobiological factors.
An evaluation of g estimation
Calculating g is a valuable approach due to the phenomenon of ‘positive manifold’ (Carroll, Reference Carroll1993; Jensen, Reference Jensen1998), its widespread practicality, associations with biological variables (Jensen, Reference Jensen1992, Reference Jensen1998, Reference Jensen, Sternberg and Grigorenko2002) and its genetic contributions (e.g. Davies et al. Reference Davies, Tenesa, Payton, Yang, Harris, Liewald, Ke, Le Hellard, Christoforou, Luciano, McGhee, Lopez, Gow, Corley, Redmond, Fox, Haggarty, Whalley, McNeill, Goddard, Espeseth, Lundervold, Reinvang, Pickles, Steen, Ollier, Porteous, Horan, Starr, Pendleton, Visscher and Deary2011). The theory of indifference suggests, given the administration of a sufficiently large and diverse selection of neuropsychological instruments, that the composite cognitive scores of any given battery are likely to be similar to that of any other battery (Jensen, Reference Jensen1998; Johnson et al. Reference Johnson, Bouchard, Krueger, McGue and Gottesman2004, Reference Johnson, te Nijenhuis and Bouchard2008; Hunt, Reference Hunt2011; Mackintosh, Reference Mackintosh2011; Deary, Reference Deary2012). This encourages the use of g as a candidate phenotype in inter-center replication studies (Loo et al. Reference Loo, Shtir, Doyle, Mick, McGough, McCracken, Biederman, Smalley, Cantor, Faraone and Nelson2012).
Here, we found differences between conventional composite score-type calculation of g versus factor analytic-derived g. The factor analytic estimation may not have been optimal in this study because of the inherent limitations of using a second-order factor where estimation becomes largely dependent on the first-order loadings (for an in-depth review, see Gignac, Reference Gignac2008). It is also possible that measures originally validated in Caucasian samples (Keefe et al. Reference Keefe, Goldberg, Harvey, Gold, Poe and Coughenour2004) may not capture latent factors specific of an Asian sample, hence there may be subtle differences in the estimation of true factor scores. The use of a second-order hierarchical factor could therefore have amplified these differences. Perhaps a much larger and diverse battery would have better stabilized the hierarchical model in our sample. For this reason, either a conservative approach of calculating composite scores or alternative factor analytic approaches for g estimation in Asian samples may be required (Gignac, Reference Gignac2006, Reference Gignac2008).
Two aspects of g estimation should be considered in subsequent studies. First, although discriminant properties of both methods of calculating general cognition appear comparable, differences at the level of score distribution are probably related to score calculation methods. Factor analytic g appeared slightly more sensitive to associations with education. However, further evaluation of sensitivity of either index is necessary (e.g. in genetic association studies where covariances with DNA polymorphism of either measure can be thoroughly evaluated). Second, in our review, the number of impaired cognitive domains ranged from six to 22 factors, which may be limited by heterogeneous sample characteristics and methodologies (Wilk et al. Reference Wilk, Gold, Humber, Dickerson, Fenton and Buchanan2004; Foriavantti et al. Reference Fioravanti, Carlone, Vitale, Cinti and Clare2005), statistical inadequacies in addressing latent substructure (Genderson et al. Reference Genderson, Dickinson, Diaz-Asper, Egan, Weinberger and Goldberg2007) and methodological variances in test administration (Nuechterlein et al. Reference Nuechterlein, Barch, Gold, Goldberg, Green and Heaton2004; Kraus & Keefe, Reference Kraus and Keefe2007). The potential presence of several cognitive architectures within a battery of tests may suggest possibilities where the diverse factor structures previously reported in the literature can be reconciled in subsequent studies. In this context, the calculation of composite g may still be required in cross-center collaborations.
Limitations
The current study benefits from its large sample sizes of controls and patients. Our study is not based on an exhaustive battery of tests covering the entirety of cognitive aspects classically assessed in patients, as decisions for test inclusion had to balance comprehensiveness of the assessment with its practicality and tolerability. Also, although medication type was recorded, its effects were not explicitly tested as part of the factor model, as the complex task of reviewing lifetime case records is disproportionate with respect to the scope of the study.
Conclusions
As the field moves towards a more dimensional approach of understanding complex psychiatric illness, measurement of cognition and refinement of measures will continue to be important in clinical practice and research (Collinson et al. Reference Collinson, Lam and Hayes2010). Separable cognitive factors were identified in the current study that may be valuable in capturing subtle aspects of cognitive processes. Cognitive and neuropsychological researchers familiar with the inherent strengths and weakness of neuropsychological testing are in the position to further develop innovative strategies, refine neuropsychological procedures, and maximize what tests can reveal about the complex nature of cognitive deficits and their underlying neural substrates in schizophrenia.
Supplementary material
For supplementary material accompanying this paper visit http://dx.doi.org/10.1017/S0033291714001020
Acknowledgements
The Singapore Translational and Clinical Research in Psychosis is supported by the National Research Foundation Singapore under the National Medical Research Council Translational and Clinical Research Flagship Programme (grant no. NMRC/TCR/003/2008). M.L. is supported by the National Medical Research Council Training Fellowship (grant no. MH095: 003/008-1014) during the preparation of the current paper.
S.-A.C. has full access to all data generated by the consortium and is responsible for the integrity and accuracy of data reported. The authors also acknowledge Dr Mythily Subramaniam and Dwight Dickinson for their advice and inputs to the manuscript and analysis of the current study.
Declaration of Interest
R.S.E.K. currently or in the past 24 months has received investigator-initiated research funding support from the Department of Veterans Affairs, Feinstein Institute for Medical Research, GlaxoSmithKline, National Institute of Mental Health, Novartis, Psychogenics, Research Foundation for Mental Hygiene, Inc. and the Singapore National Medical Research Council. R.S.E.K. currently or during the past 24 months has received honoraria, or served as a consultant or advisory board member for Abbvie, Akebia, Amgen, Astellas, Asubio, BiolineRx, Biomarin, Boehringer-Ingelheim, Bristol-Myers Squibb, Eli Lilly, EnVivo, Helicon, Lundbeck, Merck, Mitisubishi, Otsuka, Pfizer, Roche, Shire, Sunovion, Takeda and Targacept. R.S.E.K. receives royalties from the BACS testing battery and the MATRICS Battery (BACS Symbol Coding). He is also a shareholder in NeuroCog Trials, Inc., Durham NC. The other authors have no conflicts of interest to declare.
