INTRODUCTION
The past decade has witnessed a veritable explosion of forensic neuropsychology research, and the majority of this literature has addressed response validity assessment and malingering (Sweet et al., 2002). Response validity studies in neuropsychology follow two main streams: psychological and cognitive. Within the psychological realm, the Minnesota Multiphasic Personality Inventory, Second Edition (MMPI-2; Butcher et al., 1989) continues to be the most thoroughly researched personality inventory in forensic settings. This popularity is largely owing to the long tradition of success that the original MMPI-2 validity scales (e.g., L, F, K) have demonstrated in detecting spurious psychological symptoms. “Post-release” MMPI-2 validity scales have also been developed to improve identification of less-than-forthright psychological presentations. These include scales such as the Superlative Scale (S; Butcher & Han, 1995), Infrequency Psychopathology Scale (Fp; Arbisi & Ben-Porath, 1995), Dissimulation Scale (Dsr2), Fake Bad Scale (FBS; Lees-Haley et al., 1991), Malingered Depression Scale (Md; Steffan et al., 2003), and Response Bias Scale (RBS; Gervais, 2005).
Within the cognitive realm, response validity assessment has evolved dramatically in recent years. Strategies of detecting insufficient effort include use of cutoffs derived from standard ability measures (e.g., digit span, finger tapping), examination of “nonsensical” or inconsistent performances, and comparison of test performances with real-life behaviors (Nies & Sweet, 1994; Sweet, 1999). Use of symptom validity tests (see Bianchini et al., 2001, for review) developed explicitly for effort assessment is common. Since the inception of forced-choice methodology (Pankratz, 1979, 1983), multiple forced-choice effort measures have shown particular promise in detecting insufficient effort. As Slick et al. (1999, p. 551) conclude, less-than-chance performance on forced-choice testing is the “closest to an evidentiary ‘gold standard’ for malingering” that neuropsychologists have at their disposal. In their meta-analysis of cognitive effort measures and indicators, Vickery et al. (2001) found a forced-choice measure, the Digit Memory Test (DMT; Hiscock & Hiscock, 1989), to be the most effective effort test, including non–forced-choice measures (Rey-15 Item Test, Lezak, 1995). The Victoria Symptom Validity Test (VSVT; Slick et al., 1995), Test of Memory Malingering (TOMM; Tombaugh, 1996), and Letter Memory Test (LMT; Inman et al., 1998) are other common forced-choice measures that have demonstrated effectiveness in detecting insufficient effort.
Despite progress in response validity research, relatively few studies have examined whether psychological and cognitive response validity measures represent overlapping or distinct constructs in forensic samples. Of the available studies that address this issue, consensus is lacking. Some studies raise the possibility of convergence between cognitive and psychological validity measures (Greve et al., 2006; Larrabee, 1998, 2003a,b; Nelson et al., 2006; Ross et al., 2004; Slick et al., 1996); others suggest that the measures may be relatively independent (Dearth et al., 2005; Greiffenstein et al., 1995, 2002; Larrabee, 2003c; Lees-Haley et al., 2002; Nelson & Sweet, in press).
In support of the notion that cognitive and psychological validity measures may be associated, Greve et al. (2006) determined classification accuracies of MMPI-2 validity scales in detecting cognitive malingering in a sizeable traumatic brain injury (TBI) sample. The TBI group was classified by degree of malingered neurocognitive dysfunction (MND; Slick et al., 1999). As participants demonstrated greater likelihoods of cognitive malingering, they were more likely to exhibit MMPI-2 validity scale elevations. For example, the mean F scale T score for a TBI group without an external incentive was 49.6; TBI groups classified as Suspect and Definite MND demonstrated mean F scales of 63.8 and 76.4, respectively. The mean FBS score for the no incentive group was 15.4, while the Probable and Definite MND groups demonstrated mean scores of 26.9 and 29.8, respectively. Similarly, Slick et al. (1996) reported significant correlations among VSVT indices and MMPI-2 validity scales in experimental malingerers, compensation-seeking patients, patients without external incentive, and controls. VSVT easy items correlated significantly with F-K, and difficult items correlated significantly with F and FBS. VSVT easy and difficult item latencies also correlated significantly with FBS. Clearly, these results suggest that individuals with increasing likelihoods of cognitive symptom exaggeration are more likely to show concurrent evidence of psychological symptom exaggeration.
There is also evidence that some MMPI-2 validity scales may be more strongly associated with cognitive effort measures than others (Larrabee, 2003a,b). Larrabee (2003a) observed correlational relationships of MMPI-2 validity scales and the Portland Digit Recognition Test (PDRT; Binder & Kelly, 1996), a forced-choice effort measure that is similar in form to the VSVT (i.e., visual digit recognition). In groups with either Definite MND (Slick et al., 1999) or moderate–severe closed head injury, the PDRT correlated minimally with F, Fb, and Fp. In contrast, the PDRT correlated significantly with FBS. Larrabee (2003b) also found FBS to be more sensitive to Probable MND (Slick et al., 1999) than other validity scales, including F, Fb, and Fp. These results converge with other studies, suggesting that FBS may have a differential relationship with cognitive effort relative to other validity scales (e.g., Greiffenstein et al., 2002; Ross et al., 2004). Indeed, in their meta-analysis of FBS, Nelson et al. (2006) found insufficient cognitive effort to be the largest moderating variable (d = 1.50) in overreporting versus comparison groups.
The literature reviewed until this point suggests some degree of overlap among cognitive and psychological response validity measures. However, there is also evidence that cognitive and psychological response validity measures may be relatively independent and have incremental worth in overreporting groups. Larrabee (2003c) examined FBS and PDRT relationships in Probable or Definite MND participants (Slick et al., 1999). In contrast to the findings of Larrabee (2003a,b), who reported significant relationships of FBS with the PDRT, Larrabee (2003c) found that the PDRT and Rey-15 Item Test (Lezak, 1995) correlated minimally with FBS.
Dearth et al. (2005) examined MMPI-2 and effort performances in cognitive simulators and controls. Of interest, they conducted a stepwise logistic regression with MMPI-2 validity scales (F, Fb, Fp, Dsr2, FBS) and forced-choice effort measures (TOMM, LMT, DMT) as predictors, and malingering status as the criterion. The F scale entered the solution first (79.3% classification), followed by LMT, for 89.7% total classification. No other predictors entered the solution. In other words, the F scale was a better predictor of dissimulation than any of the other variables observed. In a second logistic regression, effort measures were entered first as predictors with forward stepwise conditional entry. LMT entered the model first (86% classification rate), followed by the TOMM (approximately 90% classification), and then Dsr2, for overall classification of 93%. The authors interpreted these findings as suggesting that “at least one of the MMPI-2 validity indicators potentially provides an increment in predictive power for identifying malingering beyond that achieved by the motivational tests” (Dearth et al., 2005, p. 106).
In the only known factor analysis relevant to the current review, Greiffenstein et al. (1995) examined MMPI-2 scales and effort measures in TBI and postconcussive patients, and probable malingerers. Principal Components Analysis (PCA) yielded two distinct factors: one consistent with feigned emotional problems and the other with feigned neurobehavioral deficits. On the basis of the findings, the authors recommended caution in interpreting MMPI-2 elevations as evidence of neuropsychological malingering or vice versa: “Pathologically elevated F or F-K scales do not necessarily suggest malingering of neurobehavioral deficits, nor does the absence of pathological elevations suggest compliance on neuropsychological measures” (Greiffenstein et al., 1995, p. 236).
The Greiffenstein et al. (1995) findings are compelling. However, more recent studies (including the aforementioned) have not consistently supported the notion of distinct cognitive and psychological response validity constructs. Furthermore, limitations of the study by Greiffenstein et al. include a restricted number of validity scales (e.g., FBS and other post-release validity scales were unavailable), use of only one forced-choice effort measure (PDRT), and use of only one approach for data interpretation (PCA with Varimax rotation) without independent verification of the findings via other factor analytic methods.
In sum, the relationship of psychological versus cognitive symptom validity is obscured by inconsistent findings. Do these measures represent independent constructs, or are they are interrelated? Clinically, this is an important question because some have assumed that results of cognitive effort testing should predict invalid psychological test results, and vice versa. Apart from Greiffenstein et al., we are not aware of any studies that have addressed this question through factor analysis of cognitive effort measures and MMPI-2 validity scales. The present Exploratory Factor Analysis (EFA) examines common effort measures and MMPI-2 validity scales to better clarify the construct(s) of response validity. Whereas Confirmatory Factor Analysis entails “theory testing,” EFA entails “theory building” (Bryant & Yarnold, 1995). Given the lack of clarity in this area of response validity assessment, we reasoned that further “theory building” through EFA was warranted.
METHOD
Participants
Case files were obtained from the archival database of the fifth author in compliance with institutional guidelines. Participants were included if they had (1) been evaluated in a compensation-seeking context, including either personal injury litigation or worker's compensation; (2) completed the MMPI-2 in its entirety; and (3) completed the same three cognitive effort measures (i.e., VSVT, TOMM, and LMT) during a single neuropsychological evaluation. Criminal litigants were not included. Application of these criteria resulted in 122 compensation-seeking participants: 96 (78.7%) personal injury litigants and 26 (21.3%) workers' compensation claimants. Mean age of the sample was 41.0 (SD = 12.7) years with a mean education of 12.1 (SD = 3.1) years. The group consisted of 88 (72.1%) men and 34 (27.9%) women. Most participants reported persisting cognitive and/or emotional difficulties associated with TBI (73, 59.8%). For 33 (45.2%) of these TBI participants, head injuries were very mild [e.g., loss of consciousness (LOC) was either absent or lasted only seconds; participants were described as “alert” postinjury; participants were not admitted to the hospital or treated by emergency medical services]. Of the TBI participants with available Glasgow Coma Scale (GCS) data, 33 (45.2%) had normal–mild GCS scores (score = 13 to 15), four (5.5%) were moderate (score = 9 to 12), and three (4.1%) were severe (score = 3 to 8). Of the available LOC data for the TBI group, 20 participants (27.4%) had mild (<20 min) injuries, and 2 (2.7%) were within the moderate range. Other presenting conditions included toxic encephalopathy (17, 13.9%), diffuse cognitive and physical symptoms associated with motor vehicle accidents (14, 11.5%), electrical injury (6, 5.0%), stroke (2, 1.6%), seizure disorder (1, .8%), and myocardial infarction (1, .8%). Psychiatric claims included depression (3, 2.4%), mood disorders not otherwise specified (2, 1.6%), anxiety (1, .8%), posttraumatic stress (1, .8%), and panic disorder (1, .8%).
Measures
In compliance with the recommendation that the participant to variable ratio should be 5 or greater (Bryant & Yarnold, 1995), 12 variables were selected for the EFA. MMPI-2 validity scales included L, F, K, Fp, S, Dsr2, FBS, RBS, and Md. Cognitive effort measures included VSVT Total Correct, LMT % Correct, and TOMM Trial 2 Correct. Table 1 presents data on these 12 variables.
Means and standard deviations of 12 response validity variables
ANALYSES AND RESULTS
Overview
In analyzing the dimensionality or factor structure of variables through EFA, critical decisions are made concerning the number of factors extracted, factor rotation, and method of factor estimation (Bryant & Yarnold, 1995; Thompson, 2004). Effort measures and MMPI-2 scales were analyzed using the following: (a) two rules for identifying underlying factors—retaining factors with eigenvalues greater than 1.0, and a Scree Plot of the eigenvalues; (b) two methods of factor rotation—Varimax for independent factors, and Promax for correlated factors; and (c) two methods of factor estimation—PCA and Principal-Axis Factor Analysis (PAF).
Number of factors to retain.
One approach to factor extraction involved retaining only factors with eigenvalues greater than unity (Guttman, 1954). Sometimes called “Kaiser's stopping rule,” this method can overestimate latent factors by retaining trivial factors. Another method included the Scree Test (Cattell, 1966), whereby eigenvalues are plotted across successive factors, and extraction terminates at a point of discontinuity. However, the Scree Plot does not always yield clear-cut discontinuity and may underestimate latent factors by retaining the minimum number that account for maximum variance (Gorsuch, 1983).
Methods of factor rotation.
Factor rotation determines whether obtained factors are allowed to correlate with one another. We examined Varimax (orthogonal) rotation, which forces rotated dimensions to be uncorrelated, and Promax (oblique) rotation, which allows rotated factors to correlate and estimates correlational magnitudes. Varimax transformation distributes item loadings across factors to maximize differences among squared loadings for each factor and tends to avoid a single general factor (Kaiser, 1958; McDonald, 1985). Promax rotation raises loadings to a higher power to exaggerate differences among larger and smaller loadings, then applies oblique transformation using the “powered” loading matrix as a target (Hendrickson & White, 1964; McDonald, 1985).
Given that at least some of the dimensions underlying the effort measures and MMPI-2 scales might be correlated, we examined factor intercorrelations in the Promax-rotated solutions. From a theory-building standpoint, examination of the relations among the underlying dimensions is conceptually appropriate to assess convergent and discriminant validity and the overall pattern of construct validity (Bryant, 2000).
Methods of estimation.
We used the two most commonly adopted statistical procedures in the research literature (Thompson, 2004) for factor structure estimation: PCA and PAF. PCA identifies composite dimensions that together explain the greatest amount of the total variance in each item, whereas PAF identifies dimensions that together explain the greatest amount of common variance among the items (Bryant & Yarnold, 1995; Gorsuch, 1983; Thompson, 2004). PCA tends to produce stronger loadings and larger item communalities (i.e., proportions of explained variance in measured variables) than PAF (Thompson, 2004), as item reliabilities weaken and the number of the items decreases. Table 2 presents the intercorrelations, means and standard deviations for the 12 variables included in the present analyses.
Intercorrelations, means, and standard deviations for nine MMPI-2 scales and three cognitive effort measures
Number of latent factors.
The “Kaiser's stopping rule” and Scree Test revealed four factors underlying the effort measures and MMPI-2 scales (eigenvalues = 4.97, 2.73, 1.21, 1.02). On the basis of this converging evidence, we examined only the four-factor solution, using either PCA or PAF with either Varimax or Promax rotation. Regardless of rotation approach, the four factors explained 82.7% of the total variance for the 12 variables in the PCA solution and 74.1% of the common variance among the 12 variables in the PAF solution. Supporting Thompson's (2004) observations, PCA produced larger item communalities than PAF (mean difference in communality estimate = .08). Estimated item communalities correlated .80 for the component and factor solutions. Further supporting a four-factor structure, Factors I and II merged into a single bipolar dimension in the three-factor PCA and PAF solutions, making a three-factor structure less theoretically interpretable.
Effects of methods of rotation and estimation.
Table 3 displays factor loadings and item communalities by factor rotation (Varimax and Promax) and factor extraction (PCA and PAF). Promax rotation produced larger factor loadings than Varimax rotation for both PCA (mean difference in factor loadings = .14, .07, .08, and .01 for the four factors, respectively) and PAF (mean difference in factor loadings = .18, .08, .09, and .02 for the four factors, respectively) solutions. Factor loadings were larger in the PCA solution than in the PAF solution with Varimax rotation (mean PCA–PAF differences in factor loadings = .04, .02, .02. and .01, for the four factors, respectively). Factor loadings tended to be more comparable in magnitude for the PCA and PAF solutions with Promax rotation (mean PCA–PAF differences in factor loadings = −.01, .01, .01, and −.01 for the four factors, respectively).
Factor loadings and item communalities across two methods of rotation and two methods of factor estimation
Interpreting the four underlying dimensions.
Results were remarkably similar across all four structural solutions (see Table 3). For both rotational methods and both methods of estimation, the following applies: (a) Factor I, which we have termed “underreporting of psychological symptoms,” was defined primarily by K, S, and L; (b) Factor II, which we have termed “overreporting of neurotic symptoms,” was defined primarily by RBS, FBS, and Md; (c) Factor III, which we have termed “insufficient cognitive effort,” was defined primarily by LMT, VSVT, and TOMM; and (d) Factor IV, which we have termed “overreporting of psychotic or rarely endorsed symptoms,” was defined primarily by Fp, F, and to some extent Dsr2. Loadings of greatest magnitude are bolded to emphasize the pattern(s) of relationships among the response validity variables and factors.
Across all four structural solutions, Dsr2 had both a moderate negative loading on Factor I, as well as a moderate positive loading on Factor IV. This pattern of results suggests that low scores on Dsr2 reflect the underreporting of psychological symptoms, whereas high scores reflect the overreporting of psychotic or rarely endorsed symptoms. The opposite loadings for Dsr2 on Factors I and IV further suggest that these two dimensions will be inversely related in the Promax (correlated factors) solutions. The same can be said of F, which demonstrated relatively large (inverse) loadings with Factor I via Varimax rotation. Of interest, unlike the other scales that loaded highly on Factor IV (i.e., Fp, F), Dsr2 demonstrated near equivalent loadings on Factors II and IV with Varimax rotation (for both PCA and PAF). However, with Promax rotation, Dsr2 loaded more highly on Factor IV than Factor II.
Promax Factor Intercorrelations
Table 4 presents the correlations among the four factors from the PCA and PAF Promax-rotated solutions. Three results are noteworthy. First, for both the PCA and PAF solutions, only two of the four factors are positively intercorrelated: Factor II (overreporting of neurotic symptoms) and Factor IV (overreporting of psychotic or rarely endorsed symptoms). Second, for both the PCA and PAF solutions, Factor III (insufficient cognitive effort) is relatively uncorrelated with Factor I (underreporting of psychological symptoms). And finally, across both PCA and PAF solutions, half of the factor intercorrelations are negative and moderate to large in size. Specifically, Factor II (overreporting of neurotic symptoms) correlates negatively with both Factor I (underreporting of psychological symptoms) and Factor III (insufficient cognitive effort), and Factor I (underreporting of psychological symptoms) correlates negatively with Factor IV (overreporting of psychotic or rarely endorsed symptoms). These factor intercorrelations make sense conceptually and support the structural validity of the four response validity dimensions.
Correlations among four response validity factors
We also used our conceptual understanding of the four underlying dimensions to assess the construct validity of the four-factor model more systematically. Specifically, within the Promax four-factor solutions, we tested an a priori directional hypothesis concerning the relative ordering of the magnitude of the correlations between Factor III (insufficient cognitive effort) and the other three factors. Based on our interpretation of the underlying dimensions and prior researchers' findings regarding the differential impact of cognitive effort on MMPI-2 validity scales (Greiffenstein et al., 2002; Greve et al., 2006; Nelson et al., 2006; Ross et al., 2004; Slick et al., 1996), we hypothesized that Factor III (insufficient cognitive effort) would demonstrate (a) the strongest negative correlation with Factor II (overreporting of neurotic symptoms); (b) a lower, moderately negative correlation with Factor IV (overreporting of psychotic or rarely endorsed symptoms); and (c) the weakest correlation with Factor I (underreporting of psychological symptoms).
To test this hypothesized correlational pattern, we used the method of Meng et al. (1992) for contrasting correlated correlation coefficients to assess the predicted linear trend in the magnitude of the correlations, using the orthogonal weights of −1, 0, and +1 for the correlations between Factors III and II, Factors III and IV, and Factors III and I, respectively, for both the PCA and PAF Promax solutions (see Table 3). Confirming our correlational hypothesis, the planned contrast was statistically significant when imposed on the relevant factor intercorrelations for both the PCA solution, Z = 2.02, one-tailed p < .03, and the PAF solution, Z = 2.46, one-tailed p < .007. Supporting the construct validity of the four-factor model, the magnitude of the correlations of Factor III with the other three factors thus matched our hypothesized pattern across both methods of estimation. These results strengthen confidence in our conceptual interpretations of the factors underlying responses to the 12 variables examined.
DISCUSSION
The current exploratory factor analysis of cognitive effort measures and MMPI-2 validity scales was undertaken to better clarify the nature of response validity in a forensic sample referred for neuropsychological evaluation. Four unique factors of response validity emerged from this analysis, three pertaining to psychological bias (underreporting of psychological symptoms, overreporting of “neurotic” symptoms, overreporting of “psychotic” symptoms) and one pertaining to insufficient cognitive effort.
These results should be discussed in the context of the only other known factor analysis to examine multiple cognitive effort measures and MMPI-2 scales in forensic groups (Greiffenstein et al., 1995). Greiffenstein et al. provided preliminary evidence of independent psychiatric and neurological “malingering factors.” However, this study was limited by the fact that it did not examine some of the newer post-release validity scales (e.g., FBS), it did not apply multiple strategies of factor extraction to further explore and verify the cognitive and psychological constructs, and the use of orthogonal rotation only precluded an examination of the extent of construct independence. The current design allowed not only for a more complete examination of the post-release scales, but application of two factor analytic strategies (PCA and PAF) and approaches to factor rotation (orthogonal and oblique) demonstrated the robustness and diversity of the cognitive effort and psychological validity factors. Perhaps more importantly, use of Promax rotation in the current study allowed for examination of interfactor correlations.
Regarding the relationships of cognitive effort with the psychological validity factors, three main points are relevant to forensic examiners of civil litigants and claimants. First, the present findings suggest virtually no relationship between cognitive effort and underreporting of psychological symptoms. VSVT performances, for example, do not seem to have any meaningful association with scales L, K, or S. Second, there is only a small relationship between effort and “psychotic” or rarely endorsed psychological symptoms. VSVT performances, for example, have only minimal associations with scales F and Fp. Finally, there seems to be some convergence between cognitive effort and fraudulent “neurotic” psychological symptoms, such as somatoform or depressive disorders. VSVT performances, for example, showed a moderate correlation with FBS (−.48).
That cognitive effort may have a stronger association with overreported “neurotic” as opposed to “psychotic” symptoms is consistent with other recent studies (e.g., Greiffenstein et al., 2002; Larrabee, 2003a; Ross et al., 2004; Slick et al., 1996), which have found scales such as FBS to have a differential relationship with cognitive effort relative to other scales, such as F and Fp. However, there are two caveats that should be considered with regard to this issue. First, effort and neurotic psychological symptoms remain only moderately associated, suggesting that they both provide unique information regarding their respective constructs of response validity. In keeping with Dearth et al. (2005), the moderate relationships observed in the present analysis support the notion that effort measures and MMPI-2 validity scales are unique enough to warrant application of both to accurately assess symptom validity. As such, we agree with the statement of Greiffenstein et al. (2002, p. 1599) who state, “malingering in one domain does not automatically imply malingering in another domain.” Interpretation of cognitive effort and MMPI-2 validity profiles should be made on an individual basis. Individual litigants may demonstrate extremely poor cognitive effort in the presence of unremarkable MMPI-2 profiles, or conversely, exhibit extreme elevations on multiple MMPI-2 validity scales and demonstrate excellent cognitive effort (Nelson & Sweet, in press).
The second caveat pertains to the current forensic sample. Our participants included civil litigants and individuals referred for independent neuropsychological examination associated with cognitive complaints. It is possible, if not likely, that the present findings would not be replicated in groups who are more likely to exaggerate psychotic symptoms (e.g., criminal litigants). In these samples, F, Fp, and other scales designed to detect exaggerated psychosis may demonstrate a larger association with cognitive effort than with neurotic validity scales. Furthermore, the present sample was derived retrospectively from a single independent practice within the United States, suggesting that it may not fully represent or generalize to the greater culture's forensic population. Also, while technically large enough for an exploratory factor analysis, the sample is nevertheless small enough to suggest that generalizations should be made with some degree of discretion.
Present MMPI-2 validity scale findings are in some ways remarkably consistent with what might have been anticipated given the intent of the scales' development. For example, L, K, and S, all of which were designed to evaluate defensiveness or a denial of psychopathological symptoms (Graham, 2000), demonstrated large correlations with one another and consistently loaded on the same factor. Also, all of the other “overreporting” scales (F, Fp, FBS, RBS, Md, Dsr2) included in the present study demonstrated correlations with L, K, and S in the expected direction (i.e., negative), although the magnitudes of these correlations varied substantially (ranging from −.02 for FBS/L to −.75 for S/Dsr2).
A potential source of artifact in the current factor analysis is that item overlap among scales may have been responsible for their loading on the same factors. Fp, for example, shares 17 of 27 (63%) items with F, was moderately correlated (.59) with F, and loaded on the same factor (overreporting of psychotic symptoms) as F. However, examination of other scales suggests that item overlap did not consistently account for factor loadings. For example, Dsr2 overlaps minimally with RBS (5 items) and Md (1 item), but correlates moderately with both (.67, .40, respectively) and loads to some degree with both scales on the same factor (overreporting of neurotic symptoms). More striking than this are L and K, which have only one overlapping item, but nevertheless correlate with one another (.61) and load on the same factor (underreporting of symptoms). Md and RBS share only three items, but are nevertheless correlated (.60) and load on the same factor (overreporting of neurotic symptoms). These findings suggest that item overlap does not solely account for MMPI-2 loadings in the present factor analysis.
It is interesting that Dsr2 was the only one of the overreporting validity scales to demonstrate sizeable factor loadings with overreporting of both neurotic and psychotic symptoms. This trend is evident both in terms of factor loadings and the large correlations that Dsr2 demonstrated with all of the other overreporting scales, particularly F(.82). To identify possible explanations, we examined the item content of Dsr2 and its overlap with other overreporting scales. Consisting of 58 items, Dsr2 is longer than all but one of the other overreporting scales (F consists of 60 items). Given its greater length, we thought it possible that Dsr2 may be more diversely represented with regard to the construct of response validity than other scales. Also, neurotic validity scales, such as FBS, have relatively little item overlap with psychotic scales, such as F(only 4 items or 6.7%). In contrast, 11 (19.0%) of the Dsr2 items overlap with F, and 11 (19.0%) Dsr2 items also overlap with FBS, suggesting that larger correlations might be expected between Dsr2 and F than FBS and F. In fact, Table 1 confirms that this is true; FBS correlates only moderately with F(.48), while Dsr2 correlates very highly with F(.82). This reasoning is less supported, however, on inspection of Dsr2 and its overlap with RBS and Md (as mentioned, Dsr2 overlaps minimally with these scales but, nevertheless, correlates moderately with both). Thus, item overlap does not wholly account for the multiple factor loadings that Dsr2 demonstrated; Dsr2 is likely a more general indicator of response validity, whereas the other psychological validity scales appear to reflect more specific validity dimensions.
The construct of cognitive response validity in the present study was represented solely by forced-choice measures, and this strategy can be considered a limitation. Unfortunately, other symptom validity data from non–forced-choice effort or ability measures were not uniformly available across participants, and this finding precluded their inclusion in the current analysis. Using different language, because the current factor analysis was limited to a single method to assess both the cognitive and psychological response validity variables, it lacked a multimethod–multitrait approach to the constructs of interest (Campbell & Fiske, 1959). Greiffenstein et al. (1995) were also compelled to address the multimethod–multitrait issue, largely given the orthogonal nature of their findings (i.e., Varimax rotation resulted in discrete cognitive and psychological response validity factors). However, they found their only forced-choice effort measure (i.e., PDRT) to load on the same factor as effort measures with divergent methodologies (e.g., Rey-15 Item Test) and load separately from the psychological validity factor. Based on these findings, Greiffenstein et al. concluded that method did not account for the cognitive effort factor.
Additionally, Greiffenstein et al. (1995) recognized that the multitrait–multimethod approach is one that is unlikely to be applicable to research studies given current approaches to response validity assessment. They state, “It would probably be very difficult to utilize a multitrait–multimethod validation procedure in a similar [factor analytic] study, as psychiatric symptoms are predominantly measured by self-report and neuropsychological problems are assessed with performance measures” (p. 237). For example, while there exist self-report measures addressing cognitive complaints (e.g., Frontal Systems Behavior Scale, FrSBe; Grace & Malloy, 2001), the field of neuropsychology has yet to develop a psychological analogue to cognitive ability tasks. Indeed, it is difficult to envisage an objective test of psychological “ability” (e.g., depression, anxiety, and so on).
We suggest that the construct of cognitive response validity and its relation to psychological response validity will be further elucidated if future factor analyses examine multiple non–forced-choice cognitive effort measures or indices with multiple forced-choice effort measures and MMPI-2 validity scales. For example, application of factor analysis that includes the VSVT, TOMM, and LMT, as well as other effort measures (e.g., Rey-15) and indices (e.g., WAIS-III IQ, Digit Span), will clarify the extent to which method per se may account for the cognitive response validity construct(s). However, forced-choice methodology is currently regarded as the most effective approach to cognitive symptom validity assessment (Slick et al., 1999), and the concurrent use of two or more forced-choice measures (such as the VSVT and TOMM) has been recommended in the detection of insufficient effort (Lynch, 2004). Thus, despite the uniform method of cognitive response validity assessment (forced-choice) in the current study, we consider it unique in that it is the only study to date to incorporate three forced-choice effort measures concurrently with MMPI-2 validity scales through factor analysis, and to examine the relationship of cognitive and psychological response validity constructs.
In conclusion, present findings illustrate the heterogeneous nature of response validity in a civil forensic sample. Litigants and claimants can exhibit complex presentations and unique forms of response bias (cognitive, psychological, or both). In an exhaustive review of the use of MMPI-2 validity scales in forensic settings, and particularly their use vis-à-vis the Daubert Rule, Lees-Haley et al. (2002, p. 171) state that, while MMPI-2 validity scales “cannot be used as an indicator of cognitive exaggeration, they are often helpful in providing supporting evidence for the exaggeration of psychological distress.” This statement is particularly relevant to present findings, which suggest that while cognitive and psychological response validity measures are not wholly dissimilar, they are distinct enough to warrant their incremental use in neuropsychological evaluation of civil litigants.
ACKNOWLEDGMENTS
The authors have no financial or other relationship that would constitute a conflict of interest related to the research represented in this study. There was no commercial or other financial support for this project. The authors thank Richard L. Gorsuch, Ph.D., for his thoughtful suggestions regarding the application of factor analysis in clinical samples.
