Introduction
The assessment of executive functioning is a crucial component of comprehensive neuropsychological evaluations. Executive functioning is a complex, multifaceted construct which essentially involves the organization and direction of cognition, emotion, and behavior. Specific components include control of impulse and attention, initiation, strategizing, mental flexibility, and planning (Anderson, Reference Anderson2008). Measurement of this domain is particularly relevant in patients with traumatic brain injury (TBI), in whom executive dysfunction is often a part of the clinical picture, ostensibly due to the vulnerability of the frontal lobes to such injury (Lucas & Addeo, Reference Lucas and Addeo2006). Moreover, measurement of this area is important in TBI, given its functional implications instrumental with regard to activities of daily living day-to-day such as driving, competitive employment, and interpersonal social pragmatics (McDonald, Reference McDonald2008).
Some problems are inherent with the use of executive functioning tests as measures of frontal lobe integrity (Stuss & Alexander, Reference Stuss and Alexander2000). Research has shown, for example, that purported executive functioning tests are not pure measures of frontal lobe functioning, because people with non-frontal lesions can also perform poorly on the tasks (Anderson, Damasio, Jones, & Tranel, Reference Anderson, Damasio, Jones and Tranel1991; Anderson, Bigler, & Blatter, Reference Anderson, Bigler and Blatter1995; Nyhus & Barcelo, Reference Nyhus and Barcelo2009). Similarly, people with known frontal lobe lesions do not necessarily always perform poorly on executive functioning tasks (Manchester, Priestley, & Jackson, Reference Manchester, Priestley and Jackson2004). Because TBI often involves diffuse or multifocal lesions, this population is at increased risk for executive dysfunction.
The Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001) was developed as a battery of standardized tests designed to measure a wide spectrum of abilities associated with executive functioning. The individual subtests can be used alone, or in combination (Delis et al., Reference Delis, Kaplan and Kramer2001). The present investigation examined the clinical utility of two commonly used D–KEFS subtests, Verbal and Design Fluency, in a sample of patients with TBI.
There is a plethora of research examining the sensitivity of generic verbal fluency tasks to TBI (Crawford, Knight, & Alsop, Reference Crawford, Knight and Alsop2007; Henry, & Crawford, Reference Henry and Crawford2004; Mathias, Beall, & Bigler, Reference Mathias, Beall and Bigler2004; Milders, Fuchs, & Crawford, Reference Milders, Fuchs and Crawford2003; Milders, Ietswaart, Crawford, & Currie, Reference Milders, Ietswaart, Crawford and Currie2006). A recent study has found that in the acute stages of recovery from TBI, pre-morbid education and brain injury severity both predicted verbal fluency performance (Leblanc, De Guise, Feyz, & Lamoureux, Reference Leblanc, De Guise, Feyz and Lamoureux2006). In another study with a relatively small sample of patients with TBI who had documented frontal lobe lesions, the authors found that phonemic, but not semantic, fluency performance was worse in patients in comparison to controls; at the same time, lesions size correlated significantly with semantic fluency, but not phonemic, fluency (Jurado, Mataro, Verger, Bartumeus, & Junque, Reference Jurado, Mataro, Verger, Bartumeus and Junque2000). Other studies have suggested that processing speed may mediate the impact of TBI on executive functioning (Bittner & Crowe, Reference Bittner and Crowe2007).
Studies of nonverbal fluency after TBI are less plentiful, and rarely do the same studies investigate verbal as well as design fluency. Various paper-and-pencil design fluency tests like Ruff Figural Fluency Test (RFFT; Ruff, Reference Ruff1988) and the Five-Point test (FPT; Regard, Strauss & Knapp, Reference Regard, Strauss and Knapp1982) have been used. Mathias et al. (Reference Mathias, Beall and Bigler2004) found that performance on the RFFT, but not phonemic fluency, discriminated between a group of patients with uncomplicated mild TBI and demographically matched controls. In a study using a small sample of patients with documented moderate-severe TBI, the clinical group performed worse on the RFFT than a control group; however, their figural fluency scores did not show a significant association with their relatives’ ratings of behavioral problems and social integration (Milders et al., Reference Milders, Fuchs and Crawford2003).
The D-KEFS Verbal and Design Fluency tests build on generic versions of similar tasks by adding the component of cognitive shifting, another hypothesized component of executive functioning. Widely used tests of cognitive flexibility and shifting include the Trail Making Test (TMT; Reitan & Wolfson, Reference Reitan and Wolfson1993), and the Wisconsin Card Sorting Test (WCST; Heaton, Chelune, Talley, Kay, & Curtiss, Reference Heaton, Chelune, Talley, Kay and Curtiss1993), both of which have well established utility in the assessment of TBI (Ord, Greve, Bianchini, & Aguerrevere, Reference Ord, Greve, Bianchini and Aguerrevere2010; Sherrill-Pattison, Donders, & Thompson, Reference Sherrill-Pattison, Donders and Thompson2000). A less popular test which, similar to the D-KEFS, combines the concepts of verbal fluency and cognitive shifting is the Alternating Fluency Test (AFT; Downes, Sharp, Costall, Sagar, & Howe, Reference Downes, Sharp, Costall, Sagar and Howe1993). In a sample of patients with TBI, Milders et al. (Reference Milders, Ietswaart, Crawford and Currie2006) found that patients performed worse on the AFT than controls at 2 and 12 months post-injury. Furthermore, performance on AFT was associated with poor performance on “Theory of Mind” tasks. Thus, adding a “switching” component to a fluency task may be potentially clinically useful.
While there is a vast literature studying the effects of fluency and cognitive shifting in TBI, the results of these studies conflict at times. This may be due to the variability in TBI samples in terms of the severity of injury and length of time since injury, as well as the differing methodologies and normative samples used to measure fluency and flexibility. Use of the D-KEFS to assess verbal and design fluency offers a possible advantage over using separate, distinct tests. Because all of the D-KEFS tests were normed on the same population, direct comparison between Verbal and Design Fluency scores can be made. In addition, the fluency tasks both include the component of cognitive shifting, with the potential to assess a unique component of the fluency domain.
Despite these potential benefits, we know of only one published study specifically looking at the clinical applicability of the D-KEFS fluency tests in TBI. Sigurdardottir, Jerstad, Andelic, Roe, and Schanke (Reference Sigurdardottir, Jerstad, Andelic, Roe and Schanke2010) found that impairment on the phonemic fluency and semantic switching trials occurred more frequently in patients with TBI who had olfactory dysfunction than in those without such dysfunction, whereas there was no difference between the performances of the two groups on the semantic fluency task or any of the design fluency trials. However, that study did not examine predictors or mediators of D-KEFS performance.
While there is a clear lack of research regarding the D-KEFS fluency tests in TBI, there have been several studies exploring the usefulness of its fluency subtests in other populations. A study examining the Verbal and Design Fluency subtests in normal older adults found that the tests did not contribute unique variance in observed functional ability (Mitchell & Miller, Reference Mitchell and Miller2008). Research on patients with frontal lesions have shown that performance on the Verbal Fluency tests is influenced by left frontal lesions, while Design Fluency performance is affected by bilateral frontal dysfunction (Baldo, Shimamura, Delis, Kramer, & Kaplan, Reference Baldo, Shimamura, Delis, Kramer and Kaplan2001). Other studies examining the Verbal and Design Fluency subtests have used patients with schizophrenia (Nemoto, Mizuno, & Kashima, Reference Nemoto, Mizuno and Kashima2005), as well as Alzheimer's disease and frontotemporal dementia (Houston et al., Reference Houston, Delis, Lansing, Jacobson, Cobell, Salmon and Bondi2005; Kramer et al., Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007; Razani et al., Reference Razani, Casas, Wong, Lu, Mendez, Alessi and Josephson2007). Most notably, the switching trial of Design Fluency has been shown to be a relatively pure and sensitive measure of set shifting in patients with neurodegenerative disorders (Judy et al., Reference Judy, Possin, Wilson, Quitania, Kramer, Boxer and Johnson2010).
The specific purpose of the current study was to investigate the criterion validity of the D-KEFS Verbal and Design Fluency subtests in the assessment of patients with independently verified complicated mild–severe TBI. The performance of a sample of clinical patients on the D-KEFS was compared with that of demographically matched controls from the standardization sample. It was decided a priori that, to be clinically useful, D-KEFS subtests should meet the following criteria: (a) the D-KEFS performance of patients with TBI should be statistically significantly worse than that of healthy controls, (b) brain injury severity should contribute unique variance to D-KEFS performance in clinical patients, and (c) diagnostic classification accuracy of the D-KEFS should be associated with a likelihood ratio (sensitivity/[1 – specificity]) ≥ 2, which has previously been suggested as a reasonable minimum standard for clinical decision making (Grimes & Schulz, Reference Grimes and Schulz2005).
Method
Participants
After receiving institutional review board approval, archival data was retrieved from a 5-year consecutive series of referred patients with TBI who were evaluated at a Midwestern rehabilitation facility. The 65 clinical participants met the following criteria: (1) ≥16 years and ≤84 years of age, (2) diagnosis of TBI with evidence for an acute intracranial lesion on neuroimaging; (3) neuropsychological assessment with inclusion of the D-KEFS Verbal and Design Fluency subtests completed between 1 and 12 months after injury, (4) absence of any prior history of neurological or psychiatric impairment, substance abuse, or enrollment in special education, (5) no current disputed financial compensation-seeking, and (6) scoring in the valid range on an independent measure of effort and motivation; that is, either the Test of Memory Malingering (Tombaugh, Reference Tombaugh1996; n = 36) or the Word Memory Test (Green, Reference Green2003; n = 29). During the time period that these data were collected, the D-KEFS Verbal and Design Fluency subtests had been administered routinely to all older adolescent and adult patients with TBI who were referred for neuropsychological evaluation at the organization where this investigation was pursued, except in cases where there were complicating circumstances that would have invalidated the test results (e.g., English not being the primary language or fracture of the dominant forearm). In cases where more than one assessment was performed, only results from initial evaluations were considered.
The final clinical sample was comprised of 45 men (69.23%) and 20 women (30.77%), with a mean age of 31.99 years (SD = 17.44) and a mean level of education of 12.69 years (SD = 2.14). The vast majority of patients in the sample were Caucasian (n = 60; 92.31%), with other self-reported ethnicities including African (n = 2, 3.08%), Latino/a (n = 2, 3.08%), and Asian (n = 1, 1.54%). The majority (n = 40; 61.54%) of these participants sustained their injuries as the result of a motor vehicle accident, with the remaining injuries involving recreational activities (n = 12; 18.46%), falls (n = 6; 9.23%), assaults (n = 3; 4.62%), and other (n = 4; 6.15%). Mean time since injury was 119.95 days (SD = 76.13).
Because an inclusion criterion for this study was that there had to be neuroimaging evidence for an intracranial lesion, there were no persons with uncomplicated mild TBI in the final sample (i.e., without prolonged loss of consciousness and with negative neuroimaging findings). Specific neuroimaging findings included diffuse involvement (e.g., edema, shear injury; n = 23; 35.38%) and/or focal lesions (e.g., contusion, hemorrhage; n = 60; 92.31%), with 18 patients (27.69%) having a combination of focal and diffuse findings. Lesions involving the frontal regions of the brain (n = 55; 84.62%) were relatively most common but it should be noted that these lesions were often not exclusive; that is, almost half of these individuals with frontal lesions (n = 26) also either had posterior and/or diffuse lesions. Median duration of coma, defined as the number of days until there was a reliable response to verbal commands, was 1 day (M = 4.15; SD = 8.33; range, 0–42 days). Approximately a third of the sample (n = 23, 35.39%) had coma <30 min, and approximately another third of the sample (n = 22; 33.85%) had coma >24 hr.
A comparison group of 65 persons from the standardization sample was subsequently requested from the publisher of the D-KEFS. These control participants were closely matched to clinical patients on the basis of age, gender, ethnicity (dichotomized as Caucasian vs. Other) and level of education. Post hoc comparisons of the two groups confirmed that there were no meaningful differences between them on any of these demographic variables (all p's > .50).
There is partial overlap between the current clinical sample and that used in a previous investigation (Strong & Donders, Reference Strong and Donders2008) of the criterion validity of the Continuous Visual Memory Test (Trahan & Larrabee, Reference Trahan and Larrabee1988). Specifically, approximately half (n = 33) of the current participants had also been included in that other study, which did not include any D-KEFS test data.
Measures
Six subtests from the D-KEFS were included in this investigation. The three Verbal Fluency subtests all require verbal response generation with a 1-min time limit. Letter Fluency contains three trials that each require generation of words that start with a specific letter (F, A, and S). Category Fluency includes two trials that each require generation of words that belong to a specific semantic category (animals and boys’ names). Category Switching includes a single trial that requires the examinee to continuously alternate between two different semantic categories (fruits and furniture).
The three D-KEFS Design Fluency subtests all require generation of as many unique visual designs as possible within a 1-min time limit by drawing four straight lines, connecting dots that are presented in rows of stimulus boxes. In Filled Dots, the stimulus boxes all contain five filled dots, and the examinee can use any of the dots. In Empty Dots, the stimulus boxes include five filled dots and five empty dots, and the examinee must connect only empty dots while filled dots function as visual distractors. The Dot Switching condition has the same layout as Empty Dots but the examinee must alternate between filled and empty dots in each box.
All of the clinical participants also completed either the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, Reference Wechsler1997; n = 50) or the two- or four-subtest version of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, Reference Wechsler1999; n = 15). The WASI was typically used for reasons of time constraints or patient fatigability.
Finally, all of the clinical participants also completed an independent test of executive functioning, the Wisconsin Card Sorting Test (WCST; Heaton et al., Reference Heaton, Chelune, Talley, Kay and Curtiss1993). This is a task that requires the consideration of different possible solutions to a task, learning from corrective feedback, suppression of perseverative tendencies, and adjusting to changes in task conditions. The WCST has known sensitivity to a variety of acquired neurological disorders, including TBI (Wiegner & Donders, Reference Wiegner and Donders1999). The goal of including the WCST in the analyses was to determine to what extent relationships between D-KEFS subtests and a variety of injury and psychometric variables would be similar to, or different from, those found with a separate and well-validated test of executive functioning. WCST data from one participant were excluded from the analyses because the test was discontinued prematurely in that case due to the examinee's level of frustration with it.
Procedure
The D-KEFS, WAIS-III or WASI, and WCST were administered to clinical patients in a standardized manner, almost always on an outpatient basis, when they were medically stable and could recall meaningful information from day to day. From the D-KEFS subtests, scaled scores (M = 10; SD = 3) were used. For those participants on whom WAIS-III data were available, factor index scores (M = 100; SD = 15) were used. Finally, from the WCST, the standard score (M = 100; SD = 15) for Perseverative Responses was included in the analyses because it is the most widely used and validated measure from this test. Higher scores reflect better performance on all of these variables.
Group mean differences on the D-KEFS subtests were evaluated with a multivariate analysis of variance, with post hoc contrasts involving the stepwise Bonferroni adjustment to balance the relative risk of Type I and Type II errors. The eta-squared (η2) statistic was used as an index of effect size in group comparisons. This statistic reflects the proportion of the variance in the dependent variable that can be explained by the independent variable.
Those D-KEFS subtests that yielded statistically significant univariate group contrasts were then included in a logistic regression to determine their classification accuracy. The likelihood ratio as well as the Area Under the Curve were calculated as measures of degree of classification accuracy.
Finally, hierarchical linear regression was used to determine to what extent performance on either the D-KEFS or the WCST could be predicted by injury severity (length of coma) and/or WAIS-III factor index scores. The R 2 statistic was used as the measure of the amount of variance accounted for.
Results
The performance of the two groups on the D-KEFS subtests is presented in Figure 1. A multivariate analysis of variance with groups as the independent variable and the 6 D-KEFS scaled scores as the dependent variables yielded a statistically significant main effect of groups, F(6,123) = 4.27, p < .0001, with the patients with TBI doing worse than the demographically matched controls. Post hoc contrasts revealed that the group differences were statistically significant only for Letter Fluency, F(1,128) = 13.80, p < .0003, η2 = 0.10, and Category Switching, F (1, 128) = 5.01, p < .03, η2 = 0.04, with small effect sizes.
Fig. 1 Performance on the Delis-Kaplan Executive Function System. Control group selected from the standardization data from the Delis-Kaplan Executive Function System (D-KEFS). © 2001 NCS Pearson. Used with permission. All rights reserved.
To make sure that performance on Letter Fluency and Category Switching in the clinical group was not confounded by other variables, some post hoc analyses were performed. There were no statistically significant correlations between either of these two D-KEFS variables and age, time interval since injury, or level of education (p > .05 for all correlations). Similarly, there were no statistically significant subgroup differences on either Letter Fluency or Category Switching, based on gender, ethnicity (dichotomized as Caucasian vs. other), or the presence/absence of a focal anterior lesion (p > .05 for all comparisons). When the sample was split into those who were injured in motor vehicle accidents (n = 40; 61.54%) versus other injury circumstances (n = 25; 38.46%), there was a small-effect group difference on Letter Fluency, F(1,63) = 4.25, p < .05, η2 = 0.06, with those in the former group having lower scores (M = 7.43; SD = 3.13) than those in the latter group (M = 9.04; SD = 2.98). There was no statistically significant difference between the motor vehicle accident group (M = 8.15; SD = 3.94) and the other injury group (M = 9.48; SD = 3.24) on Category Switching, F(1,63) = 2.00, p > .15, η2 = 0.03.
It is possible that the isolated finding of a statistically significant subgroup difference on Letter Fluency is spurious, related to alpha inflation with multiple independent comparisons. However, it is more likely that this finding simply reflects the fact that individuals in the motor vehicle accident groups sustained more severe TBI than those in the other group. For example, compared with those incurring their TBI as the result of recreational activities, falls, etc., the patients in the motor vehicle group were almost 6 times as likely to have been in coma for ≥24 hours, χ 2 (N = 65) = 10.35, p < .002, OR = 5.88 (95% CI = 1.91–18.11).
Next, using logistic regression analysis, we explored how well the D-KEFS Letter Fluency and Category Switching scores could classify individuals into the clinical and control groups. The overall classification accuracy was 65.39%, with a sensitivity of 66.15% and a specificity of 64.62%, yielding a likelihood ratio of 1.87 and an Area Under the Curve of 0.69. Positive and negative predictive values were 65.15% and 65.63%, respectively.
We then investigated the performance of the clinical group on the other psychometric variables of interest, and their covariance with the D-KEFS variables as well as injury severity. The average performance of the patients with TBI on the WAIS-III or WASI and on the WCST is presented in Table 1, whereas the correlations between these variables and, respectively, length of coma and the D-KEFS Letter Fluency and Category Switching measures are presented in Table 2. Inspection of Table 1 suggests that the average performance of the patients with TBI on the WAIS-III and WCST was still within the normal range, similar to the D-KEFS variables presented in Figure 1. The correlations in Table 2 reflect that Letter Fluency and Category Switching were only moderately correlated, sharing approximately 13% of common variance. Both of these D-KEFS subtests covaried in a statistically significant manner with injury severity, as did WCST Perseverative Responses, but neither Letter Fluency nor Category Switching shared much variance with the latter variable, suggesting that the D-KEFS and the WCST measure different aspects of cognitive functioning.
Table 1 Performance of patients with traumatic brain injury on other psychometric variables
Note. WAIS–III = Wechsler Adult Intelligence Scale-Third Edition; WCST = Wisconsin Card Sorting Test.
*n = 65. **n = 50. ***n = 64.
Table 2 Correlations between injury and psychometric variables in 50 patients with traumatic brain injury
ap < .05. bp < .01. cp < .001. dp < .0001.
We then investigated, for those 50 clinical participants for whom WAIS-III data were available, the impact of, respectively, injury severity and WAIS-III factor index scores on the D-KEFS and WCST variables. Separate hierarchical regression analyses were conducted for, respectively, Letter Fluency, Category Switching, and Perseverative Responses. In each analysis, coma duration (representing brain injury severity) and WAIS-III factor index scores were the independent variables. The findings from these regression analyses are presented in Table 3. Inspection of collinearity diagnostics did not suggest threats to the validity of any of the three final models (e.g., all variance inflation factors were ≤1.51).
Table 3 Hierarchical regression models in 50 patients with traumatic brain injury
Note. *p < .05. **p < .001. SRC = Standardized regression coefficient. VC = Verbal Comprehension. PO = Perceptual Organization. WM = Working Memory. PS = Processing Speed.
Examination of Table 3 reveals that when coma was entered at the first step, it was a statistically significant predictor all three variables, with longer coma being associated with worse test performance. With the addition of the WAIS-III factor index scores at the second step, the effect of brain injury severity on Letter Fluency was strongly attenuated, and only Verbal Comprehension and Processing Speed were statistically significant in the model. In light of the fact that only Processing Speed and not Verbal Comprehension was statistically significantly correlated with length of coma (see Table 2), these findings suggested that the effect of coma on Letter Fluency was mediated through Processing Speed, but that Verbal Comprehension made an independent contribution, likely reflecting the impact of premorbid ability level.
The findings were much different for Category Switching and for Perseverative Responses. In both cases, the contribution of length of coma remained a statistically significant predictor of performances on both of these tests, even after entering the WAIS-III results. In fact, none of the WAIS-III factor index scores contributed statistically significantly to the final models for either Category Switching or Perseverative Responses. This suggests that these two variables were affected directly by injury severity and not indirectly by either speed of information processing or premorbid ability level.
Discussion
The purpose of this investigation was to determine the criterion validity of the D-KEFS Verbal and Design Fluency subtests in the evaluation of patients with complicated mild–severe TBI. The findings demonstrated that these patients obtained mean scores that were statistically significantly lower than those of demographically matched healthy controls on two of the six D-KEFS variables, Letter Fluency and Category Switching. However, these two D-KEFS variables showed a suboptimal classification rate (i.e., likelihood ratio <2), and only for Category Switching was there evidence for a unique contribution of injury severity to level of performance. Thus, taking into account the a priori specified criteria for clinical utility, the D-KEFS met one (a) unequivocally, one (b) partially, and one (c) not at all. This suggests that its Verbal Fluency subtests, and Category Switching in particular, may have some value within a larger battery of neuropsychological tests, and when the findings are considered in combination with records review and a thorough history. However, at least with regard to the assessment of sequelae of complicated mild–severe TBI, the D-KEFS Design Fluency subtests appear to lack criterion validity.
The sensitivity of verbal fluency tasks to the effect of TBI is consistent with previous independent studies (Henry & Crawford, Reference Henry and Crawford2004; Leblanc, De Guise, Feyz, & Lamoureux, Reference Leblanc, De Guise, Feyz and Lamoureux2006; Milders et al., Reference Milders, Ietswaart, Crawford and Currie2006). Also consistent with previous research are the current findings that the effect of injury severity on phonemic fluency was mediated by speed of information processing (Bittner & Crowe, Reference Bittner and Crowe2007). The novel contribution from the current study is the evidence that Category Switching was directly affected by cerebral compromise and that this sensitivity could not be accounted for on the basis of either premorbid ability level or processing speed. In other words, Category Switching may be the only of the D-KEFS fluency subtest that truly measures a component of “executive” functioning after complicated mild–severe TBI. Furthermore, it appears to do so in a non-redundant manner, as evidenced by its low correlation with a more traditional test of that construct; that is, the WCST. That finding suggests that Category Switching taps into a distinctly different aspect of executive functioning; perhaps a component related to independent strategy generation and dual-task monitoring, in the absence of the continuous feedback that is provided on the WCST, with the latter likely being more related to the ability to suppress perseverative tendencies as well as the ability to adjust one's strategy in response to changing task demands.
There have been relatively fewer prior investigations of the effect of TBI on figural or design fluency. Mathias et al. (Reference Mathias, Beall and Bigler2004) found that patients with mild TBI did worse than demographically matched control on the RFFT, but this was only within the very acute phase (<1 month) after injury, and this may therefore have been a non-specific and transient effect. Milders et al. (Reference Milders, Fuchs and Crawford2003), on the other hand, gave the same test to persons with moderate–severe TBI and found that patients had fewer correct responses on it than demographically controls. Notably, this difference remained statistically significant, even when correcting alpha for the effect of multiple comparisons, and the authors also demonstrated through regression modeling that it could not be accounted for completely on the basis of motor speed. The major difference between the Milders et al. (Reference Milders, Fuchs and Crawford2003) study and the current one, in which none of the DiKEFS Design Fluency subtests yielded statistically significant group differences, is that the patients in the former study were all more than 4 years post injury. Therefore, they may have been recruited with some selection bias because of persistent complaints. In contrast, the patients in the current study were consecutive rehabilitation referrals who were all evaluated within 1 year post injury. Our findings suggest that in the subacute recovery phase, the D-KEFS Design Fluency subtests are not sufficiently able to discriminate between complicated mild–severe TBI and demographically matched controls.
There are some potential limitations of this study that must be considered. First, we used a referred convenience sample of patients with complicated mild–severe TBI. Consequently, these results should not be generalized to mild, uncomplicated TBI, which is typically not associated with persistent cognitive impairment (Iverson, Reference Iverson2005; Schretlen & Shapiro, Reference Schretlen and Shapiro2003). Second, the current sample was primarily of Caucasian ethnicity, and replication with an ethnically more diverse sample is desirable. Third, we were not able to do volumetric analyses of cerebral lesion volume and its covariance with D-KEFS performance. Such analyses have shown promise in a sample of patients with dementia (e.g., Kramer et al., Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007) but have not yet seen application in sufficiently large samples of patients with TBI, which is a goal for future research. A final limitation was that the WASI was used with some of the clinical participants. However, we did have a sufficient number of patients with TBI (n = 50) to allow regression analyses to demonstrate that Verbal Comprehension and Processing Speed contributed substantively to prediction of Letter Fluency but not Category Switching.
With these reservations in mind, we conclude that the D-KEFS Verbal Fluency subtests but not its Design Fluency subtests have modest criterion validity and may be clinically useful in the assessment of complicated mild–severe TBI. We do not recommend administering only Category Switching without the other, preceding Verbal Fluency subtests because previous research has demonstrated that performance on Category Switching is predicted at least in part by performance on its simpler component task (i.e., Semantic Fluency; Wecker, Kramer, Hallam, & Delis, Reference Wecker, Kramer, Hallam and Delis2005). Furthermore, in light of the suboptimal classification accuracy in this study, results from D-KEFS Verbal Fluency subtests should never be used in isolation, or as the only “executive” tasks in a neuropsychological assessment.
In light of the fact that there was partial overlap between the current sample and that used in one of our prior studies with a visual memory test (CVMT; Strong & Donders, Reference Strong and Donders2008), it may be helpful to compare the classification rates of the D-KEFS in the current investigation and that of the CVMT in the previous one, which also used a demographically matched control sample. In this study, the D-KEFS had a total correct classification accuracy of 65.39%, with a likelihood ratio of 1.87. In our previous investigation, the CVMT did relatively better, with a total correct percentage of 76.42 and a likelihood ratio of 3.79. Because the respective control groups were drawn from different standardization samples, we were not able to evaluate the degree to which the classification rate might be affected if CVMT and D-KEFS variables were combined.
A specific goal for future research is investigation of the utility of D-KEFS Verbal Fluency tasks to predict longer-term psychosocial outcomes (e.g., community integration or quality of life) after complicated mild–severe TBI. Previous studies in community-dwelling healthy older adults (Mitchell & Miller, Reference Mitchell and Miller2008) and persons with dementia (Razani et al., Reference Razani, Casas, Wong, Lu, Mendez, Alessi and Josephson2007) have not found strong evidence for the independent contribution of D-KEFS tasks to predict activities of daily living. However, those studies were hampered by limitations in participant-to-variable ratio, and this relationship has not yet been explored with regard to the functioning of patients with TBI, several years after injury. In the current health care environment, it will be increasingly important to demonstrate that neuropsychological assessment brings “added value” to the identification of persons at risk for poor outcome after TBI or other neurological injuries, in excess of anything that can already be predicted on the basis of demographical or radiological variables. There are some studies in samples of pediatric TBI (e.g., Miller & Donders, Reference Miller and Donders2003) as well as adult TBI (Hanks et al., Reference Hanks, Millis, Ricker, Giacino, Nakese-Richardson, Frol and Gordon2008) that show promise in this regard but this has not yet been done with D-KEFS variables. In the meantime, D-KEFS Verbal Fluency subtests can be used cautiously in the context of a more comprehensive neuropsychological evaluation that also includes other psychometric measures of executive functioning as well as careful consideration and integration of all available medical, demographic, and historical patient variables.
Acknowledgments
We thank NCS Pearson Inc., the publisher of the D-KEFS, for allowing access to the standardization sample for control participants. Portions of this work were previously presented as a poster at the annual meeting of the American Academy of Clinical Neuropsychology, Chicago, IL, 2009. None of the authors have any conflict of interest to declare.
