INTRODUCTION
Traumatic brain injury (TBI) is a common condition where cerebral
compromise is incurred as the direct or indirect result of an acute
external force to the skull. Children with TBI often do well on tests of
routinized or overlearned skills whereas they tend to have significant
difficulties with learning of, and problem solving with, novel or complex
information (for reviews see Donders & Kuldanek,
1998; Yeates, 2000). The introduction of
the California Verbal Learning Test–Children's Version
(CVLT–C; Delis et al., 1994) and the
Children's Category Test (CCT; Boll, 1993)
in the mid-1990s provided the opportunity for contrasting measures of,
respectively, verbal learning and problem solving with visual materials.
These two instruments were standardized and normed on the same sample and
also offered reliable age-corrected standard scores, which allowed for
direct comparisons of respective performances. The purpose of the current
investigation was to determine the frequency of occurrence of unusually
large discrepancies between the results from both tests in a large sample
of children with TBI, and to determine if the direction of such
discrepancies was meaningfully related to injury variables.
There has been considerable research demonstrating the criterion
validity of the CVLT–C when applied to children with TBI. This test
is sensitive to severity of injury (Levin et al.,
2000; Roman et al., 1998), is strongly
predictive of educational outcome two years postinjury (Miller & Donders, 2003), and shows only partial
recovery at even more extended follow-up (Yeates et
al., 2002). There is less extensive research with the CCT but there
is some evidence that it too is affected by severity of pediatric TBI
(Hoffman et al., 2000). To our knowledge, no
previous study has explored specifically performance discrepancies between
the CVLT–C and the CCT in clinical samples. This is regrettable
because clinicians are prone to underestimate the base rates of
differences in test scores (Schinka et al.,
1998). Research with the standardization sample of these
instruments has suggested that seemingly large and statistically
significant discrepancies between the respective T scores of
these two instruments are fairly common (Donders,
1998). In that study, it was recommended on the basis of base rate
analyses that in children between the ages of 9 and 16 years, only
discrepancies greater than 16 T-score points should be
interpreted in clinical practice. Specifically, CVLT–C T
scores that exceeded the CCT T score by 17 or more points
occurred in only 9.83% of the standardization sample, whereas 10.67% of
that same sample had a discrepancy of that magnitude in the opposite
direction. In the current exploratory investigation, we planned to
investigate how often such unusually large discrepancies occurred in
children with TBI, and whether the direction of those discrepancies was
related to injury characteristics.
Performance contrasts between tests of memory and problem solving like
the CVLT–C and CCT are of interest not only for pragmatic or
psychometric reasons but also from a theoretical perspective. A number of
recent studies have reported impaired performance on other tests of memory
(Catroppa & Anderson, 2002; Lowther & Mayfield, 2004) or problem solving
(Mangeot et al., 2002; Slomine et al., 2002) after pediatric TBI but rarely
have the reasons for doing well or poorly in one domain versus
the other been explored. A recent factor analysis study found that
measures of memory and problem solving loaded on different latent
constructs (Brookshire et al., 2004). Many tests
of memory, including the CVLT–C, make significant demands on speed
of information processing, which is often affected after pediatric TBI
(Catroppa & Anderson, 2003; Donders & Warschausky, 1997), whereas many tests
of problem solving like the CCT are not only free of time constraints but
also provide frequent feedback. It has been suggested that the extensive
supports built into many of these kinds of problem-solving tasks may
actually help the child compensate for any executive impairment that
he/she has, thereby limiting ecological validity of the instruments
(Silver, 2000). For all of these reasons, an
investigation of the frequency and correlates of unusually large
discrepancies between the CVLT–C and the CCT in a clinical sample of
children with TBI appeared to be warranted.
METHOD
Research Participants
The 100 participants for this investigation were selected from a
consecutive series of original referrals to a regional Midwestern
rehabilitation hospital. Patient selection continued for approximately
7½ years until there were 100 children who met all of the following
criteria: (1) diagnosis of TBI, defined as an external blunt force to the
head with associated alteration of consciousness; (2) age between 9 and 16
years at the time of psychometric assessment; (3) absence of a premorbid
neurological, psychiatric, personal abuse, or special education history
(as ascertained from both parent report and review of medical and academic
records); and (4) assessment with the CVLT–C and CCT within 1 year
after injury. Over the time period that these data were collected, these
two tests were administered routinely to all referred children at the
organization where this study was completed, except in cases where there
were confounding factors (e.g., non-English language background, severe
uncorrected visual impairment) that could have invalidated the
child's performance. The relative influences of demographic and
neurological variables on performance on, respectively, the CVLT–C
and CCT in this sample have been described in a separate investigation
(Donders & Nesbit-Greene, 2004). Performance
contrasts between the two instruments were not addressed in that
study.
The final sample included 57 boys and 43 girls. Ethnicities, based on
parent designation, included Caucasian (n = 78), Latino
(n = 10), African-American (n = 6), and Asian-American
(n = 6). Parental occupations were coded according to the primary
wage earner in the household and included professional (n = 19),
clerical (n = 28), skilled labor (n = 40), and unskilled
labor or homemaker (n = 13). On average, children were seen for
neuropsychological assessment at 83.00 days after injury (SD =
60.79, Median = 61.50, range = 14–330), at an age of 14.15
years (SD = 2.32, Median = 14.69, range =
9.00–16.75). Most of the participants had been injured in motor
vehicle collisions, as passengers (n = 45), pedestrians
(n = 15), cyclists (n = 7), or drivers (n =
10). Other injury circumstances included falls (n = 11),
recreational accidents (n = 8), and other (n = 4).
Injury severity was defined on the basis of length of coma (i.e., time
until there was a meaningful response to verbal commands) and results from
neuroimaging of the brain in acute care. Children with mild TBI
(n = 35) had length of coma < 24 hr and no evidence for an
acute intracranial lesion on computed tomography (CT) or magnetic
resonance imaging (MRI) scan. Children with moderate-severe TBI
(n = 65) had either coma of at least 24 hr (n = 32) or
evidence for an intracranial lesion on neuroimaging (n = 63) or
both. In the latter group, mean length of coma was 2.90 days (SD
= 5.18, Median = 0.50, range = 0–25).
Materials
The CVLT–C is an individually administered test of a
child's ability to learn and remember a shopping list containing five
items from each of three semantic categories. Although numerous indexes
can be obtained from the CVLT–C, the main variable of interest for
the current investigation was the summary T score (M =
50, SD = 10), reflecting a global index of immediate free recall
over successive trials, with higher scores reflecting better performance.
In the complete sample, the mean composite T score on the
CVLT–C was 47.18 (SD = 12.82, Median = 50.00,
range 20–70) and 21% had results in the impaired range, defined as
scores at or below the 10th percentile (T ≤ 37).
The CCT is an individually administered test of a child's ability
to solve problems on the basis of corrective feedback
(“right”/“wrong”) about each attempt at
identifying the conceptual rule underlying visually presented test items.
The total number of errors on this task can be converted into a T
score (M = 50, SD = 10), with higher scores reflecting
better performance. In the complete sample, the mean composite T
score on the CCT was 46.81 (SD = 9.75, Median = 47.00,
range 22–70) and 17% scored at or below the 10th percentile
(T ≤ 37).
Procedure
The CVLT–C and CCT were administered in a standardized manner as
part of neuropsychological evaluations that were requested in the context
of rehabilitation, most often (84%) on an outpatient basis. The CCT was
administered routinely during the interval between the short-delay cued
recall and long-delay free recall trials of the CVLT–C. Evaluations
were performed only when the children were medically stable and could
recall meaningful information from day to day. A performance contrast was
calculated by subtracting the CCT T score from the CVLT–C
T score.
RESULTS
In the complete sample, the median difference between the CVLT–C
and CCT T scores was .50 points, with a range from −44 to
33. There were 11 children who had a CVLT–C T score that
was more than 16 points lower than their CCT T score (High CCT
group), and also 11 children who had a discrepancy of that magnitude in
the opposite direction (High CVLT–C group). Figure 1 presents the respective performances of these
two groups. It should be realized the term “High” is used here
only to reflect relative performance on one test as compared to the other,
and not to indicate any classification based on age-based norms.
Performance of children with unusually large contrasts on the
California Verbal Learning Test–Children's Version
(CVLT–C) and Children's Category Test (CCT).
There were no statistically significant differences between the High
CCT and High CVLT–C groups in terms of ethnicity (dichotomized as
Caucasian vs. Other), parental occupational status (dichotomized
as professional or clerical vs. skilled or
unskilled/homemaker), age, or time since injury (p > .10
for all variables). However, the High CCT group was composed exclusively
of boys whereas only 4/11 children in the High CVLT–C group were
boys, Fisher's Exact Test p < .01. In addition, only 1
child in the High CVLT–C group had a duration of coma of 1 day (with
all others having coma < 30 min) whereas length of coma equaled or
exceeded 24 hr in 7/11 children in the High CCT group, Fisher's
Exact Test p < .05, Odds Ratio = 17.50 (90% confidence
interval = 2.35–130.57).
In order to determine if the apparent disparity in distribution of
coma among the High CVLT–C and High CCT groups reflected a more
general difference between the two tests in sensitivity to injury
severity, Spearman correlations were calculated between the T
scores from the respective tests and length of coma in the complete sample
(n = 100). The correlation with length of coma was greater for
the CVLT–C (r = −.53, p < .0001) than for
the CCT (r = −.28, p < .01). This difference
was statistically significant, t = 2.46, p < .05. We
also compared the groups with moderate-severe (n = 65) and mild
(n = 35) TBI on these two psychometric variables. There was a
relatively more robust contrast in favor of the mild TBI group on the
CVLT–C, F(1,98) = 11.06, p < .001,
η2 = .10, than on the CCT, F(1,98) = 4.47,
p < .05, η2 = .04, but this difference fell
short of statistical significance, z = 1.50, p <
.10.
Results from the Wechsler Intelligence Scale for Children–Third
Edition (WISC–III; Wechsler, 1991) were
also available for all participants in the two contrast groups. These
findings are presented in Figure 2. A
multivariate analysis of variance with groups (n = 2) as the
independent variable and WISC–III factor index scores (n =
4) as the dependent variables yielded a statistically significant main
effect, F(4,17) = 3.15, p < .05. Post hoc
analyses indicated that this was accounted for exclusively by Processing
Speed, with the High CCT group performing worse than the High CVLT–C
group, F(1,21) = 12.74, p < .01. This was associated
with a large univariate effect size, η2 = 38.91.
Performance of children with unusually large contrasts on the Wechsler
Intelligence Scale for Children–Third Edition. CVLT–C =
California Verbal Learning Test–Children's Version. CCT =
Children's Category Test. VC = Verbal Comprehension. PO = Perceptual
Organization. FD = Freedom from Distractibility. PS = Processing
Speed.
Finally, we wanted to determine which factors affected the magnitude
of the discrepancy between the CVLT–C and the CCT in the complete
sample because an exclusive focus on two groups at the extreme ends of a
spectrum can sometimes be misleading. For this reason, we performed a
multiple regression analysis with the discrepancy score as the dependent
variable and the following independent variables, entered as a single
block: gender, parental occupational status, ethnicity, presence or
absence of an intracranial lesion on neuroimaging, length of coma, time
since injury, and the four WISC–III factor index scores. Because of
the restricted age range of this sample and the fact that both
CVLT–C and CCT T scores are age-corrected, we did not
include age at assessment in this analysis. The resulting findings are
presented in Table 1. Consistent with previous
recommendations (Harrell, 2001; Millis, 2003), it was decided a priori to
leave both statistically significant and nonsignificant variables in the
model in order to allow the reader to determine independently the relative
contribution of all variables in concert.
Regression model for discrepancy score between CVLT–C and CCT in
100 children with traumatic brain injury
The regression model for the discrepancy between the CVLT–C and
the CCT was statistically significant, F(10,89) = 3.20,
p < .01, adjusted R2 = .18, and
multicollinearity diagnostics were unremarkable. Inspection of Table 1 suggests that larger positive discrepancies
between the two instruments (i.e., CVLT–C T score being
higher than the CCT T score) were associated with female gender,
shorter length of coma, and better performance on the WISC–III
Verbal Comprehension index. None of the other variables were statistically
significant predictors of the discrepancy score.
DISCUSSION
The frequencies of unusually large (> 16 points) discrepancies
between the T scores from, respectively, the CVLT–C and the
CCT in this investigation were almost identical to those reported
previously for the standardization sample (Donders,
1998). Thus, pediatric TBI is not associated with an atypical rate
of discrepancies between indexes of learning and problem solving, as
assessed by these instruments. However, the direction of the discrepancy
was meaningfully related to external criterion variables. Children who did
relatively poorly on the CVLT–C as compared to the CCT were all boys
and were much more likely to have been in coma for at least 24 hr.
The current findings suggest that relatively poor performance on the
CVLT–C was more likely to be reflective of cerebral compromise (as
reflected in length of coma) than relatively poor performance on the CCT.
The former pattern was also associated with poor performance on
WISC–III Processing Speed, a variable that was strongly correlated
with length of coma (r = −.63, p < .0001). The
gender differences between the two contrasting groups are consistent with
the fact that a female advantage on the CVLT–C has also been
described in the standardization sample (Kramer et al.,
1997). In a previous study with the current participants, it was
demonstrated that coma and gender explained incremental (i.e., additive)
variance in performance on the CVLT–C in the complete sample, with
no evidence for a moderating effect and no evidence for a gender effect on
the CCT (Donders & Nesbit-Greene, 2004).
In the complete sample, performance on the WISC–III Verbal
Comprehension index also contributed to prediction of the discrepancy
between the CVLT–C and the CCT, above and beyond the effects of
gender and length of coma. Verbal Comprehension is a variable that was
largely unaffected by severity of TBI, as reflected in its very low
correlation with length of coma (r = .09, p > .10).
These findings suggest that the general magnitude of the performance
contrast between the CVLT–C and the CCT after pediatric TBI is
related to both premorbid verbal ability level and injury severity.
However, premorbid verbal strengths do not appear to play much of a role
in the emergence of performance contrasts that are, by base rate
standards, unusually large.
The findings from this investigation are limited by the utilization of
a referred convenience sample and the relatively small sizes of the
contrasting groups. However, a broad range of injury severities was
represented and the participants had been screened carefully for possible
confounding factors. In order to have data from the same test with all
children, we also limited this investigation to the older children's
version (9–16 years) of the CCT and replication with the younger
children's version (5–8 years; Boll,
1993) is still needed. Although most children in this sample had
also received other tests of memory and problem solving, these tended to
differ with the age of the children and over the course of the 7½
years of this study, so we were unable to compare systematically the
groups with large performance discrepancies on such other tests.
With these reservations in mind, the results are consistent with a
broader literature supporting the criterion validity of the CVLT–C
with regard to pediatric TBI (Levin et al.,
2000; Roman et al., 1998; Yeates et al., 2002) whereas selective impairment on
the CCT may not necessarily be reflective of acquired deficits. This
finding cannot be explained by differences in ranges of scores because
these spanned about 5 standard deviations for both instruments in this
sample. Part of the problem with the CCT may be that the component
subtests differ considerably in sensitivity to injury severity, with
subtest III being especially problematic, which may distort the composite
T score (Nesbit-Greene & Donders,
2002). The CCT also provides a considerable degree of structure and
redirection to the child, which is a problem with many purported tests of
executive functioning (Silver, 2000), whereas
the format of the CVLT–C puts the burden squarely on the examinee to
organize the information in the most efficient manner. Another factor may
be that the CCT does not make any demands on speed of performance whereas
the CVLT–C involves presentation of stimuli at a fairly rapid rate,
and speed of information processing is commonly impaired after pediatric
TBI (Donders & Warschausky, 1997; Tremont et al., 1999). In fact, there is evidence that
speed of information processing may mediate the effect of injury severity
on performance on the CVLT–C (Donders &
Nesbit-Greene, 2004).
The fact that the CVLT–C and CCT were standardized and
age-normed on the same large, representative, national sample had
potential appeal because of the allowance for direct comparisons between
the composite indexes from both instruments. However, these two tests do
not have equivalent sensitivity to severity of TBI, with criterion
validity being much more convincing for the CVLT–C. Exclusive
reliance on the CCT as a measure of novel problem solving in
neuropsychological evaluations of children with TBI may be ill-advised and
supplementation with other measures that also include a speed component
(e.g., Tower of London; Shum & Griffith,
2000) might be considered. A goal for future research is the
exploration of performance contrasts between the CVLT–C and CCT in
other clinical samples.
