INTRODUCTION
Although our understanding of concussive injuries has increased considerably over the past several decades, one area of ongoing deliberation concerns the utility of injury severity characteristics as predictors of outcome and recovery. Within the context of sports-related concussion (SRC), traditional markers of concussion severity, including loss of consciousness (LOC), retrograde amnesia (RA), and anterograde amnesia (AA), have been inconsistently associated with clinical outcome (Harmon et al., Reference Harmon, Clugston, Dec, Hainline, Herring, Kane and Poddar2019; McCrory et al., Reference McCrory, Meeuwisse, Dvorak, Aubry, Bailes, Broglio and Castellani2018). Though some studies have identified a link between these variables and concussion recovery, several others have concluded that LOC, RA, and AA are not strong predictors of outcome in this population (Iverson et al., Reference Iverson, Gardner, Terry, Ponsford, Sills, Broshek and Solomon2017). Moreover, research examining the relationship between injury severity characteristics and post-concussion neuropsychological functioning has similarly yielded inconsistent findings (Collins et al., Reference Collins, Iverson, Lovell, McKeag, Norwig and Maroon2003; Dougan et al., Reference Dougan, Horswill and Geffen2014; Teel et al., Reference Teel, Marshall, Shankar, McCrea and Guskiewicz2017), leaving open for debate the question of whether these markers of concussion severity are clinically meaningful.
Regarding the relationship between injury severity characteristics and neuropsychological outcomes, in particular, possible reasons for the equivocal findings noted above include differences in sample characteristics across studies, such as age, level of sport participation (e.g., high school, college, and professional), and timing of the post-injury evaluation (e.g., acute, sub-acute, and chronic), as well as methodological incongruities such as the neuropsychological outcomes of interest and assessment technique utilized (e.g., computerized vs. paper-and-pencil vs. hybrid approach). Additionally, the method by which neuropsychological outcomes are assessed may also be important (MacDonald et al., Reference MacDonald, Li and Bäckman2009). Though the most common approach for assessing neuropsychological functioning is examining performance using measures of central tendency (i.e., mean performance), alternative approaches for identifying cognitive dysfunction have been of increasing interest to neuropsychologists. Specifically, evaluating intra-individual variability (IIV) metrics may provide valuable information about cognitive functioning that would otherwise be missed by strictly focusing on mean performance (Costa et al., Reference Costa, Dogan, Schulz and Reetz2019; Holtzer et al., Reference Holtzer, Verghese, Wang, Hall and Lipton2008).
IIV has traditionally been conceptualized as either “inconsistency,” referring to performance fluctuations on a single task across time, or as “dispersion,” referring to performance fluctuations across different tasks within a single testing occasion (Hultsch et al., Reference Hultsch, MacDonald and Dixon2002). Both inconsistency-related IIV and dispersion-related IIV have demonstrated clinical utility across a wide range of clinical populations, but there is accumulating evidence within the neuropsychological literature to suggest that dispersion-related IIV may be especially important and may offer a more sensitive method for evaluating cognitive dysfunction relative to mean cognitive performance (Bangen et al., Reference Bangen, Weigand, Thomas, Delano-Wood, Clark, Eppig and Bondi2019; Gleason et al., Reference Gleason, Norton, Anderson, Wahoske, Washington, Umucu and Carlsson2018; Jones et al., Reference Jones, Kuhn, Mahmood, Singer, Hinkin and Thames2018). Furthermore, neuropsychologists routinely administer a wide range of tests during a single evaluation and utilize this information clinically when making diagnostic recommendations; thus, cognitive dispersion scores may be easily generated from a standard battery of tests. Although few studies within the SRC literature have examined cognitive dispersion, the small body of existing research indicates that IIV may be an important marker of cognitive dysfunction in this unique population and offer information above and beyond that of mean performance (Merritt et al., Reference Merritt, Greenberg, Guty, Bradson, Rabinowitz and Arnett2019; Rabinowitz & Arnett, Reference Rabinowitz and Arnett2013).
Given the uncertainty of the relationship between injury severity characteristics and post-concussion neuropsychological functioning, the present study sought to further explore this relationship by focusing on a particular aspect of post-concussion cognitive functioning—cognitive dispersion. Using a well-characterized sample of acutely concussed college athletes, we evaluated whether the presence of LOC, RA, and AA independently influence dispersion-related IIV following SRC. We hypothesized that athletes experiencing LOC, RA, and AA would demonstrate elevated IIV relative to those without LOC, RA, and AA.
METHOD
Procedure
Athletes participating in this study were enrolled in a sports concussion management program at a Division I university. As part of their participation in the program, athletes are tracked and monitored for concussions throughout their tenure as a college athlete per standard of care procedures. Any athlete suspected of sustaining a concussion by a team physician or athletic trainer is referred to the sports concussion management program for a post-concussion evaluation as soon as possible following injury. Referred athletes undergo a clinical interview (administered by a clinical neuropsychologist or a trained doctoral student) to confirm that a concussion was sustained and then are administered a comprehensive neuropsychological assessment (see below under “Measures” for details).
During the clinical interview, athletes are asked detailed questions about the injury for which they were referred. In particular, the interviewer asks the athlete to describe how the injury occurred (i.e., mechanism of injury) and whether the athlete was removed from play immediately following the injury. The interviewer also assesses for the presence and duration of LOC, RA, and AA. Specifically, athletes are asked whether they experienced LOC, and if so, to estimate the duration of LOC. To assess for RA and AA, athletes are asked whether they remember the injury and whether there was any memory loss for things that happened right before and right after the injury. If either of these questions are endorsed, the interviewer gathers information about the athlete’s last memory before the injury and the first memory after the injury, and asks the athlete to estimate the duration of RA and AA (as appropriate). Finally, athletes are asked to report on the presence and severity of immediate (at time of injury) and current (at time of post-concussion evaluation) symptoms. Injuries consistent with the concussion consensus criteria put forth by the Concussion in Sport Group (CISG) (McCrory et al., Reference McCrory, Feddermann-Demont, Dvorak, Cassidy, McIntosh, Vos and Tarnutzer2017, Reference McCrory, Meeuwisse, Dvorak, Aubry, Bailes, Broglio and Castellani2018) are classified as a concussion.
Neuropsychological assessment procedures were completed by trained undergraduate research assistants or doctoral students who are supervised by a clinical neuropsychologist, and clinical interviews were completed by neuropsychology doctoral students or a clinical neuropsychologist. All procedures outlined above were conducted for clinical purposes as part of the concussion management program, but we simultaneously seek permission from each athlete to use their clinical data for research purposes (to date, no athlete has declined using their data for research purposes). Post-injury evaluations were conducted individually in a quiet room and took approximately 2 h to complete. The university’s Institutional Review Board approved the research, and all athletes participating in this study provided informed consent prior to their research participation.
Participants
Participants were 111 concussed college athletes (81.1% male) evaluated, on average, 6.04 days post-concussion (SD = 5.90; Mdn = 4 days; Range = 1–26 days). Athletes included in the current study were affiliated with the following sports teams: football, basketball, lacrosse, soccer, ice hockey, wrestling, and rugby. In order to be included in the present study, the following criteria were applied: (a) participation in the sports concussion management program described above; (b) sustained a concussive event consistent with CISG consensus criteria (McCrory et al., Reference McCrory, Feddermann-Demont, Dvorak, Cassidy, McIntosh, Vos and Tarnutzer2017, Reference McCrory, Meeuwisse, Dvorak, Aubry, Bailes, Broglio and Castellani2018); (c) completed a post-injury evaluation acutely following injury, but no later than 1-month post-injury; (d) demonstrated adequate performance on performance validity tests (PVTs; see below under “Measures” for details); and (e) completed the clinical interview in its entirety (i.e., sufficient data were available from the clinical interview to determine injury severity characteristics). In total, 169 participants were involved in the sports concussion management program and had sustained a concussion; of these, 19 were excluded due to the timing of their post-concussion evaluation, another 23 were excluded due to inadequate performance on PVTs (i.e., athlete failed one or more PVTs), and another 16 were excluded due to having incomplete data on the clinical interview.
Measures
All athletes were administered a comprehensive neuropsychological assessment that encompassed both computerized and paper-and-pencil tests, spanning the domains of attention, processing speed/reaction time, executive functioning, and memory (see Merritt et al., Reference Merritt, Greenberg, Guty, Bradson, Rabinowitz and Arnett2019 for more details regarding the specific cognitive tests used). Computerized measures included the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) and the Vigil/W Continuous Performance Test. Paper-and-pencil measures included the Symbol-Digit Modalities Test (SDMT), Penn State University (PSU) Cancellation Test, a modified version of the Digit Span subtest from the Wechsler Adult Intelligence Scale – Third Edition (WAIS-III), Comprehensive Trail-Making Test (CTMT), Stroop Color-Word Test (SCWT), Brief Visuospatial Memory Test-Revised (BVMT-R), and Hopkins Verbal Learning Test-Revised (HVLT-R).
In total, 18 variables were derived from the above measures and were used to generate the IIV indices, our primary outcomes of interest. Prior to computing the IIV indices, the 18 neuropsychological variables were converted from raw scores to standard scores using sex-specific normative data generated from a sample of college athletes evaluated at baseline (Merritt et al., Reference Merritt, Meyer, Cadden, Roman, Ukueberuwa, Shapiro and Arnett2017). Converting all raw scores to standard scores ensured that all variables could be interpreted on the same scale and in a uniform direction (i.e., higher standard score values reflect better performance). After verifying that all raw neuropsychological variables were converted to standard scores, two IIV indices were computed using similar procedures as described in Merritt et al. (Reference Merritt, Greenberg, Guty, Bradson, Rabinowitz and Arnett2019)—an intra-individual standard deviation (ISD) score and a range, or maximum discrepancy (MD), score. Briefly, the standard deviation of the 18 standard scores was used to compute the ISD score for each athlete, and the difference between each athlete’s highest and lowest score represented the MD score. Both the ISD and MD scores were calculated so that higher scores reflect greater cognitive variability.
In addition to creating the IIV variables, we also derived two additional neuropsychological summary scores—a mean cognitive composite score and a cognitive impairment score. The mean cognitive composite score was calculated by computing the average of the 18 neuropsychological variables, and the cognitive impairment score was calculated by totaling the number of impaired test scores (defined as any score falling >1.5 SD below the mean) across the test battery.
Finally, the Wechsler Test of Adult Reading (WTAR) was administered to all athletes to assess premorbid intellectual functioning, and performance validity was assessed using manual-defined cutoffs (Lovell, Reference Lovell2016) for the ImPACT Impulse Control Composite as well as four embedded validity indices from the ImPACT including Design Memory Learning Percent Correct, Three Letters Total Letters Correct, X’s and O’s Total Incorrect, and Word Memory Learning Percent Correct.
Approach to Data Analysis
Data analyses were conducted using the Statistical Package for the Social Sciences (SPSS; Version 26). Independent variables included LOC, RA, and AA status (presence [+] vs. absence [-]) and dependent variables included the two IIV indices (ISD and MD scores). Descriptive statistics were computed for the overall sample, and independent samples t-tests and chi-square or Fisher’s exact tests were used to compare LOC+/LOC−, RA+/RA−, and AA+/AA− groups across basic sociodemographic variables, injury characteristics, and cognitive summary indices (i.e., the mean cognitive composite score and the cognitive impairment score). To determine potential confounding variables for our main analyses, correlations were conducted to evaluate relationships between independent variables, dependent variables, and pertinent sample characteristics. Analyses of covariance (ANCOVAs) were then used to assess whether athletes with and without LOC, RA, and AA differed with regard to ISD and MD scores following injury. Effect sizes are reported as partial eta-squared values (η p 2).Footnote 1
RESULTS
Sample Characteristics
The overall sample included 111 concussed college athletes; of these, 19 (17.1%) athletes experienced LOC, 19 (17.1%) experienced RA, and 37 (33.3%) experienced AA. Further evaluation of the sample revealed the following: in total, 4 (3.6%) athletes experienced only LOC (no RA or AA); 6 (5.4%) experienced only RA (no LOC or AA); 19 (17.1%) experienced only AA (no LOC or RA); 4 (3.6%) experienced LOC and RA (no AA); 9 (8.1%) experienced LOC and AA (no RA), 7 (6.3%) experienced RA and AA (no LOC); 2 (1.8%) experienced LOC, RA, and AA; and 60 (54.1%) experienced no LOC, RA, or AA. Table 1 displays overall sample characteristics, and data are also presented as a function of LOC, RA, and AA group status.
Table 1. Sample characteristics
Abbreviations: LOC = loss of consciousness; RA = retrograde amnesia; AA = anterograde amnesia; WTAR FSIQ = Wechsler Test of Adult Reading Full Scale IQ; ADHD = attention-deficit/hyperactivity disorder; LD = learning disorder.
Correlation Analyses
Correlation analyses showing relationships between independent variables, dependent variables, and sample characteristics are presented in Table 2. Given the significant associations with our independent and dependent variables, time since injury and sex were used as covariates in our main analyses.
Table 2. Correlations of independent variables, dependent variables, and sample characteristics
Abbreviations: LOC = loss of consciousness; RA = retrograde amnesia; AA = anterograde amnesia; ISD = intra-individual standard deviation; MD = maximum discrepancy. Notes: *p < .05; **p < .01; ***p ≤ .001.
Main Analyses
An ANCOVA adjusting for time since injury and sex revealed a significant effect of LOC on the ISD score (F(1, 107) = 5.72, p = .018, η p 2 = .051). Specifically, athletes with LOC (M = 16.35; SD = 5.86) displayed significantly greater IIV than athletes without LOC (M = 13.50; SD = 4.00). Similarly, an ANCOVA adjusting for time since injury and sex revealed a significant effect of LOC on the MD score (F(1, 107) = 4.60, p = .034, η p 2 = .041); again, athletes with LOC (M = 64.38; SD = 30.17) demonstrated significantly greater IIV than athletes without LOC (M = 51.94; SD = 18.56). Adjusted means and standard errors associated with the ISD and MD scores for LOC are displayed in Figures 1a and 1b, respectively.
Fig. 1. Intra-individual variability (IIV) scores: (A) Intra-individual standard deviation (ISD) scores for those who did and did not experience loss of consciousness (LOC). (B) Maximum discrepancy (MD) scores for those who did and did not experience LOC. (C) ISD scores for those who did and did not experience retrograde amnesia (RA). (D) MD scores for those who did and did not experience RA. (E) ISD scores for those who did and did not experience anterograde amnesia (AA); and (F) MD scores for those who did and did not experience AA. All scores are displayed as adjusted means and standard errors. *p < .05.
In contrast, no significant differences were found on the ISD score (F(1, 107) = 0.03, p = .862, η p 2 < .001) when comparing athletes who did (M = 13.71; SD = 4.60) and did not (M = 14.05; SD = 4.47) experience RA. Additionally, the MD score also did not significantly differ between those who did (M = 50.49; SD = 20.43) and did not (M = 54.81; SD = 21.57) experience RA (F(1, 107) = 0.43, p = .512, η p 2 = .004) when adjusting for time since injury and sex. Figures 1c and 1d present adjusted means and standard errors associated with the ISD and MD scores for RA, respectively.
Finally, no significant differences were found on the ISD score (F(1, 107) = 0.16, p = .687, η p 2 = .002) when comparing athletes who did (M = 14.17; SD = 4.97) and did not (M = 13.90; SD = 4.24) experience AA. Similarly, the MD score also did not significantly differ between those who did (M = 56.14; SD = 23.53) and did not (M = 53.04; SD = 20.26) experience AA (F(1,107) = 0.49, p = .487, η p 2 = .005) when adjusting for time since injury and sex. Figures 1e and 1f present adjusted means and standard errors associated with the ISD and MD scores for AA, respectively.
Secondary Analyses
Given the significant relationships (see Table 2) between the IIV indices (ISD and MD scores) and the neuropsychological summary scores (mean cognitive composite and cognitive impairment score), we conducted follow-up analyses to control for these additional covariates. Specifically, ANCOVAs adjusting for time since injury, sex, and the mean cognitive composite were used to compare the LOC, RA, and AA groups on the IIV indices. Consistent with the above results, ANCOVAs revealed a significant effect of LOC status on the ISD and MD scores, such that athletes with LOC displayed significantly greater IIV than athletes without LOC (p’s = .021–.044; η p 2 = .038–.049), and no significant differences were found on the IIV indices when comparing athletes who did and did not experience RA and AA (p’s = .499–.920; η p 2 = .000–.004). Finally, ANCOVAs adjusting for time since injury, sex, and the cognitive impairment score similarly showed that IIV indices significantly differed as a function of LOC status (p’s = .008–.023; η p 2 = .048-.065) but not RA or AA status (p’s = .553–.955; η p 2 = .000–.003).
DISCUSSION
Prior research examining the relationship between injury severity characteristics (i.e., LOC, RA, and AA) and neuropsychological functioning has yielded inconsistent results. The present study, therefore, sought to further explore this relationship by using a novel method to examine post-concussion cognitive functioning—across-test intra-individual variability (IIV). Our findings showed that LOC, but not RA or AA, was associated with greater variability, or inconsistencies, in cognitive performance following concussion. Our results suggest that LOC is uniquely associated with IIV and that this particular marker of injury severity may be a contributing factor of less efficient post-concussion cognitive functioning.
Until recently, cognitive performance following SRC has primarily been evaluated using measures of central tendency (i.e., mean performance). However, as measures of across-test IIV, or cognitive dispersion, become more readily utilized in other clinical populations (Bangen et al., Reference Bangen, Weigand, Thomas, Delano-Wood, Clark, Eppig and Bondi2019; Gleason et al., Reference Gleason, Norton, Anderson, Wahoske, Washington, Umucu and Carlsson2018; Hilborn et al., Reference Hilborn, Strauss, Hultsch and Hunter2009; Holtzer et al., Reference Holtzer, Verghese, Wang, Hall and Lipton2008; Jones et al., Reference Jones, Kuhn, Mahmood, Singer, Hinkin and Thames2018), researchers have started applying these novel metrics to SRC samples (Merritt et al., Reference Merritt, Greenberg, Guty, Bradson, Rabinowitz and Arnett2019; Rabinowitz & Arnett, Reference Rabinowitz and Arnett2013). Our current results add to this burgeoning SRC-IIV literature by establishing a potentially important relationship between the presence of LOC and increased cognitive variability acutely following injury. Importantly, past studies evaluating mean cognitive performance have concluded that LOC does not appear to be strongly associated with cognitive functioning (Collins et al., Reference Collins, Iverson, Lovell, McKeag, Norwig and Maroon2003; Teel et al., Reference Teel, Marshall, Shankar, McCrea and Guskiewicz2017), and our analyses similarly showed that mean cognitive performance—as measured by a cognitive composite score—was not significantly associated with LOC status. However, LOC was associated with cognitive IIV, indicating that the method by which cognition is evaluated matters. It is thus possible that in the context of SRC, where there is not a typical cognitive profile, IIV indices may offer a more nuanced representation of cognitive functioning relative to mean performance. This could explain the discrepancy between our current findings and several previous studies that failed to find associations between LOC and cognition.
We also evaluated cognitive impairment across the test battery and showed that impairment scores were not associated with LOC or other injury severity variables, further supporting the notion that IIV is a unique cognitive construct and that there may be an important relationship between experiencing LOC and elevated cognitive dispersion acutely after injury. Notably, there is mounting evidence to suggest that elevated IIV is associated with underlying brain changes and may reflect central nervous system disturbance or disruption (MacDonald et al., Reference MacDonald, Li and Bäckman2009; Troyer et al., Reference Troyer, Vandermorris and Murphy2016; Vandermorris & Tan, Reference Vandermorris and Tan2015). While more research on LOC and IIV is needed to better understand the mechanisms involved in this relationship, it is possible that LOC momentarily disrupts central nervous system activity, which results in acute disruptions to cognitive functioning. Clinically, this may be observed as greater fluctuations in cognitive performance across tests and interpreted as less efficient cognitive functioning. Future studies using longitudinal data are needed to evaluate this possibility and determine the time course associated with elevated IIV following SRC.
Limitations
There are limitations associated with this study that should be considered when interpreting our findings. First, data for this study were collected as part of a clinically based concussion management program. As such, we do not manage when concussion referrals are made. Although team physicians are encouraged to refer concussed athletes for testing as soon as possible following injury, referrals may be delayed for reasons outside our control. Nevertheless, we statistically controlled for time since injury in our analyses to mitigate concerns associated with how time since injury may influence findings. Second, LOC, RA, and AA were determined based on self-report data gathered during the clinical interview. As with any data collected via self-report, the information is subject to recall bias, which is an inherent limitation when evaluating injury severity variables. Third, we specifically examined across-test intra-individual variability (i.e., cognitive dispersion) acutely following injury; we recognize that if other types of variability were examined (for example, inconsistency-related IIV or reaction time variability), results could have been different. Relatedly, using other neuropsychological measures or including greater or fewer test variables in the IIV calculations may also influence the cognitive dispersion scores and ultimate study conclusions. We also did not evaluate how post-concussion symptoms may interact with study variables. More research is necessary to understand what impact these decision points have on IIV metrics. Finally, given that our sample was composed of concussed college athletes, it is possible that our findings may not generalize to other populations susceptible to concussive injuries (e.g., older or younger samples, military populations, etc.).
Conclusions
Although the prognostic utility of injury severity markers such as LOC, RA, and AA has been questioned, the present study established that LOC may be a particularly important marker of cognitive dysfunction acutely following injury. Future research with larger samples is necessary to verify these findings, but our results suggest that LOC may be associated with less efficient post-concussion cognitive functioning and may help detect athletes at risk for poor clinical outcomes acutely following SRC.
Acknowledgments
The authors would like to thank Wayne Sebastianelli and Penn State Sports Medicine for their generous support of our research. They would also like to thank the athletes who participated in the sports concussion management program. No funding was specifically obtained for this study. However, Victoria Merritt received salary support during this work from a Career Development Award [IK2 CX001952] from the VA Clinical Science Research & Development Service.
CONFLICT OF INTEREST
The authors have nothing to disclose.
