INTRODUCTION
It is well established that adult ageing is characterized behaviorally by increased intraindividual variability in cognitive function. As the present review will show, such variability is not only a feature of normal ageing, but becomes more marked in the presence of neuropathology or impending mortality. It is likely that this widely observed behavioral characteristic of ageing reflects greater neurobiological variability stemming from compromised central nervous system integrity. Given the proposed neurobiological underpinnings of IIV, the aim of the present systematic review, therefore, was to critically evaluate the empirical literature using this marker to predict cognitive change in normal ageing, and also age-related neuropathological outcomes including mild cognitive impairment (MCI), dementia, Parkinson’s disease, and, indeed, death. First, we describe some of the characteristics of this behavioral marker before detailing theoretical and empirical linkage to potential underlying neurobiological variability.
Behavioral Intraindividual Variability
IIV refers to the within-person variation in cognitive performance. It is measured over relatively short periods using trial-to-trial fluctuations in reaction time (RT) on a given task, through repeated assessments over longer periods such as days or weeks, or across a battery of different cognitive tasks measured in the same session (Hultsch, MacDonald, & Dixon, Reference Hultsch, MacDonald and Dixon2002). The most widely used measure of IIV is RT variability, which provides the focus for the present review. Often, researchers have implicitly treated RT variability as random noise or error variance and collapsed trials across a cognitive task to obtain measures of central tendency (e.g., mean or median RT).
However, theorists have suggested that this variation is systematic and that the measure may capture meaningful information about higher-order cognitive processes such as fluctuating attentional or executive control mechanisms (Bunce, MacDonald, & Hultsch, Reference Bunce, MacDonald and Hultsch2004; Bunce, Warr, & Cochrane, Reference Bunce, Warr and Cochrane1993; West, Murphy, Armilio, Craik, & Stuss, Reference West, Murphy, Armilio, Craik and Stuss2002). Moreover, research typically shows greater IIV to increase across the adult lifespan (Dykiert, Der, Starr, & Deary, Reference Dykiert, Der, Starr and Deary2012; Hultsch et al., Reference Hultsch, MacDonald and Dixon2002; Williams, Hultsch, Strauss, Hunter, & Tannock, Reference Williams, Hultsch, Strauss, Hunter and Tannock2005). However, although it appears that IIV increases with normal ageing, due to the cross-sectional nature of many studies, the possibility of cohort effects cannot be ruled out.
Neuropathology and Brain Substrates of Intraindividual Variability
There is considerable empirical support for the suggestion that IIV reflects underlying neurobiological integrity (Hultsch, MacDonald, Hunter, Levy-Bencheton, & Strauss, Reference Hultsch, MacDonald, Hunter, Levy-Bencheton and Strauss2000; Hultsch, Strauss, Hunter, & MacDonald, Reference Hultsch, Strauss, Hunter and MacDonald2008). For example, cross-sectional studies show that relative to healthy older persons, IIV is greater in individuals living with a range of neurodegenerative disorders including Parkinson’s disease (e.g., de Frias, Dixon, Fisher, & Camicioli, Reference de Frias, Dixon, Fisher and Camicioli2007), MCI (e.g., Christensen et al., Reference Christensen, Dear, Anstey, Parslow, Sachdev and Jorm2005; Dixon et al., Reference Dixon, Lentz, Garrett, MacDonald, Strauss and Hultsch2007), and dementia (e.g., Gorus, De Raedt, Lambert, Lemper, & Mets, Reference Gorus, De Raedt, Lambert, Lemper and Mets2008; Hultsch et al., Reference Hultsch, MacDonald, Hunter, Levy-Bencheton and Strauss2000), including early stage Alzheimer’s disease (Duchek et al., Reference Duchek, Balota, Tse, Holtzman, Fagan and Goate2009).
However, given these findings of greater variability in the presence of age-related neuropathology, what evidence is there of associations between IIV and brain substrates? Several magnetic resonance imaging (MRI) studies have identified associations between greater behavioral variability and poorer neuroanatomical integrity as shown by reduced white matter volume (Jackson, Balota, Duchek, & Head, Reference Jackson, Balota, Duchek and Head2012; Walhovd & Fjell, Reference Walhovd and Fjell2007), increased white matter hyperintensity volume (Bunce et al., Reference Bunce, Anstey, Cherbuin, Burns, Christensen, Wen and Sachdev2010, Reference Bunce, Anstey, Christensen, Dear, Wen and Sachdev2007), and diffusion tensor imaging metrics such as fractional anisotropy (Deary et al., Reference Deary, Bastin, Pattie, Clayden, Whalley, Starr and Wardlaw2006; Fjell, Westlye, Amlien, & Walhovd, Reference Fjell, Westlye, Amlien and Walhovd2011; Mella, de Ribaupierre, Eagleson, & de Ribaupierre, Reference Mella, de Ribaupierre, Eagleson and de Ribaupierre2013; Moy et al., Reference Moy, Millet, Haller, Baudois, de Bilbao, Weber and Delaloye2011).
A plausible explanation for increases in IIV across the lifespan concerns the possibility that age-related dopamine depletion reduces neural signal-to-noise thereby affecting the efficiency of brain connectivity (Li, Lindenberger, & Sikstrom, Reference Li, Lindenberger and Sikstrom2001). Due to the hypothesized more intermittent signaling, variability increases, a possibility that has received support from MRI work (e.g., MacDonald, Karlsson, Rieckmann, Nyberg, & Backman, Reference MacDonald, Karlsson, Rieckmann, Nyberg and Backman2012). Additionally, with the presence of amyloid, neurofibrillary tangles and the deterioration of white and gray matter structures that accompany the advance of age-related neuropathology, the degree of faulty signaling and connectivity is likely to increase, with further increases in variability. Taken together, this behavioral and neuroimaging research provides evidence that supports the proposal that IIV is a sensitive indicator of neurobiological integrity.
As well as indicating existing neuropathology, measures of IIV may also identify individuals who are at risk of future cognitive impairment, as increased IIV may be indicative of subthreshold impairment before broader cognitive decline. In addition, increased IIV may also predict mortality as there has long been evidence for accelerated cognitive deterioration in proximity to death (Riegel & Riegel, Reference Riegel and Riegel1972). Given the weight of evidence, there is a pressing need to identify individuals who are at risk of adverse outcomes as this may allow interventions to target those most likely to benefit. Moreover, there is evidence that treatments for conditions such as Alzheimer’s disease may be more beneficial if implemented early in the disease process. Therefore, in additional to considering changes in variability over time, this review also focuses on longitudinal studies that investigate the relationship between baseline IIV and important health-related outcomes such as cognitive impairment, dementia, or mortality.
Do IIV Measures Possess Unique Properties?
Several methods have been used to compute IIV across RT trials, the most basic of which is the raw intraindividual SD. Other measures take into account systematic variance associated with influences such as time on task and experimental condition (e.g., adjusted intraindividual SD), or adjust for mean level of responding (e.g., the coefficient of variation: intraindividual SD/intraindividual mean RT). Occasionally, investigators have also used metrics that reflect the typically non-normal distribution of RTs such as the interquartile range, or fitting the ex-Gaussian distribution. This produces three metrics, mu, sigma, and tau, with the latter indexing intermittently slower responses that fall into the tail of the RT distribution.
There is typically a high correlation between measures of variability (e.g., intraindividual standard deviation) and central tendency (e.g., mean-RT) taken from the same cognitive task. It has been argued that increases in IIV associated with age (Myerson, Robertson, & Hale, Reference Myerson, Robertson and Hale2007) and mild cognitive impairment (Phillips, Rogers, Haworth, Bayer, & Tales, Reference Phillips, Rogers, Haworth, Bayer and Tales2013) reflect a general slowing of responses. By contrast, there is evidence that IIV provides greater differentiation in identifying persons with cognitive impairment (e.g., Dixon et al., Reference Dixon, Lentz, Garrett, MacDonald, Strauss and Hultsch2007), and it has been shown that IIV is associated with brain imaging metrics such as white matter hyperintensites in early old age whereas mean-RT is not (e.g., Bunce et al., Reference Bunce, Anstey, Christensen, Dear, Wen and Sachdev2007). Therefore, a further consideration for this review is whether IIV measures predict outcome over and above associations accounted for by measures of central tendency.
Another important consideration is whether there is evidence that IIV adds to the predictive power of commonly used neuropsychological assessment measures. As such tasks are sensitive to both normal and neuropathological ageing, it is important to establish if IIV provides unique information that may help identification of persons at risk of adverse health outcomes. Finally, we explored whether findings varied systematically according to methodological differences across studies, such as task complexity, the number of RT trials, and how IIV was computed. As there is currently little consensus as to the optimum IIV measure, we considered if there was evidence favoring a particular type of IIV metric.
METHODS
Studies were included in the present review if they appeared in an English language peer-reviewed journal, had a longitudinal design, used IIV measures at baseline, and the sample had a baseline mean age >55 years. This cutoff was selected to restrict the review to ageing research and also encompass early old-age. Studies were excluded if they were cross-sectional or the follow-up period was less than 1 year, as we were interested in longer-term change. Studies were also excluded if they did not have measures of RT IIV (i.e., they only reported accuracy or mean-RT, measured IIV across a battery of cognitive tasks, or investigated IIV in non-cognitive domains). Finally, as a systematic review of the association between IIV, falls, and gait disturbance has been published recently by our group (Graveson, Bauermeister, McKeown, & Bunce, Reference Graveson, Bauermeister, McKeown and Bunce2016), we excluded studies that had these outcomes.
Literature searches were completed on May 17, 2016, using the following electronic databases: EMBASE (from 1947, OVID interface), MEDLINE (from 1946, OVID interface), PsycINFO (from 1806, OVID interface), and Web of Science (from 1900). Three descriptors were used. First, IIV and common variants (“IIV”, “Intraindividual variability”, “intra-individual variability”, “within person variability”, “reaction time variability”, “response time variability”, “RT variability”, “reaction time inconsistency”, “response time inconsistency”, “RT inconsistency”), second, longitudinal research (longitudinal, “cohort stud*”, “cohort design”, “follow up stud*”, “follow up design”, “prospective stud*”, “prospective research”, “prospective design”, “retrospective stud*”, “retrospective research”, “retrospective design”), and third, terms relating to ageing or potential longitudinal outcomes (“older adults”, “aging”, “ageing”, “neurocognitive disorder”, “neurodegenerative disease”, “dementia”, “Alzheimer’s disease”, “mild cognitive impairment”, “Parkinson’s disease”, “Huntington’s disease”, “lewy body disorder”, “frontotemporal lobar degeneration”, “mortality”, “death”).
All descriptors were searched for in keywords, title and abstract, and for the searches using EMBASE, MEDLINE, and PsycInfo, Medical Subject Heading (MeSH) terms were also used where available. Additionally, citations in shortlisted papers and related review articles were inspected for further relevant studies.
Investigations were assessed according to the Quality in Prognostic Studies framework (Hayden, Cote, & Bombardier, Reference Hayden, Cote and Bombardier2006; Hayden, van der Windt, Cartwright, Cote, & Bombardier, Reference Hayden, van der Windt, Cartwright, Cote and Bombardier2013), which evaluates studies for bias and quality according to six criteria: Study participation, study attrition, prognostic factor measurement, outcome measurement, study confounding, and statistical analysis and reporting. Consensus agreement was reached for each investigation. Where studies used overlapping samples or produced conflicting results, we drew our conclusions taking into account these criteria. (Details of this evaluation are available from the corresponding author.)
RESULTS
The literature search initially identified 688 studies, of which 22 met the inclusion criteria (see Figure 1 for a flow diagram of the study selection process). After exclusions, there were nine studies that investigated change in variability over time and 17 studies that assessed whether IIV was associated with subsequent outcome (see Table 1 for details). Identified outcomes were cognitive change, conversion to mild cognitive impairment or dementia, and mortality. Three studies had a sample mean age <55 years (Bielak, Cherbuin, Bunce, & Anstey, Reference Bielak, Cherbuin, Bunce and Anstey2014; Deary & Der, Reference Deary and Der2005b; Shipley, Der, Taylor, & Deary, Reference Shipley, Der, Taylor and Deary2006), but as these all reported analyses stratified by age group, data from the older group was included in this review.
Fig. 1 Flow diagram of the study selection process.
Table 1 Summary of studies included in the review
a Longitudinal sample size; (%) refers to percentage of baseline sample included in analyses.
b Age at baseline
c Logical memory, digit span forward and backward, Trail Making Test parts A and B, WAIS-R information, WIAS-R block design, WAIS-R digit symbol, Benton delay, Benton copy, Boston Naming Test, crossing off, mental control, associate recall, letter fluency.
d MMSE, immediate and delayed recall, digit span backwards, spot-the-word, symbol digit modality test.
e Logical memory - delayed, Rey auditory verbal learning test- delayed, category fluency, coding, WAIS-R block design, Benton visual retention, Trail Making Test parts A and B, Boston Naming Test, letter fluency
f Working memory (sentence construction, listening span, computation span), episodic memory (word list recall, story recall), semantic memory (world fact recall, recognition vocabulary)
BDNF= Brain-Derived Neurotropic Factor; CDR=clinical dementia rating; CH=cognitively healthy; 95% CI=95% confidence interval; COMT= Catechol-O-Methyltransferase; CRT=complex reaction time; CV=coefficient of variation; HR=hazard ratio; IIV=intraindividual variability; ISD=intraindividual standard deviation; IQR=interquartile range; MCI=Mild Cognitive Impairment; MMSE=Mini Mental State Examination; PD=Parkinson’s disease; SD=standard deviation; SRT=simple reaction time; TMT=trailmaking test.
Although we identified 22 studies that met our inclusion criteria, several of these involved samples drawn from the same study population. Specifically, the PATH Through Life Study (Bielak et al., Reference Bielak, Cherbuin, Bunce and Anstey2014; Das et al., Reference Das, Tan, Bielak, Cherbuin, Easteal and Anstey2014), Project MIND (Bielak, Hultsch, Strauss, MacDonald, & Hunter, Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010a, Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; Grand, Stawski, & MacDonald, Reference Grand, Stawski and MacDonald2016; Yao, Stawski, Hultsch, & MacDonald, Reference Yao, Stawski, Hultsch and MacDonald2016), the Victoria Longitudinal Study (Bielak, Hughes, Small, & Dixon, Reference Bielak, Hughes, Small and Dixon2007; MacDonald, Hultsch, & Dixon, Reference MacDonald, Hultsch and Dixon2003, Reference MacDonald, Hultsch and Dixon2008; Whitehead, Dixon, Hultsch, & MacDonald, Reference Whitehead, Dixon, Hultsch and MacDonald2011), the West of Scotland Twenty-07 Study (Deary & Der, Reference Deary and Der2005a, Reference Deary and Der2005b), while two further studies used identical participants (Bayer et al., Reference Bayer, Phillips, Porter, Leonards, Bompas and Tales2014; Tales et al., Reference Tales, Leonards, Bompas, Snowden, Philips, Porter and Bayer2012). The majority of these investigations fall into different sections below. However, where there was overlap, as noted earlier, we used formalized evaluation criteria in considering the findings.
In the following sections, first, we review studies of longitudinal change in IIV as a function of age, and then we turn to studies in which IIV serves as a predictor of future outcome. We then considered whether IIV was predictive of outcome over and above mean RT derived from the same task, and assess IIV relative to traditional neuropsychological tasks. Finally, we considered whether methodological differences across studies influenced the findings.
Longitudinal Change in Variability
With the exception of a study in Parkinson’s disease patients (de Frias, Dixon, & Camicioli, Reference de Frias, Dixon and Camicioli2012), all investigations drew participants from large-scale population-based studies of ageing. The majority of studies found that IIV increased over time (Bielak et al., Reference Bielak, Cherbuin, Bunce and Anstey2014; Deary & Der, Reference Deary and Der2005b; Lovden, Li, Shing, & Lindenberger, Reference Lovden, Li, Shing and Lindenberger2007; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003). This was consistently shown for old-old individuals (Lovden et al., Reference Lovden, Li, Shing and Lindenberger2007; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003), and there was evidence that the rate of change increased above 75 years of age (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b), although it was less clear at what age increases in IIV start. Two studies found significant 8-year increases in IIV for individuals who were aged 55 to 59 years (Deary & Der, Reference Deary and Der2005b) or 60 to 64 years (Bielak et al., Reference Bielak, Cherbuin, Bunce and Anstey2014) at baseline.
By contrast, MacDonald and colleagues (Reference MacDonald, Hultsch and Dixon2003) only found 6-year increase in IIV for participants aged over 75 years, with no increase evident for those aged 55–64 or 65–74 years. In another study, IIV did not increase over time in cognitively intact adults aged 65–84 years (de Frias et al., Reference de Frias, Dixon, Fisher and Camicioli2007). However, methodological differences may underlie this result as this study had a smaller sample, shorter follow-up duration (1.5 years), and only two assessment points compared with three or more in the other studies.
Additional factors influencing change in IIV that have been investigated included lifestyle activities (Bielak et al., Reference Bielak, Hughes, Small and Dixon2007), type II diabetes (Whitehead et al., Reference Whitehead, Dixon, Hultsch and MacDonald2011), and genetics (Das et al., Reference Das, Tan, Bielak, Cherbuin, Easteal and Anstey2014). However, results were not conclusive regarding the impact of these factors. For example, increasing IIV was associated with decreases in passive lifestyle activities such as reading newspapers and watching television, and novel activities including playing Bridge or completing a tax return, but not with physical or social activity, travel, and self-maintenance (Bielak et al., Reference Bielak, Hughes, Small and Dixon2007). Also, this result was found only for the most complex RT task, suggesting that the benefits of an engaged lifestyle are more likely to appear in more cognitively demanding tasks. Conversely, however, the influence of type II diabetes on increasing variability over time was only evident in less demanding RT tasks (Whitehead et al., Reference Whitehead, Dixon, Hultsch and MacDonald2011). Lastly, although baseline IIV was influenced by genetic variants of catechol-O-methyltransferase (COMT) and brain-derived neurotropic factor (BDNF), neither genotype primary effects nor interactions involving genotype affected the trajectory of change in IIV (Das et al., Reference Das, Tan, Bielak, Cherbuin, Easteal and Anstey2014).
Associations Between Variability and Outcome
A total of 17 studies were identified that assessed relations between variability and outcomes that included cognitive change, mild cognitive impairment or dementia, and mortality. Of these, four studies also reported results for change in IIV over time and were described in the preceding section (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; de Frias et al., Reference de Frias, Dixon and Camicioli2012; Lovden et al., Reference Lovden, Li, Shing and Lindenberger2007; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003).
Cognitive change
Six studies considered whether baseline IIV was related to change in cognitive performance in community-dwelling adults (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; Ghisletta, Fagot, Lecerf, & De Ribaupierre, Reference Ghisletta, Fagot, Lecerf and De Ribaupierre2013; Grand et al., Reference Grand, Stawski and MacDonald2016; Lovden et al., Reference Lovden, Li, Shing and Lindenberger2007; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003; Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016). In an early study, adjusting for initial IIV accounted for cognitive change in several tasks including processing speed, memory and language (MacDonald et al., Reference MacDonald, Hultsch and Dixon2003). Similarly, Lovden and colleagues (2007) found that higher trial-to-trial variability both predicted and temporally preceded cognitive decline in verbal fluency and perceptual speed, whereas conversely, initial cognitive performance had a negligible influence on decline in IIV.
Three additional studies investigated IIV and cognitive change using the same dataset, with a follow-up duration of three (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b) or 6 years (Grand et al., Reference Grand, Stawski and MacDonald2016; Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016). In the first study, baseline IIV predicted the rate of cognitive change on tasks involving processing speed, fluency, reasoning, memory, and language (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b). More recent analyses over 6 years confirmed that baseline IIV moderated the rate of cognitive change, but only for memory and vocabulary and not processing speed, reasoning or fluency tasks (Grand et al., Reference Grand, Stawski and MacDonald2016). In addition, the rate of cognitive change increased with proximity to participant attrition, and higher IIV was associated with greater decline per year closer to attrition for memory and executive function (Trailmaking B: Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016).
A final study investigated the amplitude of fluctuations in IIV and temporal dependency of responses (the relationship of each RT to the RTs that immediately preceded it). While the temporal dependency did not predict cognitive change (2-year follow-up adjusting for baseline scores), the amplitude of fluctuation in IIV was associated with change in fluid intelligence, but not with change in crystallized intelligence (Ghisletta et al., Reference Ghisletta, Fagot, Lecerf and De Ribaupierre2013). Taken together, these results suggest that baseline IIV predicts cognitive change across multiple cognitive domains, particularly for tasks assessing fluid abilities.
If fluctuations in RT are a marker for cognitive decline, there is a likelihood of covariation between change in IIV and change in cognitive performance. Consistent with this prediction, three of the aforementioned studies found significant covariation over time between IIV and cognition over 1 (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b), 2 (Lovden et al., Reference Lovden, Li, Shing and Lindenberger2007) and 3 years (MacDonald et al., Reference MacDonald, Hultsch and Dixon2003). That is, IIV was inversely related to cognitive performance over time (higher IIV associated with lower cognitive performance), and greater baseline variability was related to more marked subsequent cognitive decline. In addition, when age was treated as a source of between-subject variance, it did not influence the covariation relationship (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003), suggesting the association between IIV and cognitive ability is consistent across the older adult age range.
Mild cognitive impairment and dementia
Seven studies investigated whether IIV predicted conversion to cognitive impairment or dementia over 1.5- to 10-year periods. All found that baseline IIV for at least one measure, was associated with outcome. In two large-scale population-based studies, higher initial IIV was associated with an increased risk of mild cognitive impairment over 4- (Cherbuin, Sachdev, & Anstey, Reference Cherbuin, Sachdev and Anstey2010) and 5-year (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010a) intervals. However, in the latter study, IIV measures were not able to distinguish between persons who showed stable cognitive decline compared with those exhibiting fluctuating performance over time. It is unclear whether these groups had the same long-term prognosis, or whether those who show stable decline were further along the continuum of impairment, and were more likely to progress to dementia.
Two studies looked at incident dementia in population-based samples. Baseline IIV (Kochan et al., Reference Kochan, Bunce, Pont, Crawford, Brodaty and Sachdev2016) and the ex-Gaussian parameter tau (Balota et al., Reference Balota, Tse, Hutchison, Spieler, Duchek and Morris2010), both predicted dementia conversion over 4 or 10 years, respectively. Additionally, the association was weaker but remained significant after controlling for several established dementia risk factors. Further to these population-based studies, variability has also been assessed in relation to future dementia in clinical samples. One such study assessed 1.5-year IIV change in Parkinson’s disease patients who converted to dementia relative to non-converters and disease free controls (de Frias et al., Reference de Frias, Dixon and Camicioli2012). Increased IIV over time was evident for dementia converters, but only distinguished the incipient Parkinson’s dementia group from the cognitively intact group and not the Parkinson’s disease dementia converters from non-converters.
From a clinical perspective, the differentiation between Parkinson’s dementia converters and non-converters is potentially the more important comparison if the aim is to detect individuals with Parkinson’s disease who are more likely to cognitively deteriorate. Finally, baseline IIV was greater in patients with amnestic MCI who subsequently converted to dementia compared to non-converters and cognitively healthy older adults (Bayer et al., Reference Bayer, Phillips, Porter, Leonards, Bompas and Tales2014; Tales et al., Reference Tales, Leonards, Bompas, Snowden, Philips, Porter and Bayer2012). However, as converters were also more cognitively impaired at baseline, it is unclear whether these results would remain if absolute baseline differences in cognition were controlled for.
Mortality
Four studies examined whether IIV was associated with all-cause mortality. Three reported data from older-adult samples (Batterham, Bunce, Mackinnon, & Christensen, Reference Batterham, Bunce, Mackinnon and Christensen2014; MacDonald et al., Reference MacDonald, Hultsch and Dixon2008; Shipley et al., Reference Shipley, Der, Taylor and Deary2006) and one from middle-aged participants (Deary & Der, Reference Deary and Der2005a). All of these studies found that greater baseline IIV was associated with an increased risk of death over periods of between 12 and 19 years. These associations remained statistically significant when controlling for influences typically associated with mortality such as socio-demographic factors, health behaviors, and health status. In addition, one study investigated the terminal decline hypothesis by modelling the trajectories of longitudinal IIV change as a function of time to death. Decedents exhibited increased IIV for each year closer to death and this effect was larger for those aged 80 to 95 years relative to those aged 50 to 79 years (MacDonald et al., Reference MacDonald, Hultsch and Dixon2003).
Comparisons With Other Measures
In this section, we consider the potential of IIV relative to either measures derived from the same RT task (e.g., mean or median RT). As IIV typically increases with response slowing, it is important to consider whether IIV effects are independent of more general slowing. Additionally, as variability measures may supplement current assessment methods, we assess whether IIV offers potential over and above traditional neuropsychological tasks.
Measures of central tendency
The majority of studies (13 of 17) included measures of central tendency (mean or median RT; MRT) derived from the same cognitive task. Seven of these reported results from models that contained both IIV and MRT measures. Here, IIV was predictive of outcome over and above MRT in four studies (Batterham et al., Reference Batterham, Bunce, Mackinnon and Christensen2014; Cherbuin et al., Reference Cherbuin, Sachdev and Anstey2010; Ghisletta et al., Reference Ghisletta, Fagot, Lecerf and De Ribaupierre2013; MacDonald et al., Reference MacDonald, Hultsch and Dixon2008), although this was not universal, as two studies found superior performance for MRT (Kochan et al., Reference Kochan, Bunce, Pont, Crawford, Brodaty and Sachdev2016; Shipley et al., Reference Shipley, Der, Taylor and Deary2006). In the remaining study, there was comparable performance (Deary & Der, Reference Deary and Der2005a).
Other neuropsychological tasks
Six (of 17) studies assessed the association between RT measures (IIV alone, or IIV and MRT added in a single step) and outcome relative to widely used neuropsychological tasks such as verbal recall or fluency, and IQ. In all studies, the results indicated that RT measures offered additional predictive utility over traditional tasks in predicting outcome, but there were mixed findings as to which RT measure was the best predictor. For example, relative to MRT and standardized cognitive tasks, IIV uniquely predicted MCI (Cherbuin et al., Reference Cherbuin, Sachdev and Anstey2010). Additionally, a composite measure of IIV, verbal recall and sustained attention performed better than the widely used Mini-Mental State Examination (MMSE) in detecting MCI.
Similarly, another investigation found that IIV produced a reliable increment in discrimination between dementia converters and non-converters in 14 of 15 standardized psychomotor tasks (Balota et al., Reference Balota, Tse, Hutchison, Spieler, Duchek and Morris2010). With regard to the association with mortality, greater IIV was associated with an increased risk of death independent of demographic, cardiovascular disease, MRT, and cognitive level measures (MacDonald et al., Reference MacDonald, Hultsch and Dixon2008). Two other studies used backward elimination to find the best predictors of mortality. Here, simple IIV and complex MRT (Deary & Der, Reference Deary and Der2005a), and complex MRT (Shipley et al., Reference Shipley, Der, Taylor and Deary2006), were retained in the model, whereas IQ and memory or visuospatial reasoning were not. Finally, using receiver operating characteristic analyses, one further study found that combined RT measures (IIV and MRT) compared favorably with traditional neuropsychological measures in the prediction of incident dementia (Kochan et al., Reference Kochan, Bunce, Pont, Crawford, Brodaty and Sachdev2016).
Methodological Differences Across Studies
There were methodological differences between studies in terms of the cognitive task or domain used to generate RTs, how many RT trials were included in the computation of metrics, and how IIV was calculated from the raw RTs. This raises the question of whether there is systematic variance in findings related to these factors, and whether there is evidence of an optimum IIV measure.
Task complexity
A dimension upon which tasks across studies varied, was cognitive complexity. Tasks ranged from those involving low cognitive demands (e.g., finger tapping) to more cognitively complex tasks (e.g., task switching). Thirteen (of 17) studies used multiple tasks or conditions of differing complexity, although two (Balota et al., Reference Balota, Tse, Hutchison, Spieler, Duchek and Morris2010; MacDonald et al., Reference MacDonald, Hultsch and Dixon2003) reported results based on a single composite. The findings across studies suggested that tasks of varying complexity were all associated with outcomes including cognitive change (Grand et al., Reference Grand, Stawski and MacDonald2016; Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016), cognitive impairment (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010a; Cherbuin et al., Reference Cherbuin, Sachdev and Anstey2010), and mortality (Batterham et al., Reference Batterham, Bunce, Mackinnon and Christensen2014; Deary & Der, Reference Deary and Der2005a; MacDonald et al., Reference MacDonald, Hultsch and Dixon2008; Shipley et al., Reference Shipley, Der, Taylor and Deary2006).
There were two exceptions, both of which only found significant effects for the more complex tasks (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; Kochan et al., Reference Kochan, Bunce, Pont, Crawford, Brodaty and Sachdev2016). However, this was relative to the other tasks within the study rather than on the overall continuum of cognitive tasks. For example, in Kochan and colleagues’ (Reference Kochan, Bunce, Pont, Crawford, Brodaty and Sachdev2016) study, although choice-RT IIV and not simple-RT IIV predicted future dementia, the differences in complexity between the two tasks was relatively small, and both were less cognitively demanding than tasks used elsewhere. In addition, other investigations using the same cohort and measures as Bielak and colleagues (Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b), where stronger effects for more complex tasks were evident, found significant associations for both basic and complex composite measures (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010a; Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016). In sum, it appears that the findings for task complexity are inconsistent, and comparing across studies, the evidence suggests that moderately complex tasks are sufficient to generate reliable metrics of IIV.
Number of trials
The majority of studies used between 20 and 60 RT trials in computations of IIV, although one group repeat-tested the RT tasks over multiple weeks and averaged across sessions (Bielak et al., Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010a, Reference Bielak, Hultsch, Strauss, MacDonald and Hunter2010b; Yao et al., Reference Yao, Stawski, Hultsch and MacDonald2016). Across studies, there does not appear to be reliable differences arising from the number of RT trials used, or whether the testing session was repeated. Although there was insufficient evidence to determine whether there is an optimum number of trials, the studies included here suggest that 20 to 60 trials were sufficient to generate reliable IIV metrics that were associated with future outcome.
Measure of IIV
Studies also differed in how IIV metrics were computed. The majority used either the raw SD, or adjusted SD partialing out systematic variance associated with influences such as trial number, block, experimental condition, and age group. Some investigations, however, used measures that controlled for mean level of responding (e.g., coefficient of variation), fitted alternative models to individual RTs (e.g., an ex-Gaussian distribution), or used non-parametric measures (e.g., interquartile range). Where studies reported results from multiple metrics, these tended to show converging results, with IIV from all measures showing an association with outcome (e.g., Batterham et al., Reference Batterham, Bunce, Mackinnon and Christensen2014; Cherbuin et al., Reference Cherbuin, Sachdev and Anstey2010; Lovden et al., Reference Lovden, Li, Shing and Lindenberger2007). Across studies, the different IIV metrics tended to produce similar results and there did not appear to be systematic variation according to the method used to compute the IIV metric.
DISCUSSION
This is the first systematic review to consider longitudinal change in IIV and whether such measures predict cognitive and adverse health-related outcomes. The first main finding was that IIV increased over time, particularly in participants aged over 75 years. Second, greater baseline IIV was consistently associated with an increased risk of cognitive decline, cognitive impairment, dementia, or mortality. There was also evidence that this was independent of general slowing, and that IIV added to the predictive utility of commonly used neuropsychological assessment tools. Lastly, findings did not vary systematically according to methodological differences such as task complexity, number of trials, and how IIV was calculated.
The review provided clear evidence that IIV increased over time in normal ageing. This is in line with cross-sectional studies and reviews that show higher IIV in older compared to younger groups (Dykiert et al., Reference Dykiert, Der, Starr and Deary2012), and in old-old relative to young-old individuals (Hultsch et al., Reference Hultsch, MacDonald and Dixon2002). There were, however, conflicting results as to the age at which increases in IIV began. While there were consistent findings of increasing IIV in old-old individuals, results varied across studies of younger individuals. Although theoretically it is plausible that age-related increases in variability begin in middle age (and perhaps earlier), further evidence supporting this possibility is clearly needed.
Although beyond the scope of the present review, a meta-analysis of longitudinal change in IIV would help elucidate these inconsistent findings. In addition, three studies considered other influences on the trajectory of IIV change over time including both protective (lifestyle activities) and risk factors (type II diabetes and COMT and BDNF gene variants) for broader cognitive decline. However, none of these factors consistently influenced change in IIV. Together, the findings suggest that further work is needed to understand influences on age-related increases in IIV, and whether such increases are changeable, for example, through lifestyle intervention.
The review also found clear evidence that greater baseline IIV was associated with greater cognitive decline and an increased risk of MCI, dementia, or mortality over the follow-up period. The results are consistent with cross-sectional findings of increased IIV in populations with age-related conditions such as MCI and dementia (e.g., Christensen et al., Reference Christensen, Dear, Anstey, Parslow, Sachdev and Jorm2005; Dixon et al., Reference Dixon, Lentz, Garrett, MacDonald, Strauss and Hultsch2007; Gorus et al., Reference Gorus, De Raedt, Lambert, Lemper and Mets2008). Importantly, however, as the majority of studies measured IIV at a time when participants were non-demented and cognitively intact, the results suggest that increases in IIV may precede these adverse health-related outcomes by several years.
These findings highlight the potential for IIV as a prognostic measure that may help identify individuals at risk of future deleterious outcomes. They are also consistent with the proposal that IIV is a behavioral marker of neurobiological integrity (Hultsch et al., Reference Hultsch, MacDonald, Hunter, Levy-Bencheton and Strauss2000, Reference Hultsch, Strauss, Hunter and MacDonald2008). Variability may be sensitive to early neuropathological changes that influence attentional and executive control mechanisms (cf. Bunce et al., Reference Bunce, MacDonald and Hultsch2004, Reference Bunce, Warr and Cochrane1993; West et al., Reference West, Murphy, Armilio, Craik and Stuss2002), which precede broader cognitive dysfunction.
What is also not currently clear is whether increased IIV is related to specific neurological outcomes such as Alzheimer’s disease, or is a universal sign of broader changes to brain integrity. Although it is important that future research address this question, it is of note that IIV also predicted all-cause mortality, suggesting that increased IIV is not necessarily specific to Alzheimer’s pathology. IIV may predict mortality as neurobiological compromise may reflect broader biological processes that themselves portend to death. However, as IIV was also associated with an increased hazard of mortality in a middle-aged sample (Deary et al., Reference Deary, Bastin, Pattie, Clayden, Whalley, Starr and Wardlaw2006), the mechanism linking IIV and mortality may be complex and needs to be further elucidated.
An important consideration was whether IIV predicted outcome independently of estimates of central tendency. Measures such as standard deviation are typically highly correlated with mean RT taken from the same task and it has been argued that age-related differences in IIV reflect a general slowing of responses (e.g., Myerson et al., Reference Myerson, Robertson and Hale2007). It is, therefore, appropriate to exercise caution in interpreting findings from studies that do not control for MRT. Nonetheless, across the reviewed studies, although greater baseline MRT was associated with an increased risk of cognitive impairment and mortality, IIV showed consistent results across studies, and tended to be associated with outcome after adjusting for MRT. The results suggest, therefore, that associations between IIV and outcome were not simply related to general slowing and that IIV measures possess unique predictive utility. This finding is consistent with cross-sectional studies that show differentiation between IIV and MRT (e.g., Bunce et al., Reference Bunce, Anstey, Christensen, Dear, Wen and Sachdev2007; Dixon et al., Reference Dixon, Lentz, Garrett, MacDonald, Strauss and Hultsch2007).
Although assessed in fewer studies, the results also indicated that IIV predicted outcome over and above widely used neuropsychological tests. Again, this suggests that variability measures offer unique information relative to other commonly used tasks and may, therefore, serve as a useful supplement to standard neuropsychological test batteries in healthcare settings. This is an area that would benefit from further research to determine the clinical utility of IIV, either alone or in combination with traditional measures. As an example, Cherbuin et al. (Reference Cherbuin, Sachdev and Anstey2010) showed that variability measures in combination with other cognitive tasks exhibited greater predictive power for cognitive impairment than the MMSE.
There were methodological variations across studies that included differences in RT tasks, the number of RT trials used, and how the IIV metric was computed. Previous research suggests there are greater differences associated with age (Dykiert et al., Reference Dykiert, Der, Starr and Deary2012) and dementia (de Frias et al., Reference de Frias, Dixon, Fisher and Camicioli2007; Gorus et al., Reference Gorus, De Raedt, Lambert, Lemper and Mets2008) using IIV measures from more demanding tasks. Additionally, a recent systematic review from our group suggested more sophisticated measures of IIV may possess greater utility in understanding the relationship between IIV and falls or gait disturbance (Graveson et al., Reference Graveson, Bauermeister, McKeown and Bunce2016). Nonetheless, in the present review, we did not find any systematic differences in results according to task complexity, number of RT trials, or the IIV metric used.
However, the present results indicate that moderately complex tasks, with 20 to 60 trails are sufficient to produce reliable IIV metrics. This is consistent with a recent cross-sectional study that found that 20 trials taking approximately 52 s to administer provided a reliable estimate of frontal white matter integrity (Bunce et al., Reference Bunce, Bielak, Cherbuin, Batterham, Wen, Sachdev and Anstey2013). The present findings also suggest that it is not necessary to use multiple testing sessions, or to compute mathematically complex measures of IIV. This is of note as such complexities would create practical difficulties when using the measures for neuropsychological assessment in clinical settings. Accordingly, the coefficient of variation may be appropriate in such settings, as it is relatively straightforward to compute and takes mean RT into account.
Although there were consistent findings across studies indicating that IIV measures were associated with future cognitive impairment or mortality, there are some caveats to this conclusion. First, while we identified 22 studies for inclusion in this review, several of these involved samples drawn from the same study population. This interdependence between some studies should be kept in mind when considering the findings of this review. Second, the samples were restricted to Western Europe, Australia, and North America, and there is clearly a need for similar work in populations from Central and South America, Africa, and Asia. Third, few studies reported metrics such as the sensitivity or specificity of IIV measures, although where these were reported (e.g., Cherbuin et al., Reference Cherbuin, Sachdev and Anstey2010; de Frias et al., Reference de Frias, Dixon, Fisher and Camicioli2007), variability measures performed well. Lastly, none of the studies provided concrete cutoff scores for normative performance. The absence of normative standards limits the current clinical utility of IIV metrics. It is, therefore, of pressing importance that future research determines normal ranges for IIV and IIV change in healthy older populations as this is central to developing the clinical utility of the measure.
Despite these considerations, having used a recognized framework to evaluate the reviewed studies (Hayden et al., Reference Hayden, Cote and Bombardier2006, Reference Hayden, van der Windt, Cartwright, Cote and Bombardier2013), there are several strengths of the research that should be highlighted. First, the majority of investigations were population based studies with large sample sizes including a broad spectrum of individuals at risk of measured outcomes and clearly defined inclusion and exclusion criteria. Additionally, most studies had follow-up data on more than 80% of the sample and many used statistical procedures that took missing data into account. All of the studies included clear descriptions of how IIV was measured and how the outcome was defined. Lastly, many of the studies also controlled for potential confounding factors, including known risk factors for dementia and mortality.
In conclusion, the present review considered nine studies that looked at change in IIV over time and 17 studies that investigated whether IIV was associated with cognitive and adverse health-related outcomes. The results suggest that increasing IIV over time is related to normal ageing, but that more marked increases in IIV may indicate future cognitive decline, mild cognitive impairment, dementia, or mortality. One intriguing possibility is that IIV may provide a behavioral index of “brain frailty” arising from structural and physiological changes in the ageing brain. As such, measures of IIV may have considerable potential in clinical settings as they may supplement existing neuropsychological test batteries and help identify persons for early therapeutic intervention.
Measures of IIV may, therefore, have considerable potential in clinical settings and offer practical advantages as administration is quick and requires little neuropsychological training. Additionally, the tasks often involve minimal linguistic content and may, therefore, be suitable for use with individuals from diverse backgrounds. Future work is needed to develop standardized methods for measuring variability, with clear guidelines and norms for interpretation of results in community-based and clinical populations.
Acknowledgements
There are no conflicts of interest to declare.
