Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-02-06T09:09:47.001Z Has data issue: false hasContentIssue false

An FMRI-Compatible Symbol Search Task

Published online by Cambridge University Press:  20 March 2015

Spencer W. Liebel*
Affiliation:
University of Georgia, Department of Psychology, Athens, Georgia
Uraina S. Clark
Affiliation:
Icahn School of Medicine at Mount Sinai, Department of Neurology, New York, New York
Xiaomeng Xu
Affiliation:
Idaho State University, Department of Psychology, Pocatello, Idaho
Hannah H. Riskin-Jones
Affiliation:
Warren Alpert Medical School of Brown University, Department of Psychiatry, Providence, Rhode Island
Brittany E. Hawkshead
Affiliation:
University of Georgia, Department of Psychology, Athens, Georgia
Nicolette F. Schwarz
Affiliation:
University of Georgia, Department of Psychology, Athens, Georgia
Donald Labbe
Affiliation:
Warren Alpert Medical School of Brown University, Department of Psychiatry, Providence, Rhode Island
Beth A. Jerskey
Affiliation:
Warren Alpert Medical School of Brown University, Department of Psychiatry, Providence, Rhode Island
Lawrence H. Sweet
Affiliation:
University of Georgia, Department of Psychology, Athens, Georgia Warren Alpert Medical School of Brown University, Department of Psychiatry, Providence, Rhode Island
*
Correspondence and reprint requests to: Spencer W. Liebel, 139 Psychology Building, University of Georgia, Athens, GA 30606. E-mail: swliebel@uga.edu
Rights & Permissions [Opens in a new window]

Abstract

Our objective was to determine whether a Symbol Search paradigm developed for functional magnetic resonance imaging (FMRI) is a reliable and valid measure of cognitive processing speed (CPS) in healthy older adults. As all older adults are expected to experience cognitive declines due to aging, and CPS is one of the domains most affected by age, establishing a reliable and valid measure of CPS that can be administered inside an MR scanner may prove invaluable in future clinical and research settings. We evaluated the reliability and construct validity of a newly developed FMRI Symbol Search task by comparing participants’ performance in and outside of the scanner and to the widely used and standardized Symbol Search subtest of the Wechsler Adult Intelligence Scale (WAIS). A brief battery of neuropsychological measures was also administered to assess the convergent and discriminant validity of the FMRI Symbol Search task. The FMRI Symbol Search task demonstrated high test–retest reliability when compared to performance on the same task administered out of the scanner (r=.791; p<.001). The criterion validity of the new task was supported, as it exhibited a strong positive correlation with the WAIS Symbol Search (r=.717; p<.001). Predicted convergent and discriminant validity patterns of the FMRI Symbol Search task were also observed. The FMRI Symbol Search task is a reliable and valid measure of CPS in healthy older adults and exhibits expected sensitivity to the effects of age on CPS performance. (JINS, 2015, 22, 1–8)

Type
Research Articles
Copyright
Copyright © The International Neuropsychological Society 2015 

Introduction

The proportion of older adults in the population of industrialized countries is rising substantially due to falling birth rates and increased longevity (Bloom, Reference Bloom2011). For instance, by 2050 it is estimated that there will be nearly twice as many older adults over 65 than children under 15 in the United States (Centers for Disease Control and Prevention, 2013; Cohen, Reference Cohen2003). Unfortunately, increased longevity does not ensure preserved quality of life, as all older adults are expected to experience age-associated cognitive decline, neuronal deterioration, and compromised white matter integrity (Gunning-Dixon, Brickman, Cheng, & Alexopoulos, Reference Gunning-Dixon, Brickman, Cheng and Alexopoulos2009; Park & Reuter-Lorenz, Reference Park and Reuter-Lorenz2009; Wen & Sachdev, Reference Wen and Sachdev2004). The cognitive impairments experienced by older adults are associated with compromised white matter integrity, and they include poorer performance on tasks of cognitive processing speed (CPS), memory, and executive functions (Gunning-Dixon & Raz, Reference Gunning-Dixon and Raz2000).

Deterioration of brain tissue often manifests as white matter hyperintensities (WMH) on T2-weighted magnetic resonance images (MRI; DeCarli et al., Reference DeCarli, Massaro, Harvey, Hald, Tullberg, Au and Wolf2005; Rabbitt et al., Reference Rabbitt, Scott, Lunn, Thacker, Lowe, Pendelton and Jackson2007; Söderlund et al., Reference Söderlund, Nilsson, Berger, Breteler, Dufouil, Fuhrer and Launer2006). WMH are present in 11–21% of adults aged 64, and this proportion increases to 94% by age 82 (Debette & Markus, Reference Debette and Markus2010). Additionally, prior research has shown that, in otherwise healthy older adults, WMH are linked to poorer performance on many tests of cognitive functioning, including measures of CPS (Papp et al., Reference Papp, Kaplan, Springate, Moscufo, Wakefield, Guttmann and Wolfson2014; van den Heuvel et al., Reference van den Heuvel, ten Dam, de Craen, Admiraal-Behloul, Olofsen, Bollen and van Buchem2006). Due to its prevalence and significance as a neurocognitive marker, quantifying and tracking WMH volume has been a valuable clinical and research outcome measure (Schmidt et al., Reference Schmidt, Scheltens, Erkinjuntti, Pantoni, Markus, Wallin and Fazekas2004).

An extensive body of research literature shows that cognitive functioning declines consistently with age (e.g., Rabbitt, Reference Rabbitt2002, for review). Gradual age-related declines are most robust in the cognitive domains of working memory, long-term memory, and CPS (Gunning-Dixon & Raz, Reference Gunning-Dixon and Raz2000; Salthouse, Reference Salthouse2000). Simply considered a consequence of aging, these decreases in performance occur in the absence of obvious disease or trauma. For instance, in healthy adults, raw scores on Symbol Search, a subtest of the Wechsler Adult Intelligence Scale (WAIS) that is used to assess CPS, decline by more than 50% between the ages of 25 and 65 (Wechsler, Reference Wechsler2008). CPS is also of particular interest to aging research because it is a fundamental component of many of the brain’s other functions (Rypma & Prabhakaran, Reference Rypma and Prabhakaran2009; Salthouse, Reference Salthouse1996). Indeed, age-related slowing of CPS is a significant contributor to declines in test scores in other cognitive domains (Finkel et al., Reference Finkel, Mintzer, Dysken, Krishnan, Burt and McRae2004; Salthouse & Coon, Reference Salthouse and Coon1993; Whiting & Smith, Reference Whiting and Smith1997). Because CPS is a fundamental component of many cognitive functions, this accurate assessment may be particularly useful as a sensitive predictor of changes in higher-order cognitive abilities, and an early marker of brain dysfunction (Duering et al., Reference Duering, Gesierich, Seiler, Pirpamer, Gonik, Hofer and Dichgans2014; Eckert, Reference Eckert2011; Salthouse & Ferrer-Caja, Reference Salthouse and Ferrer-Caja2003). Therefore, measures of CPS, such as Symbol Search, are frequently used to detect both abnormal and normal age-related changes in brain integrity.

Over the past 20 years, functional magnetic resonance imaging (FMRI) has demonstrated great value in improving assessments of brain function associated with age-related cognitive decline. Clinical applications for FMRI have also been emerging, such as presurgical mapping, and have contributed to presymptomatic diagnostics of a broad range of diseases (Matthews, Honey, & Bullmore, Reference Matthews, Honey and Bullmore2006). Many of these advances in aging research have relied on accurate and concurrent assessments of behavioral performance during FMRI. A fundamental assumption of the technique is that the brain activity elicited is a response to a carefully controlled behavioral challenge of a well-defined neurocognitive construct. Yet, the cognitive tests used by FMRI investigators in the scanner are not typically normed or standardized across sites. Indeed, some have not been validated at all. Therefore, it is imperative that valid FMRI paradigms are developed that also have demonstrated generalizability and sensitivity to cognitive decline in older adults.

Relatively few CPS paradigms have been adapted for administration during FMRI, and those that have been, have not been systematically examined for reliability and validity. Moreover, the study of functional neuroimaging correlates of CPS has lagged behind other neurocognitive functions, such as working memory and attention. Indeed, the use of externally validated FMRI paradigms is rare in any neurocognitive domain. When standardized measures (e.g., neuropsychological tests) have been adapted for FMRI, they often undergo extensive changes to make them feasible in the scanning environment or subsequent data analyses (e.g., Langeneker, Nielson, & Rao, Reference Langenecker, Nielson and Rao2004; Leavitt, Wylie, Genova, Chiaravalloti, & Deluca, Reference Leavitt, Wylie, Genova, Chiaravalloti and Deluca2012; Phelps, Hyder, Blamire, & Shulman, Reference Phelps, Hyder, Blamire and Shulman1997). Given the major challenges of the setting, many of these alterations are necessary and, nevertheless, have yielded valuable findings. For example, typical major alterations to tasks include nonverbal responding during a verbal task (Langeneker et al., Reference Langenecker, Nielson and Rao2004), visual presentation of an auditory task (Staffen et al., Reference Staffen, Mair, Zauner, Unterrainer, Neiderhofer, Kutzelnigg and Ladurner2002), requiring a motor response instead of an oral response (Leavitt et al., Reference Leavitt, Wylie, Genova, Chiaravalloti and Deluca2012), or standardized-pacing of a self-paced test (Langenecker et al., Reference Langenecker, Nielson and Rao2004). More questionable is the less common, and now rare, strategy of assessing behavioral performance outside of the scanner instead of during the scan. While these strategies have provided valuable insights into neurocognitive functioning, modifying these measures calls into question their generalizability to the original measure and their underlying neurocognitive constructs. Thus, establishing validated behavioral tasks for FMRI assessments would allow for greater reliability, generalizability, and clinical utility (Cohen & Sweet, Reference Cohen and Sweet2011).

The overall goal of this study was to determine if a Symbol Search paradigm developed for FMRI is a reliable and valid behavioral challenge of CPS in healthy older adults. Specifically, our aim was to determine if performance on a newly developed FMRI Symbol Search task demonstrated test–retest reliability when given in and out of the scanner, and whether it exhibited criterion validity (i.e., correlated strongly with the WAIS-III Symbol Search subtest). Construct validity was further assessed by examining convergent and discriminant validity (Cronbach & Meehl, Reference Cronbach and Meehl1955). Associations between the FMRI Symbol Search paradigm and other measures of the CPS construct, age, and WMH volume were examined to determine convergent validity. We examined discriminant validity by assessing the relationship between the FMRI Symbol Search task and measures that are known to exhibit weak relationships to CPS. In sum, we predicted that the FMRI Symbol Search task would exhibit high reliability and strong construct validity as a measure of CPS in healthy older adults.

Methods

Participants

A community sample of 45 healthy older adults (31 female) over the age of 50 were recruited via newspaper ads and flyers (Table 1). Ages ranged from 50 to 85 years (M age=63.09; SD=8.44). Mean level of education was 15.75 years (SD=2.26). WMH volume ranged from 1.9 to 45.5 mL (M=7.74 mL; SD=7.17 mL). Participants were monetarily compensated. The inclusion criteria were that the individuals be right-handed English-speakers with normal or corrected vision at the time of testing. Potential participants were excluded if they were diagnosed with significant heart problems (e.g., surgery, infarct), neurological disease (e.g., history of stroke, multiple sclerosis), traumatic brain injury (with loss of consciousness), history of substance abuse that resulted in hospitalization, diagnosis of any current psychiatric illness, or any MRI contraindications (e.g., metal implants). The study was approved by hospital and university Internal Review Boards and conformed to the Helsinki Declaration.

Table 1 Participant demographics, neuropsychological test battery data, and medications taken

Procedures

Participants completed a neuropsychological and MRI assessment during two separate visits. The neuropsychological assessment was supervised by a licensed clinical neuropsychologist. It took place before the MRI assessment in a quiet room and included the WAIS-III Symbol Search subtest. Administration of the 2-min Symbol Search subtest from the WAIS occurred approximately 40 min after the start of the 2-hr neuropsychological assessment battery. Responses were collected via paper and pencil as per the WAIS-III administration manual (Wechsler, Reference Wechsler1997). The MRI assessment was 1 hr long and the FMRI Symbol Search paradigm was presented approximately 25 min after the start of the scanning session. Stimuli were presented onto a projection screen that was visible to the participant while in the scanner. The number of correct responses during a 2-min period of time was determined based on yes/no button presses on a MRI compatible response box. The FMRI Symbol Search paradigm was also presented out of the scanner approximately 20 min before the FMRI scan.

Measures

FMRI Symbol Search

The FMRI Symbol Search paradigm was administered with minor adaptations according to the instructions in the WAIS-III manual (Wechsler, Reference Wechsler1997) and a prior in-scanner version (Sweet et al., Reference Sweet, Paskavitz, O’Connor, Browndyke, Wellen and Cohen2005). Task instructions were the same as those provided before administration of the WAIS-III Symbol Search subtest. The task was presented using E-Prime 2.0 (http://www.pstnet.com/). Although FMRI data were collected concurrently with the in-scanner Symbol Search task, these data were not analyzed as part of the current study.

The task consisted of two, four-cycle imaging runs of 6 min each. Each of the four cycles consisted of a 30-s control task block followed by a 30-s Symbol Search task block. Participants responded to as many items as possible during the time allotted. To avoid laterality effects due to visual presentation, the exemplar figures were placed directly above the five target figures (Figure 1). Instead of drawing a line through their desired response, participants used a button box to identify whether either exemplar was present in the target symbol group using a “YES” or “NO” button. A new item appeared immediately following button press. A visuomotor control task was also administered. The control task asked participants to respond “YES” or “NO” to indicate whether a particular symbol was present anywhere on the screen. The target symbol participants were asked to identify during the control task remained the same for the duration of the task. The control task was self-paced. The number of correct items during each block was recorded using E-Prime. The symbols presented during the FMRI Symbol Search task were comprised of syllabic characters (excluding ideograms) from the Mycenaean alphabet Linear B, which has not been in use for over 3200 years.

Fig. 1 Sample FMRI Symbol Search and visuomotor control task items. This figure demonstrates how the FMRI Symbol Search task and visuomotor control task were presented to participants, including instructions before task items.

Measure of Reliability

The same FMRI Symbol Search task was presented both inside and outside of the scanner to examine reliability. The outside-of-scanner administration occurred approximately 45 min before the in-scanner version was presented using a computer running E-Prime 2.0 and a connected response box. When administered outside of the scanner, the FMRI Symbol Search task included the same instructions, presentation, and response format as given in the scanner. High reliability, indicated by a high correlation coefficient between performances on the FMRI Symbol Search task administered in and out of the scanner, would support the generalizability of the FMRI Symbol Search task despite the uniqueness of the scanning environment (e.g., interactions with individual differences, measurement error).

Measure of Criterion Validity

WAIS Symbol Search

Symbol Search is a subtest of the WAIS (Wechsler, Reference Wechsler2008) and is often included in clinical neuropsychological assessments to measure visuospatial attention and CPS. This is a self-paced task during which examinees are allotted 2 min to complete as many items as possible. The task requires rapid comparisons to determine if a set of five target geometric designs include one of two exemplars, which are positioned just to the left of the target designs. If one of the exemplars is present in the set of five target designs participants draw a line through the target design; if neither of the two exemplars is found in the group of target designs a line is drawn through a “NO” box. Participants are instructed to work as quickly and accurately as possible until they are told to stop. It was predicted that a high correlation coefficient between performance on the WAIS Symbol Search and the FMRI Symbol Search would represent strong criterion validity.

Measures of Convergent Validity

The following measures were examined to assess convergent validity. All available CPS measures from the neuropsychological battery were included in our analyses. They were expected to exhibit strong associations with the FMRI Symbol Search task. Because these measures are sensitive to the effects of age and the conversion from raw scores to normed scores takes into account participant age, raw scores were used in all analyses

Delis-Kaplan Executive Functioning System (D-KEFS) Trails Making Test

The D-KEFS Trails Making Test (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001) is a test frequently used as a measure of visuo-motor scanning, executive functioning, and CPS in neuropsychological test batteries (Lezak, Howieson, Bigler, & Tranel, Reference Lezak, Howieson, Bigler and Tranel2012). For this study, Trails 2 (number sequencing) and 3 (letter sequencing) were used because they are considered measures of CPS. Administration and scoring of Trails 2 and 3 were completed according to the D-KEFS administration manual (Delis et al., Reference Delis, Kaplan and Kramer2001).

Delis-Kaplan Executive Functioning System (D-KEFS) Color-Word Interference Test

The D-KEFS Color-Word Interference Test (CWIT; Delis et al., Reference Delis, Kaplan and Kramer2001) consists of four parts: color naming, word reading, inhibition, and inhibition/switching. The present study examined performance on color naming and word reading because these are routinely used as a measure of CPS in neuropsychological test batteries. For color naming, participants were presented with a page containing a series of red, green, and blue squares. The participants were asked to say the names of the colors as quickly as possible without making mistakes. For word reading, the participants were presented with a page containing the words “red”, “green”, and “blue” printed in black ink. Participants were asked to read the words aloud as quickly as possible without making mistakes. Participant performance was measured by completion time on each trial.

Repeated Battery for the Assessment of Neuropsychological Status (RBANS) Coding

RBANS Coding (Randolph, Tierney, Mohr, & Chase, Reference Randolph, Tierney, Mohr and Chase1998) is a frequently used measure of CPS in neuropsychological assessments of older adults. Briefly, participants were instructed to fill in missing digits corresponding to shapes in a key located at the top of the page as quickly as possible in 90 s. Administration and scoring followed the instructions provided in the RBANS manual.

The Purdue Pegboard Test (PPT)

The PPT (Model 32020; Lafayette Instrument Co., Lafayette, IN) is a functional assessment tool of hand dexterity and psychomotor processing speed. It consists of 30 holes arranged in two identical columns and pegs located in four cups at the top of the board. Subjects were instructed to start the test after a verbal cue from an examiner and the examiner timed the test with a stopwatch. Subjects were allotted 30 s to fill the holes with pegs using their dominant hand, then 30 s to fill them with their non-dominant hand, and finally with both hands simultaneously. Each subtest was repeated three times to obtain an average. In this study, only dominant hand data were used as a measure of CPS. The test scores equaled the number of correctly placed pegs.

WMH quantification

WMH volume was quantified from high resolution (1 mm3) fluid attenuated inversion recovery (FLAIR) and T1-weighted MRI sequences as described in detail elsewhere (Riskin-Jones, Xu, Clark, Labbe, & Sweet, Reference Riskin-Jones, Xu, Clark, Labbe and Sweet2014). Briefly, high-resolution whole-brain T1-weighted images were individually segmented into white and gray matter using FreeSurfer following established procedures (Fischl et al., Reference Fischl, Salat, Busa, Albert, Dieterich, Hasselgrove and Dale2002). The segmentations were aligned to high-resolution whole-brain FLAIR images in native space using AFNI-SUMA alignment tools (Cox, Reference Cox1996; Saad, Reference Saad2004). An iterative region growing algorithm was applied to the white matter segmentation to identify WMH. Initially, seeds were identified as voxels with intensities of at least 25% above the median intensity. The iterative algorithm searched within a 27-voxel rectangle around each voxel in a seed region for adjacent voxels that fall within 5% of the seed mean. Those neighboring voxels were then added to the total seed region volume used in the next iteration. The algorithm continued until no new voxels were added. Total WMH volume was the sum of the voxels included in this iterative process.

Measures of Discriminant Validity

The following measures are known to be weakly related to the CPS construct and age. Thus, their associations with the FMRI Symbol Search task were predicted to be relatively weak and substantially accounted for by age.

RBANS Picture Naming

RBANS Picture Naming (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998) is a test of language, specifically confrontation naming, which includes 10 line drawings of objects that must be named by the examinee. The examinee is given up to 20 s to name each object. This test is not considered to consist of a strong CPS component.

RBANS List Learning Recognition

RBANS List Learning Recognition (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998) is a measure of delayed verbal memory that requires examinees to accurately identify items from a previously learned word list. Examinees respond either “YES” or “NO” to words presented from the list or 10 distractors. This subtest yields a score between zero and 20.

RBANS Figure Copy

RBANS Figure Copy (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998) is a measure of visuospatial and constructional skills. Examinees are asked to copy a complex geometric figure. The figure consists of 10 components and yields a maximum score of 20.

RBANS Line Orientation

RBANS Line Orientation (Randolph et al., Reference Randolph, Tierney, Mohr and Chase1998) is a test of visuospatial abilities. The task presents examinees with an array of 13 lines, fanning out from the same origination point but in different directions. For each item, two target lines were shown beneath the array and subjects identified which lines match within the array. There are 10 items, each containing two lines to be matched, for a total maximum score of 20.

Wechsler Test of Adult Reading (WTAR)

The WTAR (Wechsler, Reference Wechsler2001) assesses the participants’ familiarity with irregularly pronounced words and is regularly used for clinical and research purposes to assess premorbid IQ. Administration and scoring followed prescribed methods from the WTAR manual. Briefly, examinees were presented with a page of words and asked to read each one aloud. This is a self-paced test, and the number of correctly read words constituted the participants’ score.

Mini-Mental State Examination (MMSE)

The MMSE (Folstein, Folstein, & McHugh, Reference Folstein, Folstein and McHugh1975) is the most commonly used geriatric cognitive screening tool in the United States, Canada, and United Kingdom (Shulman et al., Reference Shulman, Herrmann, Brodaty, Chiu, Lawlar, Ritchie and Scanlan2006). It is a very brief instrument, but it has been shown to reliably screen for cognitive deficit in older adults. Administration and scoring followed guidelines prescribed in the test manual.

Statistical Procedures

To examine the reliability and validity of the newly adapted FMRI Symbol Search paradigm, the following statistical procedures were conducted. To assess reliability, the association between the same FMRI Symbol Search task that was administered in and out of the scanner was examined by calculating a Pearson correlation coefficient. To measure construct validity, we first assessed criterion validity and then convergent and discriminant validity. Criterion validity was measured using a Pearson correlation between the FMRI Symbol Search task and raw WAIS Symbol Search subtest scores. To measure convergent validity, Pearson correlation coefficients between the FMRI Symbol Search task and raw scores of other well-established measures of CPS were examined. Absolute correlation coefficients less than .3 were considered poor, .3–.6 adequate, and .6 or greater were good to very good. These values differ from the widely known “conventions” suggested by Cohen (Reference Cohen1988, Reference Cohen1992) and represent an effort to place more stringent criteria on the data (Hemphill, Reference Hemphill2003). Discriminant validity was measured by calculating correlation coefficients between the FMRI Symbol Search task and raw scores on measures that should not be strongly related to CPS; poor correlations lower than .3 were considered evidence of discriminant validity. Significance threshold for all analyses was set at p<.05, two-tailed. Statistical Package for the Social Sciences (SPSS; version 21) was used for all analyses.

Results

The FMRI Symbol Search task demonstrated good to very good test–retest reliability with a significant positive correlation when compared to performance on the same task administered out of the scanner (r=.791; p<.001). Mean performance on the task administered outside of the scanner improved compared to in-scanner performance (t(44)=−3.257; p<.002). Criterion validity analysis revealed that the FMRI Symbol Search task exhibited significant positive correlation with the WAIS Symbol Search (r=.717; p<.001), which corresponded to the good to very good range. Correlations between the FMRI Symbol Search and other measures of CPS are shown in Table 2 (i.e., convergent validity). Before controlling age, the FMRI Symbol Search task was significantly associated with all measures. Associations ranged from adequate to good to very good. Notably, there was a strong expected inverse relationship between the FMRI Symbol Search task and age. Age was controlled in subsequent analyses to explore the underlying relationships between these other measures of CPS independent of the known effects of age. Therefore, partial correlations between FMRI Symbol Search and the other CPS measures were thought to provide an extremely stringent representation of the underlying CPS construct by controlling extraneous variables associated with age. After controlling age, two measures remained in the good to very good range (RBANS Coding and D-KEFS Trails 3), with the remaining measures falling at or near the adequate cutoff. Table 3 contains the results of discriminant validity analyses. Before controlling age, the FMRI Symbol Search task showed predicted weak correlation (r<.3) with five of the six non-CPS measures falling in the poor range. After controlling for age, all six non-CPS measures demonstrated predicted weak correlations (r<.3). These results suggest that the FMRI Symbol Search task exhibits only weak associations with the measures of cognitive domains chosen for discriminant validity.

Table 2 Convergent validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age

Table 3 Discriminant validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age

Discussion

The FMRI Symbol Search task was developed to provide clinicians and researchers a reliable and valid measure of CPS that would generalize to the WAIS Symbol Search subtest. Findings strongly support the reliability and construct validity of our FMRI-adapted Symbol Search task as a measure of CPS in healthy older adults. Indeed, the reliability of in- and out-of-scanner results was good to very good (r=.791; Hemphill, Reference Hemphill2003). When compared to the WAIS Symbol Search, a widely accepted and carefully standardized measure of CPS, our FMRI Symbol Search task also showed strong criterion validity that was in the good to very good range. To our knowledge, this is the first systematic demonstration that a CPS measure may be adapted for use in the FMRI environment while maintaining high reliability and generalizability.

These data also suggest that the FMRI Symbol Search task demonstrated excellent convergent validity when compared to the other measures of CPS. In fact, the FMRI Symbol Search task showed adequate to good to very good Pearson correlation coefficients with age and all CPS measures (Hemphill, Reference Hemphill2003). These effects remained despite controlling age, which is a well-documented correlate of the WAIS Symbol Search and other measures of CPS. Age was controlled to examine residual effects (i.e., the underlying CPS construct) without the influence of this potential extraneous variable confound (i.e., third variable effects associated with age). Three of the other measures of CPS exhibited adequate to good to very good correlation with the FMRI Symbol Search task when age was controlled. The four that did not yield strong correlation coefficients after age correction, nevertheless, yielded effects that were very close to the adequate cutoff of r=.3. It is hypothesized that these measures no longer reached the adequate cutoff due to a reduction in power when age was controlled. Overall, these findings provided robust support for convergent validity because controlling common variance in all CPS measures that was associated with age likely yields an underestimation of the true relationships among the measures.

The FMRI Symbol Search task demonstrated excellent discriminant validity as well. It was only weakly associated (i.e., not adequate) with measures that do not include a strong CPS component. Overall, these correlations became even weaker after controlling the effects of age. The discriminant validity results suggest, as expected, that the FMRI Symbol Search task is not associated with non-CPS constructs (e.g., memory, visuospatial skills, or language).

These findings are important because they support the utility of this FMRI-compatible measure of CPS. The development of reliable FMRI paradigms with demonstrated construct validity and generalizability is crucial to understanding CPS and the other cognitive domains affected by the aging brain. With an increasing geriatric population, FMRI paradigms that assess cognitive domains important to aging will be in greater demand. Due to its high level of association with the WAIS Symbol Search, the FMRI Symbol Search task may find use in future studies as an efficient proxy of CPS in this population. Furthermore, the format of the FMRI Symbol Search task relies less on motor coordination abilities than does the WAIS Symbol Search. This is often an area of difficulty for older persons and individuals with clinical conditions that result in motor coordination dysfunctions. Additionally, the potential exists for this FMRI measure to be incorporated into cognitive batteries used to assess patients in which CPS is known to be one of the cardinal markers of cognitive deficit (e.g., survivors of traumatic brain injury or individuals with multiple sclerosis). Because routine clinical care of these patients often includes separate cognitive and MRI assessments, a standardized CPS assessment could be shifted to the MRI session to yield concurrent acquisition of complementary behavioral and functional data. Validation of our FMRI Symbol Search measure allows clinicians and researchers to link clinically relevant behavioral performance to routinely assessed structural MRI indices and adds novel information about concurrent brain function. This is likely to enhance diagnostics, prognostics, and outcome monitoring.

The interpretation of the results of this study must, however, be considered in light of some limitations. The reliability and validity of the FMRI-adapted Symbol Search paradigm were only examined in an older adult sample with above average levels of education. While the greatest expected CPS decline and variance was expected in this sample, administering the task to younger adults with a greater range of educational achievement is necessary to support its validity across the lifespan and levels of education.

There is a great need for standardized FMRI assessments in both existing research settings and emerging clinical applications. The FMRI Symbol Search task provides researchers and clinicians such a test and represents the first measure of CPS adapted for use in FMRI. Because clinical applications for FMRI are emerging, behavioral measures adapted for use inside the scanning environment will be useful to increase generalizability to the normed clinical measure about which much is already known. Indeed, there are many additional neuropsychological measures that can be adapted for use with FMRI so that clinicians and researchers may benefit from the advantages and strengths of the behavioral measure and FMRI technology to reveal neural correlates.

Acknowledgments

This work was supported by grants from the National Institutes of Health Heart, Lung and Blood Institute (L.H.S., R01 HL084178) and the National Institutes of Health (U.S.C., K23 MH096628). The authors do not report any conflicts of interest.

References

Bloom, D.E. (2011). 7 billion and counting. Science, 333(6042), 562569. doi:10.1126/science.1209290 CrossRefGoogle ScholarPubMed
Centers for Disease Control and Prevention. (2013). The state of aging and health in America. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services.Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155159. doi:10.1037/0033-2909.112.1.155 CrossRefGoogle ScholarPubMed
Cohen, J.E. (2003). Human population: The next half century. Science, 302(5648), 11721175. doi:10.1123/science.1088665 CrossRefGoogle ScholarPubMed
Cohen, R.A., & Sweet, L.H. (2011). Brain imaging in behavioral medicine and clinical neuroscience: Synthesis. In R.A. Cohen & L.H. Sweet (Eds.), Brain imaging in behavioral medicine and clinical neuroscience (pp. 383394). New York: Springer. doi:10.1007/978-1-4419-6373-4 CrossRefGoogle Scholar
Cox, R.W. (1996). AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29, 162173.CrossRefGoogle ScholarPubMed
Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281302. doi:10.1037/h0040957 CrossRefGoogle ScholarPubMed
Debette, S., & Markus, H.S. (2010). The clinical importance of white matter hyperintensities on brain magnetic resonance imaging: Systematic review and meta-analysis. British Medical Journal, 341, c3666. doi:10.1136/bmj.c3666 CrossRefGoogle ScholarPubMed
DeCarli, C., Massaro, J., Harvey, D., Hald, J., Tullberg, M., Au, R., … Wolf, P.A. (2005). Measures of brain morphology and infarction in the Framingham heart study: Establishing what is normal. Neurobiology of Aging, 26(4), 491510. doi:10.1016/j.neurobiolaging.2004.05.004 CrossRefGoogle Scholar
Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). Delis Kaplan Executive Function System. San Antonio, TX: The Psychological Corporation.Google Scholar
Duering, M., Gesierich, B., Seiler, S., Pirpamer, L., Gonik, M., Hofer, E., … Dichgans, M. (2014). Strategic white matter tracts for processing speed deficits in age-related small vessel disease. Neurology, 84, 19461950. doi:10.1212/WNL.0000000000000475 CrossRefGoogle Scholar
Eckert, M.A. (2011). Slowing down: Age-related neurobiological predictors of processing speed. Frontiers in Neuroscience, 5, 25. doi:10.3389/fnins.2011.00025 CrossRefGoogle ScholarPubMed
Finkel, S.I., Mintzer, J.E., Dysken, M., Krishnan, K.R.R., Burt, T., & McRae, T. (2004). A randomized, placebo-controlled study of the efficacy and safety of sertraline in the treatment of the behavioral manifestations of Alzheimer’s disease in outpatients treated with donepezil. International Journal of Geriatric Psychiatry, 19, 918. doi:10.1002/gps.998 CrossRefGoogle ScholarPubMed
Fischl, B., Salat, D.H., Busa, E., Albert, M., Dieterich, M., Hasselgrove, C., … Dale, A.M. (2002). Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain. Neuron, 33(3), 341355.CrossRefGoogle ScholarPubMed
Folstein, M.F., Folstein, S.E., & McHugh, P.R. (1975). “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189198. doi:10.1016/0022-3956(75)90026-6 CrossRefGoogle ScholarPubMed
Gunning-Dixon, F.M., Brickman, A.M., Cheng, J.C., & Alexopoulos, G.S. (2009). Aging of cerebral white matter: A review of MRI findings. International Journal of Geriatric Psychiatry, 24(2), 109117. doi:10.1002/gps.2087 CrossRefGoogle ScholarPubMed
Gunning-Dixon, F.M., & Raz, N. (2000). The cognitive correlates of white matter abnormalities in normal aging: A quantitative review. Neuropsychology, 14, 224232. doi:10.1037//0894-4105.14.2.224 CrossRefGoogle ScholarPubMed
Hemphill, J.F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58(1), 7879. doi:10.1037/0003-066X.58.1.78 CrossRefGoogle ScholarPubMed
Langenecker, S.A., Nielson, K.A., & Rao, S.M. (2004). fMRI of healthy older adults during Stroop interference. Neuroimage, 21, 192200. doi:10.1016/j.neuroimage.2003.08.027 CrossRefGoogle ScholarPubMed
Leavitt, V.M., Wylie, G., Genova, H.M., Chiaravalloti, N.D., & Deluca, J. (2012). Altered effective connectivity during performance of an information processing speed task in multiple sclerosis. Multiple Sclerosis, 18(4), 409417. doi:10.1177/1352458511423651 CrossRefGoogle ScholarPubMed
Lezak, M.D., Howieson, D.B., Bigler, E.D., & Tranel, D. (2012). Neuropsychological assessment (5th ed.). New York: Oxford University Press.Google Scholar
Matthews, P.M., Honey, G.D., & Bullmore, E.T. (2006). Applications of fMRI in translational medicine and clinical practice. Nature Reviews, Neuroscience, 7(9), 732744. doi:10.1038/nrn1929 CrossRefGoogle ScholarPubMed
Papp, K.V., Kaplan, R.F., Springate, B., Moscufo, N., Wakefield, D.B., Guttmann, C.R., & Wolfson, L. (2014). Processing speed in normal aging: Effects of white matter hyperintensities and hippocampal volume loss. Aging, Neuropsychology, and Cognition, 21(2), 197213. doi:10.1080/13825585.2013.795513 CrossRefGoogle ScholarPubMed
Park, D.C., & Reuter-Lorenz, P. (2009). The adaptive brain: Aging and neurocognitive scaffolding. Annual Review of Psychology, 60, 173196. doi:10.1146/annurev.psych.59.103006.093656 CrossRefGoogle ScholarPubMed
Phelps, E.A., Hyder, F., Blamire, A.M., & Shulman, R.G. (1997). FMRI of the prefrontal cortex during overt verbal fluency. Neuroreport, 8, 561565.CrossRefGoogle ScholarPubMed
Rabbitt, P. (2002). Aging and cognition. In: H. Pashler (Ed.), Steven’s handbook of experimental psychology (3rd ed.). New York: John Wiley & Sons.Google Scholar
Rabbitt, P., Scott, M., Lunn, M., Thacker, N., Lowe, C., Pendelton, N., … Jackson, A. (2007). White matter lesions account for all age-related declines in speed but not in intelligence. Neuropsychology, 21(3), 363370. doi:10.1037/0894-4105.21.3.363 CrossRefGoogle ScholarPubMed
Randolph, C., Tierney, M.C., Mohr, E., & Chase, T.N. (1998). The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): Preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology, 20(3), 310319.CrossRefGoogle ScholarPubMed
Riskin-Jones, H.H., Xu, X., Clark, U.S., Labbe, D.R., & Sweet, L.H. (2014, April). New tool for quantification of white matter hyperintensities. Poster presented at the meeting of the Cognitive Aging Conference, Atlanta, Georgia.Google Scholar
Rypma, B., & Prabhakaran, V. (2009). When less is more and when more is more: The mediating roles of capacity and speed in brain-behavior efficiency. Intelligence, 37(2), 207222. doi:10.1016/j.intell.2008.12.004 CrossRefGoogle ScholarPubMed
Saad, Z.S. (2004). SUMA: An interface for surface-based intra- and inter-subject analysis with AFNI. Biomedical Imaging: Nano to Macro, 2, 15101513. doi:10.1109/ISBI.2004.1398837 Google Scholar
Salthouse, T.A. (1996). The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3), 403428. doi:10.1037/0033-295X.103.3.403 CrossRefGoogle ScholarPubMed
Salthouse, T.A. (2000). Aging and measures of processing speed. Biological Psychology, 54, 3554. doi:10.1016/S0301-0511(00)00052-1 CrossRefGoogle ScholarPubMed
Salthouse, T.A., & Coon, V.E. (1993). Influence of task-specific processing speed on age differences in memory. Journal of Gerontology, 48, 245255.CrossRefGoogle ScholarPubMed
Salthouse, T.A., & Ferrer-Caja, E. (2003). What needs to be explained to account for age-related effects on multiple cognitive variables? Psychology and Aging, 18(1), 91110. doi:10.1037/0882-7974.18.1.91 CrossRefGoogle ScholarPubMed
Schmidt, R., Scheltens, P., Erkinjuntti, T., Pantoni, L., Markus, H.S., Wallin, A., … Fazekas, F. (2004). White matter lesion progression: A surrogate endpoint for trials in cerebral small-vessel disease. Neurology, 63(1), 139144.CrossRefGoogle ScholarPubMed
Shulman, K.I., Herrmann, N., Brodaty, H., Chiu, H., Lawlar, B., Ritchie, K., & Scanlan, J.M. (2006). IPA survey of brief cognitive screening instruments. International Psychogeriatrics, 18(2), 281294. doi:10.1017/S1041610205002693 CrossRefGoogle ScholarPubMed
Söderlund, H., Nilsson, L.G., Berger, K., Breteler, M.M., Dufouil, C., Fuhrer, R., … Launer, L.J. (2006). Cerebral changes on MRI and cognitive function: The CASCADE study. Neurobiology of Aging, 27(1), 1623. doi:10.1016/j.neurobiolaging.2004.12.008 CrossRefGoogle ScholarPubMed
Staffen, W., Mair, A., Zauner, H., Unterrainer, J., Neiderhofer, H., Kutzelnigg, A., … Ladurner, G. (2002). Cognitive function and fMRI in patients with multiple sclerosis: Evidence for compensatory cortical activation during an attention task. Brain, 125, 12751282. doi:10.1093/brain/awf125 CrossRefGoogle ScholarPubMed
Sweet, L.H., Paskavitz, J.F., O’Connor, M.J., Browndyke, J.N., Wellen, J.W., & Cohen, R.A. (2005). FMRI correlates of the WAIS-III Symbol Search subtest. Journal of the International Neuropsychological Society, 11, 471476. doi:10.1017/S1355617705050575 CrossRefGoogle ScholarPubMed
van den Heuvel, D.M., ten Dam, V.H., de Craen, A.J., Admiraal-Behloul, F., Olofsen, H., Bollen, E.L., … van Buchem, M.A. (2006). Increase in periventricular white matter hyperintensities parallels decline in mental processing speed in a non-demented elderly population. Journal of Neurology, Neurosurgery, and Psychiatry, 77(2), 149153. doi:10.1136/jnnp.2005.070193 CrossRefGoogle Scholar
Wechsler, D. (1997). Wechsler Adult Intelligence Scale, Third Edition. New York: Pearson.Google Scholar
Wechsler, D. (2001). Wechsler Test of Adult Reading. New York: Pearson.Google Scholar
Wechsler, D. (2008). Wechsler Adult Intelligence Scale, Fourth Edition. New York: Pearson.Google Scholar
Wen, W., & Sachdev, P. (2004). The topography of white matter hyperintensities on brain MRI in healthy 60- to 64-year-old individuals. Neuroimage, 22, 144154. doi:10.1016/j.neuroimage.2003.12.027 CrossRefGoogle ScholarPubMed
Whiting, W.L. IV, & Smith, A.D. (1997). Differential age-related processing limitation in recall and recognition tasks. Psychology and Aging, 12(2), 216224. doi:10.1037/0882-7974.12.2.216 CrossRefGoogle ScholarPubMed
Figure 0

Table 1 Participant demographics, neuropsychological test battery data, and medications taken

Figure 1

Fig. 1 Sample FMRI Symbol Search and visuomotor control task items. This figure demonstrates how the FMRI Symbol Search task and visuomotor control task were presented to participants, including instructions before task items.

Figure 2

Table 2 Convergent validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age

Figure 3

Table 3 Discriminant validity of the FMRI Symbol Search task (raw scores) and residual effects after controlling age