Hostname: page-component-7b9c58cd5d-hxdxx Total loading time: 0 Render date: 2025-03-15T00:31:49.708Z Has data issue: false hasContentIssue false

The Brixton Spatial Anticipation Test as a test for executive function: Validity in patient groups and norms for older adults

Published online by Cambridge University Press:  01 September 2009

ESTHER VAN DEN BERG*
Affiliation:
Department of Neurology, Rudolf Magnus Institute of Neuroscience, University Medical Center Utrecht, Utrecht, The Netherlands
GUDRUN M. S. NYS
Affiliation:
Laboratory for Neuropsychology, Department of Neurology, Ghent University, Ghent, Belgium
AUGUSTINA M. A. BRANDS
Affiliation:
Neuropsychology, Zuwe Hofpoort/Regional Psychiatric Center, Woerden, The Netherlands Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
CARLA RUIS
Affiliation:
Department of Neurology, Rudolf Magnus Institute of Neuroscience, University Medical Center Utrecht, Utrecht, The Netherlands Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
MARTINE J. E. VAN ZANDVOORT
Affiliation:
Department of Neurology, Rudolf Magnus Institute of Neuroscience, University Medical Center Utrecht, Utrecht, The Netherlands Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
ROY P. C. KESSELS
Affiliation:
Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands Donders Institute for BrainCognition, and Behavior, Radboud University Nijmegen, Nijmegen, The Netherlands Departments of Medical Psychology and Geriatrics, Radboud University, Nijmegen Medical Center, Nijmegen, The Netherlands
*
*Correspondence and reprint requests: Esther van den Berg, University Medical Center Utrecht, Department of Neurology G03.228, P.O. Box 85500, 3508 GA Utrecht, The Netherlands. E-mail: e.vandenberg-6@umcutrecht.nl
Rights & Permissions [Opens in a new window]

Abstract

Impairments in executive functioning frequently occur after acquired brain damage, in psychiatric disorders, and in relation to aging. The Brixton Spatial Anticipation Test is a relatively new measure for assessing the ability to detect and follow a rule, an important aspect of executive functioning. To date, normative data on this task are limited, particularly concerning the elderly. This study presents age- and education-adjusted regression-based norms obtained in a group of healthy older participants (n = 283; mean age 67.4 ± 8.5 years). The applicability and validity of these norms were further examined in different groups of patients with stroke (n = 106), diabetes mellitus (n = 376), MCI/early dementia (n = 70), psychiatric disorders (n = 63), and Korsakoff’s syndrome (n = 41). The results showed that patients with Korsakoff’s syndrome, stroke, and psychiatric disorders performed significantly worse than healthy controls. Test-retest correlation (n = 83), learning effects, and correlations with other neuropsychological tests were also explored. Based on the present study, the Brixton test appears a useful addition to existing measures of executive functioning. Moreover, the test can be reliably applied in different groups of clinical patients. (JINS, 2009, 15, 695–703.)

Type
Research Articles
Copyright
Copyright © The International Neuropsychological Society 2009

INTRODUCTION

Executive functions refer to the ability to respond in an adaptive manner to novel situations and are the basis of many cognitive, emotional, and social skills. Executive functions include volition, planning, purposive action, and effective performance, each of which involves a set of activity-related behaviors (Lezak, Howieson, & Loring, Reference Lezak, Howieson and Loring2004) that can be referred to as “cognitive control.” Deficits in executive functions frequently occur after acquired brain injury, particularly after damage to the dorsolateral prefrontal circuit that connects the lateral convexity or the frontal lobe to subcortical structures, such as the dorsolateral portions of the caudate nucleus, and distinct areas of the globus pallidus, substantia nigra, and the thalamus (Cummings, Reference Cummings1995). Decreased executive functioning is also linked to age-related cognitive decline (Brennan, Welsh, & Fisher, Reference Brennan, Welsh and Fisher1997), possibly in relation to impairment of (parts of) the supervisory attentional system (SAS) and largely independent of a more general decline in processing speed (Andres & Van der Linden, Reference Andres and Van der Linden2000). Furthermore, impaired executive functioning has been reported in several psychiatric disorders, for example, schizophrenia (Kerns, Nuechterlein, Braver, & Barch, Reference Kerns, Nuechterlein, Braver and Barch2008) and major depressive disorder (Channon & Green, Reference Channon and Green1999).

Several measures have been developed to tap crucial aspects of executive functioning, such as rule detection, concept shifting, and sorting, in clinical populations. Well-known tests that are used in clinical practice are the Wisconsin Card Sorting Test (WCST; Heaton, Reference Heaton1981), the Tower of London (ToL; Shallice, Reference Shallice1982), and the Trail Making Test, Part B (TMT-B; Reitan, Reference Reitan1956). Although frequently used, these tests typically only tap single aspects of executive functioning (e.g., either planning or rule detection) and a patient’s performance on one executive function test may thus have little predictive value for performance on another test, let alone in complex real-world situations (“ecological validity”; see also Burgess, Alderman, Evans, Emslie, & Wilson, Reference Burgess, Alderman, Evans, Emslie and Wilson1998; Chan, Shum, Toulopoulou, & Chen, Reference Chan, Shum, Toulopoulou and Chen2008; Odhuba, van den Broek, & Johns, Reference Odhuba, van den Broek and Johns2005). Tests that resemble “real-world” tasks, such as the Behavioural Assessment of the Dysexecutive Syndrome (BADS; Wilson, Alderman, Burgess, Emslie, & Evans, Reference Wilson, Alderman, Burgess, Emslie and Evans1996) may help to overcome this issue, but are usually time-consuming as part of a full neuropsychological examination. Accurate and valid assessment of executive functions in clinical populations is also hampered by the paradoxical need to structure a situation in which patients can show whether and how well they can make structure for themselves (Lezak et al., Reference Lezak, Howieson and Loring2004).

A relatively new measure designed to assess aspects of executive functioning is the Brixton Spatial Anticipation Test, developed by Burgess and Shallice (Reference Burgess and Shallice1997) as part of the Hayling and Brixton tests. The task primarily measures a person’s ability to detect a rule, to follow it, and to switch to a new rule. It does not require a verbal or complex motor response by the patient, it has no time restriction, and takes up to 10 minutes to administer. To date, normative data on this task are limited, particularly in the elderly. Burgess and Shallice (Reference Burgess and Shallice1997) report on a relatively small sample (n = 121) of healthy individuals aged 18–80 years, which included only 18 persons >66 years. Another study showed significantly worse performance of older persons (n = 48; aged between 60 and 70 years) compared to younger adults (n = 47; aged between 20 to 30 years) (Andres & Van der Linden, Reference Andres and Van der Linden2000). More recently, Bielak, Mansueti, Strauss, & Dixon (Reference Bielak, Mansueti, Strauss and Dixon2006) published age-stratified normative data in a large sample of elderly persons (n = 457; ages between 53 and 90) Although several authors of the aforementioned studies point out modest correlations between the Brixton test and measures of (fluid) intelligence, educational level has not been included and corrected for in the normative data in any of these studies. This is important because executive functioning is closely related to intelligence in healthy (de Frias, Dixon, & Strauss, Reference de Frias, Dixon and Strauss2006), as well as in brain-injured persons (Wood & Liossi, Reference Wood and Liossi2007). Furthermore, although elderly persons in the general population more frequently have a lower level of education, the education levels in the reported samples was substantially above average (Burgess & Shallice: estimated mean IQ ~110, Andres et al.: all participants ≥12 years of schooling, Bielak et al.: >90% of the participants had at least 12 years of education).

Additionally, studies on the applicability and validity of the Brixton test in different patient groups are also limited. Examples include a study by Burgess and Shallice (Reference Burgess and Shallice1997) who report on 77 neurological patients with heterogeneous lesions of mixed etiology, showing worse performance than healthy controls. A limited number of recent studies report on Brixton test performance in patients with Parkinson’s disease (Rochester et al., Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2004, Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2005), frontal lesions (Reverberi, Lavaroni, Gigli, Skrap, & Shallice, Reference Reverberi, Lavaroni, Gigli, Skrap and Shallice2005), eating disorders (Tchanturia et al., Reference Tchanturia, Anderluh, Morris, Rabe-Hesketh, Collier, Sanchez and Treasure2004), unipolar depression (Gohier et al., Reference Gohier, Ferracci, Surguladze, Lawrence, El and Kefi2008), and in children with Attention-Deficit Hyperactivity Disorder (Shallice et al., Reference Shallice, Marzocchi, Coser, Del, Meuter and Rumiati2002), but a careful examination of the Brixton test performance in clinical populations is lacking. In addition, test-retest reliability was reported by Burgess and Shallice only (r = .71, p < .001).

The aim of the present study is twofold. First, we present age- and education-adjusted regression-based normative data for the Brixton test based on a large sample of healthy older persons. Second, we examine the applicability and validity of these norms in different patient groups and explore test-retest reliability using the classification according to Strauss, Sherman, & Spreen (Reference Strauss, Sherman and Spreen2006).

METHODS

Participants

The present study included a group of healthy participants (n = 283; ages between 55 and 92 years) and five different patient groups, namely, patients with stroke (n = 106; ages between 17 and 83 years), diabetes mellitus (n = 376; ages between 52 and 86 years), MCI/early dementia (n = 70; ages between 32 and 86 years), psychiatric disorders (n = 63; ages between 18 and 57 years), and patients with Korsakoff’s syndrome (n = 41; ages between 34 and 68 years).

The healthy participants were recruited through an advertisement in the local newspaper, through the patients participating in the Hoorn study (van den Berg et al., Reference van den Berg, Dekker, Nijpels, Kessels, Kappelle and De Haan2008), the Anglo-Danish-Dutch Study of Intensive Treatment in People with Screen Detected Diabetes in Primary Care (ADDITION) study (Sandbaek et al., Reference Sandbaek, Griffin, Rutten, Davies, Stolk and Khunti2008), the Utrecht Diabetic Encephalopathy Study (UDES) (Brands et al., Reference Brands, van den Berg, Manschot, Biessels, Kappelle, De Haan and Kessels2007), and through a database of Utrecht University, the Netherlands. Only healthy participants >50 years of age were included.

The patients with stroke were recruited through the departments of Neurology of the University Medical Center Utrecht and the St. Elisabeth and TweeSteden Hospital in Tilburg and were examined between 6 and 10 months after the stroke (mean 7.5 ± 1.3 months) (Cognition in Acute Stroke Patients and After Recovery study (CASPAR; Nys et al., Reference Nys, Van Zandvoort, De Kort, Jansen, Van der Worp, Kappelle and De Haan2005). Ninety-six patients had an ischemic cerebral infarct, 10 patients had an intracerebral hemorrhage. Forty-two patients had a stroke in the left hemisphere, 37 in the right hemisphere, 3 patients had a bilateral stroke, 15 patients had a stroke in the brain stem or cerebellum, and 9 patients had a stroke with an unclear localization.

The patients with diabetes (diabetes duration >1 year) were recruited through general practitioners in the regions of Breda (ADDITION), Hoorn (Hoorn Study), Utrecht, the Hofpoort Hospital in Woerden, and the Groene Hart Hospital in Gouda (UDES). Forty patients had type 1 diabetes and 336 had type 2 diabetes.

The patients with Mild Cognitive Impairment (MCI) or early dementia were recruited through the Alzheimer Center at the Radboud University Nijmegen Medical Center in Nijmegen, the Netherlands. Most patients had relatively mild cognitive decrements: mean Mini-Mental State Examination (MMSE) score was 26.3 ± 2.7, 16% had an MMSE score ≤23.

The patients with psychiatric disorders were recruited thought the outpatient clinic of the Regional Psychiatric Center in Woerden, the Netherlands, who were referred with cognitive complaints. The majority of patients was diagnosed with schizophrenia/psychosis (n = 30) or mood/anxiety disorders (n = 15). Other diagnoses included addiction/substance abuse (n = 2), personality disorder (n = 3), obsessive-compulsive disorder (n = 3), or other diagnoses (n = 10). None of the patients were psychotic at the time of testing.

The patients with Korsakoff’s syndrome were inpatients of the Korsakoff clinic of the Vincent van Gogh Institute for Psychiatry in Venray, the Netherlands. All patients were in the chronic, amnesic stage of the syndrome; none of the patients was in the confusional phase at the moment of testing. The amnesic syndrome was confirmed by extensive neuropsychological testing. All patients were alcohol-abstinent since their admittance to the clinic. They fulfilled the criteria for Korsakoff’s syndrome as described by Kopelman (Reference Kopelman2002). None of the patients fulfilled the clinical criteria for alcohol dementia (Oslin, Atkinson, Smith, & Hendrie, Reference Oslin, Atkinson, Smith and Hendrie1998).

All participants were of Caucasian descent and were able to communicate in the Dutch language. None of the patients or healthy participants had visual impairments or neglect that hampered the administration of the tests. Exclusion criteria for healthy participants were a fasting blood glucose ≥7.0 mmol/l (i.e., diabetes mellitus), history of alcohol abuse, psychiatric illness, or neurological disease (self report). Healthy participants recruited via the Hoorn Study were included only if they had no more than one risk factor for the metabolic syndrome (i.e., increased waist circumference, increased triglycerides, low high-density lipoprotein cholesterol, high blood pressure). The studies were approved by the local medical ethics committees and were completed in accordance with the guidelines of the Helsinki Declaration. Education level was recorded using seven categories (1 being the lowest, less than primary school, and 7 the highest, academic degree; Verhage, Reference Verhage1964) and transformed to number of years of education (Hochstenbach, Mulder, van Limbeek, Donders, & Schoonderwaldt, Reference Hochstenbach, Mulder, van Limbeek, Donders and Schoonderwaldt1998).

Brixton Spatial Anticipation Test

Although the Brixton test was developed as part of the Hayling and Brixton tests (Burgess & Shallice, Reference Burgess and Shallice1997), the present study focuses on performance on the Brixton test only. The Brixton test was administrated and scored according to the instructions in the manual (Burgess & Shallice, Reference Burgess and Shallice1997). Participants were presented with a 56-page stimulus booklet. Each page contained an array of 10 circles (2 rows of 5 circles), which were numbered from 1 to 10 (Figure 1). On each page, one circle was blue. The position of the blue circle changed form one page to the next and the changes were governed by a series of simple rules that changed without warning. Participants were presented with one page at a time and were asked to point to where they thought the blue circle would be on the next page based on the rule inferred form the previous pages. Responses were considered correct if they followed the present rule, and on trials where the rule changed, a response was correct if it followed the previous rule. The total number of errors across 55 trials was used as an outcome measure, higher scores thus reflect worse performance (maximum number of errors = 55).

Fig. 1. Illustration of an actual sequence of the Brixton Spatial Anticipation Test.

Other Neuropsychological Measures

All participants performed a full neuropsychological examination covering the major cognitive domains. However, for the present analysis of the Brixton test, we included only the TMT, as another measure of executive functioning, and the Rey Auditory Verbal Learning Test (RAVLT; van der Elst, van Boxtel, van Breukelen, & Jolles, Reference van der Elst, van Boxtel, van Breukelen and Jolles2005), as a measure of verbal learning, to explore construct validity. It should be noted that performance on these tests will not be independent, for example, because memory at least in part relies on intact executive functioning (Duff, Schoenberg, Scott, & Adams, Reference Duff, Schoenberg, Scott and Adams2005), but theoretically it can be argued that performance on the Brixton test is more closely related to TMT (part B) performance than to performance on the RAVLT. For the TMT the score for part A (seconds), part B (seconds), and the (B − A)/A ratio were included, to control for performance on the TMT part A. For the RAVLT, the total score (the number of words recalled over the five learning trials, possible range 0–75), the delayed recall score (the total number of words after a 30 minute delay, possible range 0–15), and the delayed recognition (total correct hits and rejections after a 30 minute delay, possible range 0–30) were computed.

Participants also performed the Dutch version of the National Adult Reading Test (NART; Schmand, Lindenboom, & Van Harskamp, Reference Schmand, Lindenboom and Van Harskamp1992) as an estimate of (premorbid) verbal general intelligence. The NART and the TMT were not administered in the Korsakoff patients, and only half (n = 18) of the patients in this group completed the RAVLT. The TMT was not performed by the persons participating in the CASPAR study. The NART, TMT, and RAVLT scores were missing in 24, 15, and 17 healthy control participants, respectively, because of study design and/or logistic reasons.

Statistical Analysis

For presentation purposes only, the age distribution and mean Brixton error score of the healthy control group was calculated in 10-year sections (Table 1). Between-group differences in Brixton error scores were examined with analysis of covariance (ANCOVA) with age, sex, and years of education entered as covariates. The relation between the Brixton number of errors, age, sex, years of education, and NART-IQ was determined with Pearson correlations. Regression-based norms for use in clinical practice were determined by means of linear regression analysis, which included the Brixton error score as the dependent variable and the relevant demographical variables (based on the correlation analysis) as predictors (see Results section for details). Subsequently, these normative data were used to examine performance of the five patient groups, and the number of patients scoring “below average” (<16%) and “impaired” (<5%) were calculated. The test’s construct validity was explored by means of Principal Axis Factoring (PAF) with oblique (direct oblimin) rotation, including only those factors with an eigenvalue >1, which allows for correlation between different factors. Receiver Operating Characteristics (ROC) analyses were performed to examine classification accuracy for the control group compared to the patients with stroke and the patients with Korsakoff’s syndrome, respectively. The Area Under the Curve (AUC) was determined for each ROC curve, and cut-off points, sensitivity, and specificity were determined for the Brixton error score. Test-retest correlation was examined in 83 healthy controls who performed the Brixton test two times.

Table 1. Age distribution and mean Brixton error score of the control group (n = 283)

RESULTS

The age distribution of the healthy control group (Table 1) shows that the majority of this group was aged between 60 and 79 (mean 67.4 ± 8.5 years). Table 2 shows characteristics of the healthy control group and the five patient samples. Between-group differences in Brixton error score, examined with age-, sex- and years of education-adjusted ANCOVA, revealed a main effect of group on the Brixton errors score, F(5, 930) = 18.0, p < .001. Subsequent age-, sex- and years of education-adjusted analysis showed that the patients with stroke, F(1, 384) = 21.5, p < .001, Korsakoff’s syndrome, F(1, 319) = 48.9, p < .001, and psychiatric disorders, F(1, 341) = 5.1, p < .05, performed significantly worse than the healthy control group. The Brixton error score of the other patient groups did not significantly differ from the healthy control group. Within the MCI/early dementia group, persons with a low MMSE score (≤23 points) performed significantly worse than persons with a higher MMSE score, F(1, 64) = 7.0, p < .05. Within the group of patients with stroke there was no difference in Brixton error score between the right- and left-hemisphere patients (p = .26). Also, no difference in Brixton score was found between the type 1 and type 2 diabetes patients, p = .77).

Table 2. Characteristics of the healthy controls and the four patient groups

Note

Data are presented as mean ± SD unless otherwise specified.

Correlation analysis within the healthy control group showed that the Brixton number of errors correlated with age (r = .30, p < .001), years of education (r = −.18, p < .01), and NART-IQ (r = −.22, p < .001), but not with sex (r = .04, p = .50). To determine regression-based normative data, linear regression analysis was performed for the Brixton number of errors, with age and years of education entered as predictor variables to determine the function for the expected score (ES). Since a NART-IQ is not always available in clinical practice, this variable was not used as a predictor variable for the normative data. Linear regression analyses resulted in the following function for the Brixton number of errors:

(1)
$$Expected\,Total\,score\, = \,6.12\, + \,\left({0.23\, \times \,Age} \right)\, - \,\left({0.24\, \times \,Education\,in\,years} \right)$$

Residue scores (RS) were computed by subtracting each individual’s expected score from the observed score (RS = OS − ES). The frequency distribution for the residue scores for the Brixton errors were converted into a percentile distribution (Table 3). Clinically, a score below the 5th percentile can be used as a cut-off point for an “impaired” performance (Lezak et al., Reference Lezak, Howieson and Loring2004).

Table 3. Percentile distribution of the residue scores of the Brixton error score, based on the healthy control group (n = 283)

Application of these age- and education-adjusted norms on the five patient samples (Table 4) shows significant between-group differences of the residue scores, F(5, 933) = 21.1, p < .001. The patients with Korsakoff’s syndrome, stroke, and psychiatric disorders performed significantly worse than the control group. Performance of the patients with diabetes was largely similar to the control group. The number of patients performing at the “below average” and “impaired” level also differed significantly between the groups, χ2 (5) = 52.6, p < .001 and χ2 (5) = 72.9, p < .001, respectively, (Table 4). Interestingly, whereas only the patients with Korsakoff’s syndrome showed “impaired” performance significantly more often, three of the five patients groups (Korsakoff, stroke, psychiatric disorders) performed in the “below average” range more frequently, suggesting that the Brixton test is also sensitive to more subtle impairments. Although the patients with MCI/early dementia showed clear impairments on the RAVLT compared to the controls and most patient groups (Table 2), they did not show worse performance on the Brixton test (Table 4).

Table 4. Brixton performance of different patient groups compared to the age- and education-adjusted normative data from healthy controls (n = 283)

Note

All comparisons were performed against the reference group.

* p < .05.

PAF with direct oblimin rotation was used to explore the rest’s construct validity. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.60, and Bartlett’s test of sphericity was significant, χ2(15) = 335.5, p < .001, indicating high sampling adequacy. The PAF yielded a multicomponent solution consisting of 2 factors with eigenvalues >1.0 that accounted for 60% of the variance. The RAVLT total score and delayed recall score loaded high on the first factor, and the TMT A showed a negative factor loading on this factor, which could be regarded as a verbal memory factor (eigenvalue 2.33, 39% of variance explained, factor loadings RAVLT total score 0.90, RAVLT delayed recall 0.82, TMT A –0.40). The second factor encompassed high loadings for the TMT (B − A)/A ratio score and the Brixton score. The RAVLT total score, delayed recall score, as well as the NART score showed negative factor loadings on this factor, which could be regarded as a mental flexibility factor (eigenvalue 1.28, 21% of variance explained, factor loadings TMT (B − A/A) ratio 0.76, Brixton 0.37, RAVLT total score −0.44, RAVLT delayed recall −0.38, NART-IQ –0.44).

The ROC-analysis of the Brixton test for the control group compared to the patients with Korsakoff’s syndrome resulted in a reasonable overall classification accuracy (AUC = 0.74; 95% confidence interval [CI] 0.66 – 0.82, p < .001). The optimal cut-off score of 19/20 errors resulted in a sensitivity of 81% and a specificity of 62%, which can be interpreted as adequate (Blake, McKinney, Treece, Lee, & Lincoln, Reference Blake, McKinney, Treece, Lee and Lincoln2002). For the control group, compared to the patients with stroke, however, the AUC was 0.56 (95% CI 0.50 – 0.63, p = .06), which is only marginally above chance level. Similarly, the optimal cut-off score of 18/19 resulted in a sensitivity of 56% and a specificity of 58%.

Eighty-three healthy controls performed the Brixton test again after 6 to 48 months. At baseline, the mean Brixton error score for these controls was 19.6 ± 8.0, at follow-up the mean score was 19.6 ± 5.3. Test-retest correlation was 0.61 (p < .001).

DISCUSSION

The current study presents age- and education-adjusted regression-based normative data for the Brixton Spatial Anticipation Test based on a sample of 283 older healthy adults that can be used in clinical practice. Application of these normative data in five different patient samples further showed that patients with Korsakoff’s syndrome, the stroke patients, and the patients with psychiatric disorders, as a group, showed significantly worse performance than healthy controls. Dysexecutive symptoms have been previously described in Korsakoff patients in relation to frontal lobe damage (Kessels, Kortrijk, Wester, & Nys, Reference Kessels, Kortrijk, Wester and Nys2008; Reed et al., Reference Reed, Lasserson, Marsden, Stanhope, Stevens and Bello2003). In stroke patients, the presence of impairments in executive functioning is largely dependent on lesion location, the frontal lobe in particular (Vataja et al., Reference Vataja, Pohjasvaara, Mantyla, Ylikoski, Leppavuori and Leskela2003), but also related to damage in other parts of the brain (Nys et al., Reference Nys, Van Zandvoort, De Kort, Jansen, Van der Worp, Kappelle and De Haan2005). Although an in-depth examination of the relation of stroke location and performance on the Brixton test is beyond the scope of the present study, lateralization differences in Brixton performance could not be demonstrated. In addition to a significantly worse performance on the Brixton test, the group of patients with psychiatric disorders also showed a significantly greater proportion of patients with a “below average” score, indicating that the Brixton test is also sensitive to more subtle impairments. When considering that the main psychiatric diagnosis of our sample of patients was psychotic disorders, this finding is in line with other studies showing dysexecutive problems in these patients (Hutton et al., Reference Hutton, Puri, Duncan, Robbins, Barnes and Joyce1998; Kerns et al., Reference Kerns, Nuechterlein, Braver and Barch2008). The results of this study further support the notion that persons with MCI or early dementia, probably due to Alzheimer’s disease, generally do not show marked decline in executive functioning in the early stages of their disease (Levinoff et al., Reference Levinoff, Phillips, Verret, Babins, Kelner, Akerib and Chertkow2006; Petersen et al., Reference Petersen, Smith, Waring, Ivnik, Tangalos and Kokmen1999). As expected, these patients predominantly showed a worse performance on the RAVLT (Table 2). However, it should be noted that because the Brixton data for the MCI and early dementia group were collected as part of clinical assessment at a memory clinic, selection bias could have occurred. That is, dementia patients with more severe cognitive deficits may have been incapable of adequately performing complex executive tasks, such as the Brixton or the TMT B according to the test instructions (Ashendorf et al., Reference Ashendorf, Jefferson, O’Connor, Chaisson, Green and Stern2008), resulting in exclusion from the present study. The patients with diabetes in the present study largely performed at a similar level as the healthy controls, although there was a trend towards a larger proportion of patients performing at the “below average” level. This finding is in line with previous findings of cognitive decrements in relation to diabetes (Awad, Gagnon, & Messier, Reference Awad, Gagnon and Messier2004; Brands, Biessels, De Haan, Kappelle, & Kessels, Reference Brands, Biessels, De Haan, Kappelle and Kessels2005).

To our knowledge this is the first study that carefully examined the applicability and validity of the Brixton test in circumscribed clinical population samples. Several small case-control studies have used the Brixton test as part of a broader neuropsychological examination. These studies show that older patients with Parkinson’s disease (n = 20) show worse performance on the Brixton test compared to age-matched healthy controls (n = 10) (Rochester et al., Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2004, Reference Rochester, Hetherington, Jones, Nieuwboer, Willems, Kwakkel and Van2005). Patients with unipolar depression (n = 20) also showed a marginally worse performance compared to age-matched healthy controls (n = 20) (Gohier et al., Reference Gohier, Ferracci, Surguladze, Lawrence, El and Kefi2008). In addition, females with anorexia nervosa (n = 34), but not females with bulimia nervosa (n = 19) performed worse than age-matched controls (n = 35) (Tchanturia et al., Reference Tchanturia, Anderluh, Morris, Rabe-Hesketh, Collier, Sanchez and Treasure2004). Two studies used a modified version of the original Brixton test and showed worse performance in a group of patients with prefrontal lesions (n = 40) (Reverberi et al., Reference Reverberi, Lavaroni, Gigli, Skrap and Shallice2005) and in children with Attention Deficit Hyperactivity Disorder (n = 31) (Shallice et al., Reference Shallice, Marzocchi, Coser, Del, Meuter and Rumiati2002), compared to age-matched controls.

The normative data presented in this study are based on linear regression analysis and included age and years of education as predictor variables. Although the reliability of regression-based norms is still debated (e.g., Fastenau, Reference Fastenau1998; Heaton, Avitable, Grant, & Matthews, Reference Heaton, Avitable, Grant and Matthews1999), we feel that this method allows for (1) careful examination of relevant predictor variables of the Brixton error score and (2) the use of the data of the entire sample instead of stratified norms based on different age and education groups, resulting in small comparison groups (see also, van Breukelen & Vlaeyen, Reference van Breukelen and Vlaeyen2005). Although normative data for the Brixton test are limited, the norms presented in the present study can be compared with the previously published norms in older persons by Bielak et al. (Reference Bielak, Mansueti, Strauss and Dixon2006). Examination of the residue scores (RS = OS ES) for each age group presented in their study shows that their participants generally perform at the 30th–40th percentile of our distribution. This shows that our results are consistent with previously published norms, and further suggests that, compared to our norms, the norms by Bielak et al. may overestimate Brixton performance by ~10 percentiles, which may have been caused by the relatively high level of education of their study sample.

To date, examination of construct validity of the Brixton test is lacking altogether. The present factor analysis resulted in a two-factor solution in which a mental flexibility factor, which included the Brixton error score and TMT (B − A)/A ratio score, could be dissociated from a verbal memory factor. These results indicate that the Brixton error score is indeed more related with another measure of executive functioning (TMT (B − A)/A ratio) than with measures of memory and speed, which would be expected because both tests rely on mental flexibility and set shifting. This suggests satisfactory convergent validity for the Brixton test. Aspects such as rule detection are, however, not measured by the TMT, and future studies on the relation between the Brixton test and other measures of executive functioning should complement these findings. Moreover, the Brixton error score was not unrelated to memory function (RAVLT total score and delayed recall score). Although inferences about causality cannot be made based on the present cross-sectional study, this association suggests that adequate performance on learning and memory tasks presumes, at least in part, intact executive functioning (for a discussion see, Duff et al., Reference Duff, Schoenberg, Scott and Adams2005). The present study also provides a first indication of the diagnostic accuracy of the Brixton test. ROC analyses resulted in adequate sensitivity and specificity for the Brixton test when comparing the patients with Korsakoff’s syndrome with healthy controls (AUC = 0.74 (0.66 – 0.82)). However, for the stroke patients, compared to the healthy controls, diagnostic accuracy was less satisfying (AUC = 0.56 (0.50 – 0.63)), possibly reflecting larger heterogeneity in cognitive impairments in the stroke group compared to the Korsakoff group. Test-retest correlation was 0.61 in the present study, which is somewhat lower than the 0.71 that was reported by Burgess and Shallice (Reference Burgess and Shallice1997). According to the classification by Strauss et al. (Reference Strauss, Sherman and Spreen2006), this would imply marginal reliability, but it is comparable to test-retest reliability coefficients that are reported for several traditional measures of executive functioning, such as the WCST (test-retest reliability coefficients 0.39 to 0.72 for different WCST measures; Heaton, Chelune, Talley, & Kay, Reference Heaton, Chelune, Talley and Kay1993) and the BADS (0.22 to 0.85 for different BADS subtests; Jelicic, Henquet, Derix, & Jolles, Reference Jelicic, Henquet, Derix and Jolles2001). There was no apparent learning effect between the two measurements.

Strengths of the present study include the large number of healthy older participants, the use of regression-based norms that are adjusted for the confounding effects of age and educational level, and the systematic examination of the applicability and validity in different patient samples. Although the regression-based norms can be applied across a wide age-range, one should keep in mind that our population of healthy persons was aged >50 and <80, and application of these norms in very young or very old persons should be performed with some caution. Furthermore, the present study included both persons who participated in a research project (and therefore invariably performed the Brixton test) and clinical samples of patients, in which selection bias may have played a role.

In summary, the Brixton Spatial Anticipation Test is a relatively new measure of executive functioning. Although it measures only a single aspect of executive functioning, namely to detect and follow a rule, the Brixton test can be viewed as a useful addition to existing measures of executive functions because it does not require a verbal or complex motor response by the patient, it has a relatively short duration, and no apparent learning effect as demonstrated in test-retest analysis. The present study provides age- and education-adjusted regression-based norms for elderly persons, and shows that the Brixton test is a valid and useful measure of executive functioning in different groups of clinical patients.

ACKNOWLEDGMENTS

The authors report no conflict of interest. The information in this manuscript and the manuscript itself have never been published either electronically or in print.

Esther van den Berg and Ineke Brands were supported by Grant No. 2001.00.023 and 2003.01.004 of the Dutch Diabetes Research Foundation. Roy Kessels was supported by a 2006 Utrecht University High Potential research grant. Gudrun Nys and Martine van Zandvoort were supported by the Netherlands Heart Foundation (NHS 2000.023) and the Brain Foundation (The Netherlands). The ADDITION study in the Netherlands was funded by grants from Novo Nordisk Netherlands, GlaxoSmithKline Netherlands, and Merck Netherlands.

The authors would like to thank P. de Kort and B. Jansen (CASPAR), M. de Goede, H. Kortrijk and A. Wester (Korsakoff patients), L. Joosten-Weyn Banningh and N. van Schuylenborg-van Es (MCI/early dementia patients) for enrolment of patients. We also thank the Hoorn study group of the EMGO institute (VU University Medical Center), the Utrecht Diabetic Encephalopathy Study Group (University Medical Center Utrecht), the “Utrecht Diabetes Programma (UDP)” and the “IJsselstein Diabetes Project” (mentor: Ph.L. Salomé) for their assistence.

References

REFERENCES

Andres, P., & Van der Linden, M. (2000). Age-related differences in supervisory attentional system functions. Journal of Gerontology, Series B: Psychological Sciences and Social Sciences, 55, 373380.CrossRefGoogle ScholarPubMed
Ashendorf, L., Jefferson, A.L., O’Connor, M.K., Chaisson, C., Green, R.C., & Stern, R.A. (2008). Trail Making Test errors in normal aging, mild cognitive impairment, and dementia. Archives of Clinical Neuropsychology, 23, 129137.Google ScholarPubMed
Awad, N., Gagnon, M., & Messier, C. (2004). The relationship between impaired glucose tolerance, type 2 diabetes, and cognitive function. Journal of Clinical and Experimental Neuropsychology, 26, 10441080.CrossRefGoogle ScholarPubMed
Bielak, A.A., Mansueti, L., Strauss, E., & Dixon, R.A. (2006). Performance on the Hayling and Brixton tests in older adults: Norms and correlates. Archives of Clinical Neuropsychology, 21, 141149.CrossRefGoogle ScholarPubMed
Blake, H., McKinney, M., Treece, K., Lee, E., & Lincoln, N.B. (2002). An evaluation of screening measures for cognitive impairment after stroke. Age and Ageing, 31, 451456.CrossRefGoogle ScholarPubMed
Brands, A.M., Biessels, G.J., De Haan, E.H., Kappelle, L.J., & Kessels, R.P. (2005). The effects of type 1 diabetes on cognitive performance: A meta-analysis. Diabetes Care, 28, 726735.CrossRefGoogle ScholarPubMed
Brands, A.M., van den Berg, E., Manschot, S.M., Biessels, G.J., Kappelle, L.J., De Haan, E.H., & Kessels, R.P. (2007). A detailed profile of cognitive dysfunction and its relation to psychological distress in patients with type 2 diabetes mellitus. Journal of the International Neuropsychological Society, 13, 288297.CrossRefGoogle ScholarPubMed
Brennan, M., Welsh, M.C., & Fisher, C.B. (1997). Aging and executive function skills: An examination of a community-dwelling older adult population. Perceptual and Motor Skills, 84, 11871197.CrossRefGoogle ScholarPubMed
Burgess, P.W., Alderman, N., Evans, J., Emslie, H., & Wilson, B.A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4, 547558.CrossRefGoogle ScholarPubMed
Burgess, P.W., & Shallice, T. (1997). The Hayling and Brixton Tests. Thurston, UK: Thames Valley Test Company.Google Scholar
Chan, R.C., Shum, D., Toulopoulou, T., & Chen, E.Y. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23, 201216.CrossRefGoogle ScholarPubMed
Channon, S., & Green, P.S. (1999). Executive function in depression: The role of performance strategies in aiding depressed and non-depressed participants. Journal of Neurology, Neurosurgery and Psychiatry, 66, 162171.CrossRefGoogle ScholarPubMed
Cummings, J.L. (1995). Anatomic and behavioral aspects of frontal-subcortical circuits. Annals of the New York Academy of Sciences, 769, 113.CrossRefGoogle ScholarPubMed
de Frias, C.M., Dixon, R.A., & Strauss, E. (2006). Structure of four executive functioning tests in healthy older adults. Neuropsychology, 20, 206214.CrossRefGoogle ScholarPubMed
Duff, K., Schoenberg, M.R., Scott, J.G., & Adams, R.L. (2005). The relationship between executive functioning and verbal and visual learning and memory. Archives of Clinical Neuropsychology, 20, 111122.CrossRefGoogle ScholarPubMed
Fastenau, P.S. (1998). Validity of regression-based norms: An empirical test of the comprehensive norms with older adults. Journal of Clinical and Experimental Neuropsychology, 20, 906916.CrossRefGoogle ScholarPubMed
Gohier, B., Ferracci, L., Surguladze, S.A., Lawrence, E., El, H.W., Kefi, M.Z., et al. . (2009). Cognitive inhibition and working memory in unipolar depression. Journal of Affective Disorders, 116, 100105.CrossRefGoogle ScholarPubMed
Heaton, R.K. (1981). Wisconsin Card Sorting Test. Odessa, FL: Psychological Assessment Resources.Google Scholar
Heaton, R.K., Chelune, G.J., Talley, J.L., & Kay, G.G. (1993). Wisconsin Card Sorting Test (WCST) manual, revised and expanded. Odessa, FL: Psychological Assessment Resources.Google Scholar
Heaton, R.K., Avitable, N., Grant, I., & Matthews, C.G. (1999). Further crossvalidation of regression-based neuropsychological norms with an update for the Boston Naming Test. Journal of Clinical and Experimental Neuropsychology, 21, 572582.CrossRefGoogle ScholarPubMed
Hochstenbach, J., Mulder, T., van Limbeek, L.J., Donders, R., & Schoonderwaldt, H. (1998). Cognitive decline following stroke: A comprehensive study of cognitive decline following stroke. Journal of Clinical and Experimental Neuropsychology, 20, 503517.CrossRefGoogle ScholarPubMed
Hutton, S.B., Puri, B.K., Duncan, L.J., Robbins, T.W., Barnes, T.R., & Joyce, E.M. (1998). Executive function in first-episode schizophrenia. Psychological Medicine, 28, 463473.CrossRefGoogle ScholarPubMed
Jelicic, M., Henquet, C.E., Derix, M.M., & Jolles, J. (2001). Test-retest stability of the behavioural assessment of the dysexecutive syndrome in a sample of psychiatric patients. International Journal of Neuroscience, 110, 7378.CrossRefGoogle Scholar
Kerns, J.G., Nuechterlein, K.H., Braver, T.S., & Barch, D.M. (2008). Executive functioning component mechanisms and schizophrenia. Biological Psychiatry, 64, 2633.CrossRefGoogle ScholarPubMed
Kessels, R.P., Kortrijk, H.E., Wester, A.J., & Nys, G.M. (2008). Confabulation behavior and false memories in Korsakoff’s syndrome: Role of source memory and executive functioning. Psychiatry and Clinical Neurosciences, 62, 220225.CrossRefGoogle ScholarPubMed
Kopelman, M.D. (2002). Disorders of memory. Brain, 125, 21522190.CrossRefGoogle ScholarPubMed
Levinoff, E.J., Phillips, N.A., Verret, L., Babins, L., Kelner, N., Akerib, V., & Chertkow, H. (2006). Cognitive estimation impairment in Alzheimer disease and mild cognitive impairment. Neuropsychology, 20, 123132.CrossRefGoogle ScholarPubMed
Lezak, M.D., Howieson, D.B., & Loring, D.W. (2004). Neuropsychological Assessment (4th ed.). New York: Oxford University Press.Google Scholar
Nys, G.M., Van Zandvoort, M.J., De Kort, P.L., Jansen, B.P., Van der Worp, H.B., Kappelle, L.J., & De Haan, E.H. (2005). Domain-specific cognitive recovery after first-ever stroke: A follow-up study of 111 cases. Journal of the International Neuropsychological Society, 11, 795806.CrossRefGoogle ScholarPubMed
Odhuba, R.A., van den Broek, M.D., & Johns, L.C. (2005). Ecological validity of measures of executive functioning. British Journal of Clinical Psychology, 44, 269278.CrossRefGoogle ScholarPubMed
Oslin, D., Atkinson, R.M., Smith, D.M., & Hendrie, H. (1998). Alcohol related dementia: Proposed clinical criteria. International Journal of Geriatric Psychiatry, 13, 203212.3.0.CO;2-B>CrossRefGoogle ScholarPubMed
Petersen, R.C., Smith, G.E., Waring, S.C., Ivnik, R.J., Tangalos, E.G., & Kokmen, E. (1999). Mild cognitive impairment: Clinical characterization and outcome. Archives of Neurology, 56, 303308.CrossRefGoogle ScholarPubMed
Reed, L.J., Lasserson, D., Marsden, P., Stanhope, N., Stevens, T., Bello, F., et al. . (2003). FDG-PET findings in the Wernicke-Korsakoff syndrome. Cortex, 39, 10271045.CrossRefGoogle ScholarPubMed
Reitan, R. (1956). Trail Making Test: Manual for administration, scoring, and interpretation. Bloomington, IN: Indiana University Press.Google Scholar
Reverberi, C., Lavaroni, A., Gigli, G.L., Skrap, M., & Shallice, T. (2005). Specific impairments of rule induction in different frontal lobe subgroups. Neuropsychologia, 43, 460472.CrossRefGoogle ScholarPubMed
Rochester, L., Hetherington, V., Jones, D., Nieuwboer, A., Willems, A.M., Kwakkel, G., & Van, W.E. (2004). Attending to the task: Interference effects of functional tasks on walking in Parkinson’s disease and the roles of cognition, depression, fatigue, and balance. Archives of Physical Medicine and Rehabilitation, 85, 15781585.CrossRefGoogle Scholar
Rochester, L., Hetherington, V., Jones, D., Nieuwboer, A., Willems, A.M., Kwakkel, G., & Van, W.E. (2005). The effect of external rhythmic cues (auditory and visual) on walking during a functional task in homes of people with Parkinson’s disease. Archives of Physical Medicine and Rehabilitation, 86, 9991006.CrossRefGoogle ScholarPubMed
Sandbaek, A., Griffin, S.J., Rutten, G., Davies, M., Stolk, R., Khunti, K., et al. . (2008). Stepwise screening for diabetes identifies people with high but modifiable coronary heart disease risk. The ADDITION study. Diabetologia, 51, 11271134.CrossRefGoogle ScholarPubMed
Schmand, B., Lindenboom, J., & Van Harskamp, F. (1992). Nederlandse leestest voor volwassenen. Lisse, NL: Swets & Zeitlinger.Google Scholar
Shallice, T. (1982). Specific impairments of planning. Philosophical Transactions of the Royal Society B: Biological Sciences, 298, 199209.Google ScholarPubMed
Shallice, T., Marzocchi, G.M., Coser, S., Del, S.M., Meuter, R.F., & Rumiati, R.I. (2002). Executive function profile of children with attention deficit hyperactivity disorder. Developmental Neuropsychology, 21, 4371.CrossRefGoogle ScholarPubMed
Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A compendium of neuropsychological tests (3rd ed.) New York: Oxford University Press.Google Scholar
Tchanturia, K., Anderluh, M.B., Morris, R.G., Rabe-Hesketh, S., Collier, D.A., Sanchez, P., & Treasure, J.L. (2004). Cognitive flexibility in anorexia nervosa and bulimia nervosa. Journal of the International Neuropsychological Society, 10, 513520.CrossRefGoogle ScholarPubMed
van Breukelen, G.J., & Vlaeyen, J.W. (2005). Norming clinical questionnaires with multiple regression: The Pain Cognition List. Psychological Assessment, 17, 336344.CrossRefGoogle ScholarPubMed
van den Berg, E., Dekker, J.M., Nijpels, G., Kessels, R.P., Kappelle, L.J., De Haan, E.H., et al. . (2008). Cognitive functioning in elderly persons with type 2 diabetes and metabolic syndrome: The Hoorn study. Dementia and Geriatric Cognitive Disorders, 26, 261269.CrossRefGoogle ScholarPubMed
van der Elst, W., van Boxtel, M.P., van Breukelen, G.J., & Jolles, J. (2005). Rey’s verbal learning test: Normative data for 1855 healthy participants aged 24–81 years and the influence of age, sex, education, and mode of presentation. Journal of the International Neuropsychological Society, 11, 290302.CrossRefGoogle ScholarPubMed
Vataja, R., Pohjasvaara, T., Mantyla, R., Ylikoski, R., Leppavuori, A., Leskela, M., et al. . (2003). MRI correlates of executive dysfunction in patients with ischaemic stroke. European Journal of Neurology, 10, 625631.CrossRefGoogle ScholarPubMed
Verhage, F. (1964). Intelligence and age (in Dutch). Assen, NL: van Gorcum.Google Scholar
Wilson, B.A., Alderman, N., Burgess, P.W., Emslie, H., & Evans, J.J., (1996). Behavioural Assessment of the Dysexecutive Syndrome. Bury St. Edmunds, UK: Thames Valley Test Company.Google Scholar
Wood, R.L., & Liossi, C. (2007). The relationship between general intellectual ability and performance on ecologically valid executive tests in a severe brain injury sample. Journal of the International Neuropsychological Society, 13, 9098.CrossRefGoogle Scholar
Figure 0

Fig. 1. Illustration of an actual sequence of the Brixton Spatial Anticipation Test.

Figure 1

Table 1. Age distribution and mean Brixton error score of the control group (n = 283)

Figure 2

Table 2. Characteristics of the healthy controls and the four patient groups

Figure 3

Table 3. Percentile distribution of the residue scores of the Brixton error score, based on the healthy control group (n = 283)

Figure 4

Table 4. Brixton performance of different patient groups compared to the age- and education-adjusted normative data from healthy controls (n = 283)