Introduction
Attention deficit hyperactivity disorder (ADHD) is a neuropsychiatric disorder characterized by hyperactivity, inattention and impulsivity. Traditionally, ADHD has been described as a child psychiatry diagnosis, but research has shown that a large proportion of children with ADHD have persisting symptoms into adulthood (Faraone & Biederman, Reference Faraone and Biederman2005; Lara et al., Reference Lara, Fayyad, de Graaf, Kessler, Aguilar-Gaxiola, Angermeyer and Sampson2009; Mick, Faraone, & Biederman, Reference Mick, Faraone and Biederman2004; Rasmussen & Gillberg, Reference Rasmussen and Gillberg2000). Although not shown by all individuals within the diagnostic group (Willcutt, Doyle, Nigg, Faraone, & Pennington, Reference Willcutt, Doyle, Nigg, Faraone and Pennington2005), impairments of executive functioning (EF) have frequently been reported in ADHD (Barkley, Reference Barkley2010; Castellanos, Sonuga-Barke, Milham, & Tannock, Reference Castellanos, Sonuga-Barke, Milham and Tannock2006).
Deficits in more basic functions have not been adequately addressed in all models of EF in ADHD, despite the fact that lower-order processes are necessary components of higher-order cognitive operations (Rommelse & Buitelaar, Reference Rommelse and Buitelaar2008). For this reason, it is unclear how much of the variance in EF is due to variance in lower order processes. Tests of EF typically involve cognitive functions at different levels of information processing, only some of which probably reflect the intended function (Anderson, Reference Anderson2002; Castellanos et al., Reference Castellanos, Sonuga-Barke, Milham and Tannock2006). It is, therefore, important to include control tasks (Denckla, Reference Denckla1996) in studies of EF in individuals with ADHD. It is also a challenge that such studies often include neuropsychological tests originally designed to differentiate between individuals with brain damage and normal controls: These tests may, therefore, be insensitive to the mild cognitive difficulties commonly seen in individuals with ADHD.
The Delis-Kaplan Executive Function System (D-KEFS) (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001) was developed to deal with such shortcomings. The test battery enables more sensitive and “pure” measures of EF by including more demanding subtests to prevent ceiling effects and by calculating contrast scores to control for basic functions. One of the few studies of ADHD that included tests from D-KEFS showed that measures from the Color Word Interference Test (CWIT), a variant of the original Stroop test, separated children with ADHD from controls (Wodka et al., Reference Wodka, Loftis, Mostofsky, Prahme, Larson, Denckla and Mahone2008). This test measures response inhibition, which has been described as a core symptom (Barkley, Reference Barkley1997b, Reference Barkley1997c) and as one of the most promising endophenotypes for ADHD, defined as “heritable, quantitative traits that index an individual's liability to develop or manifest a given disease” (Castellanos & Tannock, Reference Castellanos and Tannock2002). Although ADHD is thought to be associated with dysfunctional inhibitory processes, inhibition has not been consistently defined across cognitive theories (Nigg, Reference Nigg2001). In addition, it has been difficult to compare the results of studies that have used different tests. Thus, it is imperative to develop coherent, widely accepted test definitions and to implement tests that include valid measures of the inhibition effect.
CWIT also includes a measure of set-shifting. Set-shifting, defined as the ability to move back and forth between multiple tasks, operations and mental sets (Miyake et al., Reference Miyake, Friedman, Emerson, Witzki, Howerter and Wager2000), has been described as another potential endophenotype for ADHD (Boonstra et al., Reference Boonstra, Kooij, Buitelaar, Oosterlaan, Sergeant, Heister and Franke2008). Several studies have investigated set-shifting in ADHD (Barkley, Murphy, & Fisher, 2008; Piek, Dyck, Francis, & Conwell, Reference Piek, Dyck, Francis and Conwell2007; Rohlf et al., Reference Rohlf, Jucksch, Gawrilow, Huss, Hein, Lehmkuhl and Salbach-Andrae2012), but the results have been inconclusive. Tests designed to measure this function have been criticized for having an inadequate level of difficulty, especially for adults, leading to a ceiling effect. Furthermore, most of the research on executive functioning in ADHD has focused on children; more studies are needed to clarify the role of set-shifting abilities in adults.
The CWIT includes two control conditions that measure more basic functions. The third is designed to measure inhibition and the fourth designed to measure both inhibition and set-shifting. However, it is still unclear how much of the variance in the two latter conditions are explained by the two more basic conditions, and how much of the variance in condition 4 are due to condition 3. Studies investigating this relationship with appropriate contrast measures are, therefore, warranted. A strength of D-KEFS is that it includes standardized contrast scores and that the basic functions can be controlled for by running regression analyses (Kramer et al., Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007; Pa et al., Reference Pa, Possin, Wilson, Quitania, Kramer, Boxer and Johnson2010).
Inhibition and set-shifting are not only influenced by more basic functions, but also by the individual's working memory and intellectual abilities (IQ). Kramer et al. (Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007) have suggested that set-shifting is anatomically and functionally related to working memory because the subjects need to attend simultaneously to multiple task parameters, track progress and maintain task instructions online. However, both the results from Kramer et al. and results from other studies indicate that, despite this overlap, set-shifting seems to represent a distinct construct (e.g., Ravizza & Ciranni, Reference Ravizza and Ciranni2002). The influence of IQ on tests of EF (e.g., Adolfsdottir, Sorensen, & Lundervold, Reference Adolfsdottir, Sorensen and Lundervold2008) and ADHD symptoms (Barkley, Reference Barkley1997a; Tillman, Bohlin, Sorensen, & Lundervold, Reference Tillman, Bohlin, Sorensen and Lundervold2009) are documented in studies of children. However, inclusion of working memory and IQ as covariates may also remove some of the variance caused by the disorder itself (Barkley, Reference Barkley1997a). On this background, it has been suggested that results should be presented both with and without relevant covariates (Barkley, Reference Barkley1997a).
In the present study, we first compared inhibition and set-shifting in adults with ADHD and a group of controls by analyzing their CWIT performance. From studies of children we expected the ADHD group to be more impaired than a control group on primary summary measures of inhibition and set-shifting. Second, we investigated contrast scores, both those calculated according to the D-KEFS manual, and standard residuals calculated from raw scores obtained by our Norwegian sample. Finally, we asked if group differences would remain when including IQ and working memory as covariates in the statistical analyses.
Methods
Participants in the Parent Study
The participants with ADHD (>18 years old) were recruited as part of a study that included participants from a national registry of adults diagnosed in Norway from 1997 to May 2005. Diagnostic assessment was conducted by three national expert committees for ADHD/hyperkinetic disorder. Three to five clinicians (mainly psychiatrists and psychologists) with specialized experience in diagnosing ADHD in children and adults served on the committees. Patients were referred to the committees by their psychiatrists, general practitioners or hospital doctors. The procedure for the referral required patient records with thorough descriptions of current symptoms and functioning, comparent information about childhood behavior and functioning and results from both physical and psychiatric examinations. The committees reviewed the patients’ records to confirm or disprove the diagnosis of ADHD.
ICD-10 is the official diagnostic system in Norway. However, allowance was made for the diagnosis of the inattentive subtype in DSM-IV. A total of 1700 invitation letters were sent to adults with ADHD between 2005 and 2007, mainly targeting individuals referred after the year 2000. Psychiatrists and psychologists nation-wide were also invited to recruit adults with ADHD who were formally diagnosed according to the national guidelines based on the criteria used by the expert teams, but without the mandatory evaluation of the committees.
The control group was recruited through the database of the Medical Birth Registry of Norway (MBRN), which includes all individuals born in Norway after January 1, 1967. A total of 2963 invitation letters were sent to a randomly selected nation-wide sample aged between 18 and 40 years old. In addition, a subsample was recruited by means of local advertisements. The control group was not screened for ADHD before entering the study. Although this may have led to reduced power, the prevalence of ADHD in this group is assumed to be lower than 5%. This was supported by the estimated prevalence of 1.7% in a population based study of 8- to 10-year-old Norwegian children (Heiervang et al., Reference Heiervang, Stormark, Lundervold, Heimann, Goodman, Posserud and Gillberg2007).
All participants, those with ADHD and the controls, were asked to complete a set of questionnaires. The project was approved by the Regional Committee for Medical and Health Research Ethics of Western Norway (IRB 00001872) and the Norwegian Social Science Data Services (NSD). For other details concerning recruitment and sample of the parent study, see Halleland, Lundervold, Halmoy, Haavik, and Johansson (Reference Halleland, Lundervold, Halmoy, Haavik and Johansson2009), Johansson et al. (Reference Johansson, Halleland, Halmoy, Jacobsen, Landaas, Dramsdahl and Haavik2008), Halmoy et al. (Reference Halmoy, Halleland, Dramsdahl, Bergsholm, Fasmer and Haavik2010), and Lundervold et al. (Reference Lundervold, Adolfsdottir, Halleland, Halmoy, Plessen and Haavik2011).
When the present study was conducted, the parent study included 572 adults with ADHD and a control group of 895 ethnically matched adults. In the control group, 700 individuals were recruited through MBRN and the rest through advertisements. In the ADHD group, 352 were recruited from the expert teams and the rest by clinicians. According to the Adult ADHD Self-Report Scale (ASRS) screening questionnaire, 18.1% of the ADHD patients were classified as belonging to the inattentive subtype, while 4.8% were classified as inattentive in the control group. Cutoff values were defined as 21 points on this subscale according to the description by Kessler et al. (Reference Kessler, Adler, Ames, Demler, Faraone, Hiripi and Walters2005).
Participants in the Present Study
Participants from the parent study (64 with an ADHD diagnosis and 61 controls) were randomly selected from those living in the city and the geographical area close to Bergen. The participants were invited to take part in a neuropsychological examination, which was performed by a trained test technician. Sixteen percent of the individuals with ADHD and 19% of the controls who were invited declined to participate. The control group included 44 individuals who were originally recruited from the MBRN and 17 by advertisement.
Subjects with IQ below 80 (N = 2, both from ADHD group) and participants with autism spectrum disorder, tics, Tourette's syndrome or epilepsy (N = 3, two from ADHD group) were excluded.
In the ADHD group, 67.3% used central stimulants or other medication for ADHD. They were instructed not to take any of those medications during the day of testing.
Questionnaires/Tests
Self-report questionnaires
Wender Utah Rating Scale (WURS: Ward, Wender, & Reimherr, Reference Ward, Wender and Reimherr1993) was used to measure ADHD symptoms in childhood and the ASRS (Kessler et al., Reference Kessler, Adler, Ames, Demler, Faraone, Hiripi and Walters2005) was used to assess current ADHD symptoms.
Screening questions
The participants answered 31 questions concerning socio-demographic and clinical factors. Information about life-time comorbidities was collected by asking the participants about other disorders, including bipolar disorder, depression/anxiety, reading/writing difficulties, etc. The validity of these self-reported diagnoses was recently reported (Halmoy et al., Reference Halmoy, Halleland, Dramsdahl, Bergsholm, Fasmer and Haavik2010).
Psychiatric interview
The semi-structured diagnostic interview MINI Plus version 5.0.0 (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller and Dunbar1998) was used by psychiatrists to assess current substance abuse/dependence and current depressive/manic episode.
Color Word Interference Test (CWIT) from D-KEFS
Primary summary scores
The test includes four conditions: (1) Color Naming, (2) Word Reading, (3) Inhibition, and (4) Inhibition/Switching. In the first condition, the task is to name color patches as fast as possible. In the second condition, the task is to read color words as fast as possible. The two first conditions measure basic lower-level cognitive skills of color naming and reading. The third and fourth conditions measure higher level cognitive functioning (i.e., aspects of EF). Inhibition is measured by the third condition, where the task is to inhibit reading words denoting colors while naming the incongruent color of the word. In the fourth condition, the test person is asked to alternate between inhibiting an automatic response of reading (as in the third condition) and reading the color word if the word is framed. This condition, therefore, requires both inhibition and set-shifting abilities. In the present study, we will refer to these primary subtests as condition 1 through 4.
Primary contrast scores
To obtain more pure measures of inhibition and inhibition/set-shifting, the D-KEFS manual offers contrast scores that control for more basic conditions. The scaled scores for basic conditions are subtracted from the scaled scores on measures of higher cognitive functions. Then, this difference score is transformed to a scaled score corrected for age according to US norms. The contrast scores include the following: (1) condition 3 minus condition 1, (2) condition 4 minus the sum of condition 1 and 2, and (3) condition 4 minus condition 3; the contrast scores were included in the present study, referred to as P-inhibition, P-inhibition/set-shifting and P-set-shifting, respectively.
Standardized residual (SR) scores for inhibition and set-shifting
Contrast measures were also calculated as residuals from raw scores (Kramer et al., Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007; Pa et al., Reference Pa, Possin, Wilson, Quitania, Kramer, Boxer and Johnson2010). The “higher order” skills were regressed on the “lower order” ones by using a linear regression analysis for each variable (cfr. the paragraph on statistical analyses for more details) to generate standard residuals of inhibition (SR-inhibition), inhibition/set-shifting (SR-inhibition/set-shifting) and set-shifting (SR-set-shifting).
Wechsler Abbreviated Scale of Intelligence
Two subtests (Matrix Reasoning and Vocabulary) from the Wechsler Abbreviated Scale of Intelligence (WASI) were used as an estimate of IQ according to the norms presented in the test manual (Wechsler, Reference Wechsler1999).
Paced Auditory Serial Addition Task
Paced Auditory Serial Addition Task (PASAT: Gronwall, Reference Gronwall1977) was used to measure working memory. Participants were presented (auditory) a number between 1 and 9 every third second, and should always add the two last numbers. Working memory was defined as the total number of correct responses. PASAT requires active manipulation of stimuli in which an action is required on the presented stimuli. Gallagher and Blader (Reference Gallagher and Blader2001) point out that problems with working memory under stressful conditions, as the one measured by the PASAT, are common in individuals with ADHD.
Statistical Analyses
SPSS, version 15.0, was used for statistical analyses. Analyses were run both with and without outliers. For the D-KEFS primary conditions and primary contrast scores, outliers were defined as more than three interquartile ranges from the rest of the scores. Four univariate outliers and two multivariate outliers were found. No outliers were found for the primary contrast scores. In the regression analyses, standard residuals more than 3.3 or less than −3.3 were defined as outliers and removed. No outliers were found for the SR-inhibition measure, while three outliers were found for SR-inhibition/set-shifting and SR-set-shifting (all from the ADHD group). New regression analyses were run for the conditions where outliers were found, and one new outlier (from the ADHD group) was identified. This was also removed, and new analyses were run. All subsequent analyses were run both with and without outliers. The results were similar with regard to significance, and, therefore, only results from analyses without correction for outliers will be presented. As it is difficult to determine if working memory and total IQ also remove some of the variance caused by the disorder itself, the analyses were run both with and without the two variables as covariates.
Socio-demographic and clinical data
One-way analysis of variance (ANOVA) and χ 2 analyses were used to examine group differences for socio-demographical and clinical data.
Primary summary conditions
The assumption of Equality of Covariance (Box's Test of Equality of Covariance Matrices) was not met and the Equality of Error Variances (Levene’s Test of Equality of Error Variances) was not met for CWIT condition 1, 2, and 4. An alpha level of p < .01 was, therefore, set when interpreting these results. Factorial MANCOVAs with age as a covariate were used to examine group differences regarding the raw scores of the four primary conditions. Significant multivariate results were followed by Bonferroni-corrected univariate tests (ANCOVAs). The alpha level was defined by dividing 0.05 (or 0.01 if violation of assumptions) by the number of comparisons within each set of analyses to avoid type 1 error. Age was used as a covariate in all analyses. For significant results, working memory and IQ were added as covariates.
Primary contrast scores
The primary contrast scores between the two groups were investigated with separate ANOVAs. ANCOVAs were performed to control for IQ and working memory on significant results. The assumption of homogeneity of variances was not met for the measures of P-inhibition and P-inhibition/set-shifting. We, therefore, used an alpha level of p < .01 when interpreting those results. Age was not controlled for since age corrected standard scores were used in the analyses.
Standardized residual scores
Hierarchical regression analyses included each of the more complex measures of executive functions as the dependent variable and the lower level variables as predictors. From these analyses, lower level variables that contributed uniquely to the dependent variable were included in standard regression analyses. The resulting standard residuals were then used in further analyses to represent the SR-inhibition, SR-inhibition/set-shifting and SR-set-shifting measures. Multicollinearity, linearity, and homoscedasticity were evaluated, and Kolomogorov-Smirnoff test of normality and casewise diagnosis were used to evaluate normality of standard residuals (SRs).
SR-inhibition
Since reading problems are associated with ADHD, we included both condition 1 and 2 as independent variables in the hierarchical regression analyses. Performance on these conditions explained 36.4% of the variance in condition 3. When exploring the unique contribution of each variable, the contribution of condition 2 was non-significant and only condition 1, which explained 36.3% of the variance in condition 3, was used as a predictor in the final analyses to generate SRs.
SR-inhibition/set-shifting
First we included the two primary summary conditions (condition 1 and 2) as predictors in two consecutive steps. Together, they explained 14.2% of the variance in condition 4. When exploring the unique contribution of each variable, only condition 1 contributed significantly by explaining 13.6% of the variance and was used as a predictor in the final analyses to generate SRs.
SR-set-shifting
To obtain a measure of set-shifting, we added condition 3 as a predictor of condition 4. Together, the three first conditions explained 35% of the variance. Only condition 3 contributed significantly with unique variance and explained 34.6% of the variance of condition 4. The resulting residuals were included in the final analyses to generate SRs. See Table 1 and 2 for more details about the regression analyses.
Table 1 Hierarchical Regression (N = 120) Including Variables Predicting Inhibition and Set-shifting
Table 2 Standard Regression for Variables Predicting Inhibition and Set-shifting (N = 120)
To examine differences between the groups, the standard residual from each regression analysis was included in separate ANCOVAs. Age was used as a covariate in all analyses. For significant results, follow-up ANCOVAs included IQ and working memory as covariates.
Significance level was set to 0.05 for all analyses, except for measures where assumptions of Equality of Covariance or Equality of Error Variance were violated (then the alpha level was reduced to 0.01).
Socio-demographic and Clinical Data
Statistically significant differences were observed between the ADHD group and control group in age, mean IQ and years of education, but not in sex distribution. A significantly higher proportion of individuals in the ADHD group than in the control group were without work, had difficulties related to reading and writing, alcohol and drugs, depression/anxiety, or had parents or siblings with ADHD. Only 11.9% of the ADHD group had received the diagnosis in childhood. Separate group-analyses were run between the two control samples, revealing no significant differences in IQ (p = .097), ASRS (p = .567), and WURS (p = .767) scores. See Table 3 for socio-demographic and clinical information and Table 4 for data on comorbidity. Different sample sizes in different conditions are due to missing data.
Table 3 Socio-demographical and Clinical Data
aN varies due to missing data and reflects number of participants investigated for each variable
bcut off were defined as 21 points or more on one or both subscales as described in Kessler et al. (Reference Kessler, Adler, Ames, Demler, Faraone, Hiripi and Walters2005).
call participants were asked not to take medication for ADHD the day of testing.
Table 4 Comorbidity
aN varies due to missing data and reflects number of participants investigated for each variable.
To examine whether the subsamples studied here (n = 60 + 60) were representative for the total sample studied in the parent project (n = 572 + 895), we compared the total WURS and the ASRS symptom scores for these groups. There were no statistically significant differences between these scores for either group (WURS ADHD, p = .289; ASRS ADHD, p = .067; WURS control, p = .836; ASRS control, p = .983). The samples on and without ADHD medication were compared on primary and contrast measures (including SR’s), revealing no statistically significant group differences (condition 1, p = .514; condition 2, p = .151; condition 3, p = .489; condition 4, p = .189; P-inhibition, p = .145; P-inhibition/set-shifting, p = .196; P-set-shifting, p = .447; SR-inhibition, p = .182; SR-inhibition/set-shifting, p = .108; SR-set-shifting, p = .267).
Results
Primary Summary Conditions and Contrast/Standardized Residual Scores
A factorial MANCOVA showed that the ADHD group obtained significantly lower scores than the control group on the four primary summary conditions (F(4,114) = 5.55; p < .001; Wilks’ Lambda = 0.84; η p2 = 0.16). A Bonferroni adjusted post hoc test with an alpha level of 0.003 showed statistically significant group differences on condition 2 (F(1,117) = 9.85; p = 0.002; η p2 = 0.078) and condition 4, (F(1,117) = 14.94; p < .001; η p2 = 0.113). When running a separate univariate ANCOVA, including age, total IQ, and working memory as covariates, the result for condition 4 was still statistically significant (F(1,112) = 13.27; p < .001; η p2 = 0.106).
The results for the primary contrast measures were analyzed separately with ANOVAs, Bonferroni corrected for multiple comparisons (alpha level 0.017). The ADHD group obtained significantly lower scores than the control group on the P-set-shifting score (F(1,118) = 8.57, p = 0.004, η p2 = 0.068). This was also true after controlling for working memory and total IQ (F(1,113) = 9.394; p = .003; η p2 = 0.077).
ANCOVA with age as a covariate showed a statistically significant difference (Bonferroni corrected alpha level 0.003) between the groups on the SR-inhibition/set-shifting score (F(1,117) = 11.449; p = .001; η p2 = 0.089), with lower scores in the ADHD group. This result was still significant when including total IQ and working memory as covariates (F(1,112) = 11.937; p = .001; η p2 = 0.096).
ANCOVA with age as a covariate showed a statistically significant difference (Bonferroni corrected alpha level 0.003) between the groups on the SR-set-shifting score (F(1,117) = 11.524; p = .001; η p2 = 0.090). The result was still significant when IQ and working memory were included as covariates (F(1,112) = 12.359; p = 0.001; η p2 = 0.099). See Table 5 for more details about primary summary measures, primary contrast measures and residual analyses.
Table 5 Performance on CWIT in the ADHD and Control group
Note. Age was used as a covariate in all analyses, except for the primary contrast measures.
aηp 2 = partial eta squared.
Additional Analyses
We performed additional analyses where we included sex as an independent variable in the MANCOVA and all ANCOVA analyses. There were no statistically significant effects of sex or sex-by-group interactions for any of the measures. Analyses with control for reading and writing problems (measured by a screening question that asked if the person had ever experienced problems with writing or reading) were still statistically significant. We also removed all participants in the control group who screened positive for ADHD and reran the analyses. Cutoff was defined as 21 points or more on the inattention or hyperactivity scale, or both, as described by Kessler et al. (Reference Kessler, Adler, Ames, Demler, Faraone, Hiripi and Walters2005). These additional analyses did not change the results.
Discussion
Analyses of the CWIT primary summary conditions revealed a multivariate statistically significant difference between the ADHD and the control group, but no significant differences between the groups on the inhibition condition (condition 3) were found when each condition was examined separately. However, there were significant differences between the groups on the fourth condition and both the primary and SR contrast measures of set-shifting, even after controlling for working memory and IQ. Although the effect sizes for the whole range of set-shifting measures were low, the fact that the lowered scores of set-shifting were found regardless of how it was analyzed, strengthened the conclusion of a set-shifting difficulty in the ADHD group. ANCOVAs based on the US-normed standard scores from D-KEFS and the SR measures based on raw scores showed similar results, suggesting that the US-norms are applicable in a Norwegian sample of adults with ADHD.
The lack of significant group differences on measures of inhibition are in disagreement with results from a meta-analysis performed by Lansbergen, Kenemans, and van Engeland (2007) and the results from the UMASS study (Barkley et al., Reference Barkley, Murphy and Fischer2008), in which participants with ADHD showed impaired performance on the Stroop inhibition condition, even after controlling for basic conditions. However, in a meta-analysis by Schwartz and Verhaeghen (Reference Schwartz and Verhaeghen2008), the Stroop inhibition effect was not larger for individuals with ADHD than for normal controls.
The inconsistent results may partly be due to different methods used for calculating the inhibition effect. Several meta-analyses have used either the “Golden's method” or the “difference score” (Hervey, Epstein, & Curry, Reference Hervey, Epstein and Curry2004; Homack & Riccio, Reference Homack and Riccio2004; van Mourik, Oosterlaan, & Sergeant, Reference van Mourik, Oosterlaan and Sergeant2005). The difference score is calculated as the difference between the color naming and color word inhibition scores and was also used in the UMASS study. “Golden's Method” (Golden, Reference Golden1978) is calculated by predicting the score on the color word inhibition condition from the scores on the basic conditions and then subtracting this score from the actual score on the color word inhibition condition. However, both methods have been criticized by Lansbergen et al. (Reference Lansbergen, Kenemans and van Engeland2007) and Delis et al. (Reference Delis, Kaplan and Kramer2001) for leading to incorrect estimates of the inhibition effect. Lansbergen et al. (Reference Lansbergen, Kenemans and van Engeland2007) and Schwartz and Vaerhagen (2008), therefore, used a ratio score to calculate the inhibition measure, and Delis et al. (Reference Delis, Kaplan and Kramer2001) constructed the contrast scores to standardize the difference between the variables in the D-KEFS. Although both used a ratio score to calculate the inhibition effect, the meta-analyses performed by Lansbergen et al. and Schwartz and Vaerhagen yielded different results. This may be explained by methodological differences: while Lansbergen et al. also included computerized Stroop tests, Schwartz and Vaerhagen only included the original Golden Stroop test. This example reveals how different tests of inhibition may measure different aspects of inhibition as well as related functions.
The fourth condition from the CWIT is a complex task, and this may at least partly explain why the ADHD group obtained significantly lower scores than the control group. This complexity probably reflects interplay between set-shifting and inhibition, and the test results thus represent more than just an additive effect. However, Lippa and Davis (Reference Lippa and Davis2010) found that many patients actually showed better performance on the fourth condition than on the third condition, as was also reported in a study of individuals with schizophrenia (Savla et al., Reference Savla, Twamley, Thompson, Delis, Jeste and Palmer2011). This shows that the fourth condition is not necessarily more difficult, and an alternative explanation may, therefore, be that the training effect from the former condition is reduced in the ADHD group. It is also possible that adults with ADHD have more problems with the type of shifting measured by the CWIT than other clinical groups. Kramer et al. (Reference Kramer, Quitania, Dean, Neuhaus, Rosen, Halabi and Miller2007) suggest that the inconsistency in the literature concerning set-shifting may be attributed to the diversity in defining and measuring the concept. While some tests measure the subjects’ ability to alter their sorting strategies when reinforcement contingencies change, other tasks require serial alternation between two different types of stimuli. They also suggest that it is likely that the different types of set-shifting tasks have different underlying neuroanatomical correlates. More research is needed to determine if it is a deficit in the specific executive function after considering its non-executive and other executive components, or the general complexity of the task that cause the group differences.
Limitations
Before generalizing the results from this study to other adult ADHD samples, some limitations should be taken into account.
The parent study has a naturalistic design and the participants were diagnosed by different clinicians. This may have yielded a more heterogeneous ADHD sample than in studies with more stringent inclusion criteria based on diagnostic information gathered by one clinician. Furthermore, the high non-response rate in the parent study may have led to a selection bias toward the more motivated participants, and the high IQ in both the ADHD group and the control group shows that there may have been a bias toward recruitment of the more well-functioning individuals. Caution is, therefore, necessary in generalizing the results of this study to the general population or other samples of adults with ADHD.
Only 11.9% of our adult ADHD patients received a formal diagnosis of ADHD during childhood. As the number of children diagnosed with ADHD in Norway has recently increased, the low percentage diagnosed in childhood in previous years probably reflects a low awareness of ADHD at the time when the participants were children.
There is a high prevalence of co-existing psychiatric conditions in ADHD samples (Sobanski et al., Reference Sobanski, Bruggemann, Alm, Kern, Deschner, Schubert and Rietschel2007). Other comorbid conditions may have influenced our results and the inclusion of clinical control groups, with for example anxiety or depression, might have provided more insight into the specificity of the neuropsychological deficits observed in the present study. We believe that the clinical ADHD group in the present study is representative for the clinical population of ADHD patients. This suggests that some of the participants were on medication for comorbid conditions at the time of participation and that some also met criteria for current substance abuse. Excluding those participants may have yielded a less representative group.
Kopp (Reference Kopp2011) has argued that contrast measures are associated with a low reliability and may simply be a function of the average reliability of its two components and of the correlation between them (Crawford, Sutherland, & Garthwaite, Reference Crawford, Sutherland and Garthwaite2008). For example, the D-KEFS defined inhibition/set-shifting minus inhibition contrast score had a test–retest reliability of −0.041 in the group aged 20–49 years, and a score of −0.107 in the age range 50–89. In this study, similar results obtained by using SR measures based on raw scores and the D-KEFS contrast scores indicate that the results add to the validity evidence base for the contrast scores. However, more studies of the reliability and validity of the D-KEFS contrast scores are called for.
Conclusions and Implications for Further Research
The present study showed difficulties related to set-shifting tasks in adults with ADHD. This result was strengthened by the fact that these problems were consistently observed regardless of how the data were analyzed. This kind of robustness is important when evaluating individuals with ADHD because impairment of only a single primary measure may disguise problems in other aspects of cognitive function.
Rommelse and Buitelaar (Reference Rommelse and Buitelaar2008) argue that it is useful to make a distinction between primary and secondary EF difficulties for scientific and therapeutic reasons. They suggested that a clarification of the hierarchical origin of executive difficulties in patients with ADHD could help us better understand the nature of the pathology of the disorder and that intervention targets may differ when difficulties stem from the bottom (basic encoding and/or motor processes) instead of the top of the hierarchy (executive processes). The significant differences after controlling for covariates could indicate that adults with ADHD have specific problems with set-shifting, but more studies are needed to validate our findings. If such specific difficulties are found, this may help to explain some of the common symptoms reported by patients with ADHD in the clinic, such as the inability to shift efficiently from one mental activity to another.
The main contributions of the present study are the use of two different contrast measures to control for more basic functions and the inclusion of IQ and working memory function as covariates. Further studies should combine such measures of neuropsychological function with more detailed clinical information as well as biomarkers generated from brain imaging and genetics. These types of studies would contribute important knowledge about neurocognitive dysfunctions in ADHD and other neuropsychiatric disorders.
Acknowledgments
The project has received financial support from The Research Council of Norway, The Western Norway Regional Health Authority and the K.G. Jebsen Foundation. The study was also supported by “The National Program for Integrated Clinical Specialist and PhD-training for Psychologists” in Norway by financial support to H.H. This program is a joint cooperation between the Universities of Bergen, Oslo, Tromsø, the Regional Heath Authorities, and the Norwegian Psychological Association. The program is funded jointly by The Ministry of Education and Research and The Ministry of Health and Care Services. None of the authors have any conflicts of interest. We thank the participants, Johanne Telnes Instanes, Anne Halmøy, Margaretha Dramsdahl, Liv Heldal, Ragnhild Nordenborg, Lisa Vårdal, Vivica Næss, and the other members of the research group. We also thank Steinunn Adolfsdottir for the discussions concerning this study.
