Introduction
Inaccurate appraisal of one's condition and its consequences, which may be referred to in terms of reduced awareness, insight, metacognition, anosognosia or denial, is frequently reported in conditions involving cognitive impairment, for example, in dementia, or following brain injury or stroke, where it may interfere with treatment, add to carer burden and lead to problem escalation (Aalten, van Valen, Clare, Kenny, & Verhey, Reference Aalten, van Valen, Clare, Kenny and Verhey2005; Nelis et al., Reference Nelis, Clare, Martyr, Markova, Roth, Woods and Morris2011). Inaccurate self-appraisal may be observed in healthy people, related to psychosocial factors (Clare, Nelis, et al., 2011), but is most commonly seen as a consequence of brain lesions (in particular where the prefrontal cortex is involved) and is often associated with impairment in executive functions (EF) (Bramham, Morris, Hornak, Bullock, & Polkey, Reference Bramham, Morris, Hornak, Bullock and Polkey2009; Stuss, Picton, & Alexander, Reference Stuss, Picton and Alexander2001; Wheeler, Stuss, & Tulving, Reference Wheeler, Stuss and Tulving1997).
Frontal lobe functions involved in performance of executive tasks are frequently compromised in Parkinson's disease (PD) as dopamine depletion in the striatum causes a disruption of frontostriatal networks. This affects the motor loop (connecting the putamen and the supplementary motor area), as well as the cognitive loop (connecting the dorsolateral prefrontal cortex and the dorsal caudate nucleus, associated with executive deficits) (Cools, Stefanova, Barker, Robbins, & Owen, Reference Cools, Stefanova, Barker, Robbins and Owen2002; Leh, Petrides, & Strafella, Reference Leh, Petrides and Strafella2010). A significant proportion of people diagnosed with PD (PwPD) experience cognitive decline, particularly in EF, that may impact negatively upon quality of life (Klepac, Trkulja, Relja, & Babic, Reference Klepac, Trkulja, Relja and Babic2008; Schrag, Jahanshahi, & Quinn, Reference Schrag, Jahanshahi and Quinn2000) and activities of daily living (Cahn et al., Reference Cahn, Sullivan, Shear, Pfefferbaum, Heit and Silverberg1998). Little is known about the extent of awareness of cognitive problems shown by PwPD. Lack of awareness might mean that impairments are unrecognized by PwPD and not reported to the clinician, with possible implications for treatment outcomes (Koerts et al., Reference Koerts, van Beilen, Leenders, Brouwer, Tucha and Tucha2012).
Executive deficits and poor awareness of one's own limitations may impact on various aspects of everyday life in PD, including driving (Devos et al., Reference Devos, Vandenberghe, Nieuwboer, Tant, Baten and De Weerdt2007; Rizzo, Uc, Dawson, Anderson, & Rodnitzky, Reference Rizzo, Uc, Dawson, Anderson and Rodnitzky2010; Stolwyk, Charlton, Triggs, Iansek, & Bradshaw, Reference Stolwyk, Charlton, Triggs, Iansek and Bradshaw2006; Uc et al., Reference Uc, Rizzo, Anderson, Sparks, Rodnitzky and Dawson2007) and adherence to medication regimes (Grosset, Bone, & Grosset, Reference Grosset, Bone and Grosset2005; Grosset, Bone, Reid, & Grosset, Reference Grosset, Bone, Reid and Grosset2006; Kulkarni et al., Reference Kulkarni, Balkrishnan, Anderson, Edin, Kirsch and Stacy2008; Leopold, Polansky, & Hurka, Reference Leopold, Polansky and Hurka2004). Where the view of cognitive functioning held by the PwPD is discrepant from that held by the carer, there may be particular stresses in the caregiving relationship. Understanding how cognitive problems are perceived by both the PwPD and the carer is, therefore, crucial for providing appropriate support.
The Levels of Awareness Framework proposed by Clare, Marková, Roth, and Morris (Reference Clare, Marková, Roth and Morris2011) describes awareness in terms of dynamic perceptual and appraisal processes operating at various levels: sensory registration, performance monitoring, evaluative judgment and meta-representation. Sensory registration relates to core consciousness and attentional processes; performance monitoring reflects an immediate judgment about performance on a specific task as it is completed; evaluative judgment refers to more general judgments about functioning in a particular area; and meta-representation is a complex reflection on the situation, which integrates individual knowledge, emotions and attitudes. The phenomena of awareness elicited at each level are different. They may be influenced by several internal (e.g., mood, personality) and external (e.g., social norms, carer responses) factors, and are additionally shaped by the object in relation to which awareness is assessed (e.g., cognitive deficits or the experience of illness and its implications); hence, they are not directly comparable. To understand the implications of someone's level of awareness for everyday functioning, it is useful to know how accurately the individual perceives his/her overall cognitive functioning and appraises his/her performance in particular tasks.
Studies of awareness in PD to date have focused exclusively on the evaluative judgment level, with informant ratings being used as a benchmark against which self-ratings are compared, and less frequent comparisons to objective measures. Only a few studies have directly explored awareness of cognitive problems in PwPD; Seltzer, Vasterling, Mathias, and Brennan (Reference Seltzer, Vasterling, Mathias and Brennan2001) and Sitek, Soltan, Wieczorek, Robowski, and Slawek (Reference Sitek, Soltan, Wieczorek, Robowski and Slawek2011) reported good agreement between self- and informant ratings of general cognition and of memory, respectively. Ivory, Knight, Longmore, and Caradoc-Davies (Reference Ivory, Knight, Longmore and Caradoc-Davies1999) analyzed the accuracy of evaluative judgments about memory and attention by correlating performance on three cognitive tests with responses on a metamemory questionnaire. Only 1 of 11 correlations was significant, suggesting limited accuracy of evaluative judgment in PwPD. While the above studies have investigated awareness of cognitive functions in PD, they did not focus specifically on EF. It would be particularly relevant to focus on awareness in relation to EF, as this is the cognitive domain most commonly impaired in PwPD (Kudlicka, Clare, & Hindle, Reference Kudlicka, Clare and Hindle2011; Muslimovic, Schmand, Speelman, & De Haan, Reference Muslimovic, Schmand, Speelman and De Haan2007).
Three studies which did not refer directly to awareness have explored evaluative judgment of EF in PwPD by comparing self- and informant questionnaire-based ratings. In McKinlay et al. (2008), PwPD reported more difficulties than their carers, while Koerts et al. (Reference Koerts, van Beilen, Leenders, Brouwer, Tucha and Tucha2012) and Mathias (Reference Mathias2003) revealed good agreement between ratings. The inconsistency might be related to differences in cognitive status in the study samples. Poor awareness is commonly described in relation to executive deficits (Stuss et al., Reference Stuss, Picton and Alexander2001), and studying awareness in PwPD without distinguishing those with actual executive deficits may produce mixed findings. The observed inaccuracies might be clarified by establishing how the awareness level in PwPD compares to awareness in a similar but healthy population. There are two studies on executive and neurobehavioral functioning that examined self- and informant ratings in PwPD and controls (Koerts et al., Reference Koerts, van Beilen, Leenders, Brouwer, Tucha and Tucha2012; Mathias, Reference Mathias2003), but without direct comparison of the actual level of agreement between participants and informants.
In summary, studies on awareness in PD have only considered the level of evaluative judgment, have rarely investigated awareness in relation to well-specified EF impairment, and have not compared awareness levels in PD and healthy controls. The present study aimed to address these issues by distinguishing PwPD with and without EF impairments, and by comparing their performance to healthy controls. Awareness phenomena were examined in relation to the two levels of evaluative judgment and performance monitoring. The following research questions were addressed: (1) How accurate are PwPD with and without EF impairments in assessing their overall executive functioning and performance in a given task, in comparison to controls? (2) What are the correlates of decreased awareness in PwPD?
Method
Design
The study used a cross-sectional design comparing PwPD with and without EF deficits, and healthy controls. Awareness of EF was assessed at the two levels of evaluative judgment and performance monitoring. In relation to evaluative judgment, awareness was assessed as follows: (a) discrepancy between self- and informant ratings on a questionnaire evaluating executive functioning (BRIEF-A; Roth, Isquith, & Gioia, Reference Roth, Isquith and Gioia2005); (b) relationship between BRIEF-A ratings and EF test performance. In relation to performance monitoring, awareness was assessed through comparison of test performance on two EF tests [Trail Making Test and Color Word Interference (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001)] with self-ratings of that performance, made immediately after the tasks had been completed. Ethical approval was granted by the relevant University and National Health Service ethics committees and written informed consent was obtained from all participants.
Participants
People with Parkinson's disease, recruited from local Movement Disorders clinics, were diagnosed according to the UKPDS Brain Bank criteria (Daniel & Lees, Reference Daniel and Lees1993), and were in the mild to moderate stages of the disease (Hoehn & Yahr, Reference Hoehn and Yahr1967). They had no dementia, as indicated by an Addenbrooke's Cognitive Examination – Revised (ACE-R) score ≥ 82 (Mioshi, Dawson, Mitchell, Arnold, & Hodges, Reference Mioshi, Dawson, Mitchell, Arnold and Hodges2006) and an MMSE score ≥ 24 (Folstein, Folstein, & McHugh, Reference Folstein, Folstein and McHugh1975), and no significant depression, as indicated by a Hospital Anxiety and Depression Scale depression score ≤ 11 (Snaith & Zigmond, Reference Snaith and Zigmond1994). Controls were recruited from various community sources (e.g., over-50 s clubs, University of the Third Age branches, church groups), had no dementia, as indicated by an ACE-R score ≥ 82 (Mioshi et al., Reference Mioshi, Dawson, Mitchell, Arnold and Hodges2006) and an MMSE score ≥ 24 (Folstein et al., Reference Folstein, Folstein and McHugh1975), and no significant depression, as indicated by a Geriatric Depression Scale (GDS-15) score ≤ 5 (Burke, Roccaforte, & Wengel, Reference Burke, Roccaforte and Wengel1991; Sheikh & Yesavage, Reference Sheikh and Yesavage1986). Informant ratings were provided by people who knew the participants very well (e.g., spouses, adult children or close friends). All participants were fluent in English and had adequate eyesight and hearing.
Measures
Cognitive screening
The Addenbrooke's Cognitive Examination Revised (ACE-R; Mioshi et al., Reference Mioshi, Dawson, Mitchell, Arnold and Hodges2006) assesses five cognitive domains: attention and orientation, memory, verbal fluency, language, and visuospatial abilities. The maximum total score of 100 indicates error-free performance. The ACE-R also provides an MMSE score (Folstein et al., Reference Folstein, Folstein and McHugh1975).
Executive functions
The Trail Making Test (TMT) of the Delis-Kaplan Executive Function System (D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001) consists of five visual-motor tasks assessing basic visuospatial and motor skills (TMT 1, 2, 3, and 5), and flexibility in thinking (TMT 4), which is regarded as one of the core EF abilities (Royall et al., Reference Royall, Lauterbach, Cummings, Reeve, Rummans, Kaufer and Coffey2002). In TMT 1, participants cross out all instances of the number 3 on a sheet of paper. In the following three conditions, participants draw a line connecting numbers (TMT 2), letters (TMT 3) or numbers and letters in alternating sequence (TMT 4) in ascending order. In TMT 5, participants draw a line connecting circles in the indicated order. Greater time to complete each task indicates poorer performance. A ratio score TMT 4/TMT 2 is suggested as the most accurate index of EF that differentiates EF from visuospatial and motor abilities (equivalent to TMT B/TMT A in a widely used version of the TMT) (Arbuthnott & Frank, Reference Arbuthnott and Frank2000; Delis et al., Reference Delis, Kaplan and Kramer2001). Raw scores for each condition (in seconds) are converted to age-scaled scores (M = 10; SD = 3). For the purposes of this study, the age-scaled scores were classified into five bands: impaired (≤5), below average (6–8), average range (9–11), above average (12–14) and superior (≥15). The five bands formed a 5-point scale (1 to 5), with lower scores indicating worse test performance. The scale was used for comparing test performance with self-appraisal of that performance (as reported in Martyr et al., Reference Martyr, Clare, Nelis, Marková, Roth, Woods and Morris2012). See details in the Planned Analyses section below.
The Color-Word Interference test (CWI) of the D-KEFS (Delis et al., Reference Delis, Kaplan and Kramer2001) assesses inhibition and flexibility in thinking. There are two baseline conditions (naming the ink color of color patches – CWI 1, and reading a list of color names in black ink – CWI 2), and two higher-level conditions, naming the dissonant ink color instead of reading the color name (CWI 3, inhibition; the traditional Stroop task), and switching between naming the dissonant ink color and reading the word (CWI 4, inhibition and flexibility in thinking). As in the TMT, raw scores (time to complete the task) are converted to age-scaled scores. For the purposes of this study, scores were then classified into five bands. PwPD completed both the TMT and the CWI, and controls completed the TMT only.
EF-related behavioral problems
The Behavior Rating Inventory of Executive Function – Adult Version (BRIEF-A; Roth et al., Reference Roth, Isquith and Gioia2005) provides information about executive functioning (self-regulation skills) in the everyday environment, as rated by participants and informants in two parallel questionnaires (self and informant versions). Using a 3-point scale (never, sometimes, often; scored 1, 2, 3, respectively) participants indicate which of the 75 behaviors described have been a problem during the past month. Higher Global Executive Composite scores (GEC; range, 70–210) indicate more reported problems in regulating behavior and emotional responses, distributing and sustaining attention, and solving problems. The GEC scores are converted to age-scaled T scores (M = 50; SD = 10), which indicate whether the reported level of difficulty suggests clinically significant problems in EF (T ≥ 65, 1.5 SD above the mean).
Premorbid IQ
The National Adult Reading Test (NART; Nelson & Willison, Reference Nelson and Willison1991) estimates lifelong intellectual ability. Participants read aloud 50 phonetically irregular words. The number of words pronounced incorrectly is converted into an estimated IQ score, with more errors producing a lower estimated IQ score.
Mood
PwPD completed the Hospital Anxiety and Depression Scale (HADS; Snaith & Zigmond, Reference Snaith and Zigmond1994), a self-rating questionnaire assessing levels of depression and anxiety, validated for use in PD (Schrag et al., Reference Schrag, Barone, Brown, Leentjens, McDonald, Starkstein and Goetz2007). The 14 questions form two subscales: HADS-Anxiety and HADS-Depression. Higher scores indicate higher levels of self-rated anxiety/depression. This study adopted the cut-off of 11 that has been suggested for depression screening purposes (Crawford, Henry, Crombie, & Taylor, Reference Crawford, Henry, Crombie and Taylor2001). Healthy controls completed the Geriatric Depression Scale (GDS-15; Sheikh & Yesavage, Reference Sheikh and Yesavage1986), a 15-item scale assessing levels of depression, with higher scores indicating higher levels of self-rated depression. The study adopted the cut-off of ≤5 recommended for depression screening (Burke et al., Reference Burke, Roccaforte and Wengel1991).
Caregiver burden
Informants of the PwPD completed the Caregiver Burden Inventory (CBI; Novak & Guest, Reference Novak and Guest1989), a 24-item questionnaire describing caregivers’ feelings about and responses to the burden of care. The maximum score of 96 indicates the highest levels of caregiver stress.
Procedure and Data Collection
The majority of participants were visited at home; six PwPD and nine controls chose to meet at the University. PwPD were assessed during their “on” phase. The assessment took between 1.5 and 3 hr and was part of a wider study, which included some measures not reported here. Some participants completed the assessment over two shorter visits and some opting to send the self-completion questionnaires by post.
Planned Analysis
Before analysis the normality of distributions and the homogeneity of variance were assessed (using the Shapiro-Wilk test and a Q-Q plots, and the Leven test, respectively), and non-parametric statistics were used where appropriate.
Evaluative Judgment Analysis
Self- versus informant rating of executive functioning
The level of agreement between self- and informant ratings was calculated for the BRIEF-A summary score (GEC) in the form of a Corrected Discrepancy score, which is a rigorous measure correcting for between-subject differences in actual level of scoring (Clare, Whitaker, & Nelis, Reference Clare, Whitaker and Nelis2010). The corrected discrepancy score was calculated by subtracting the self-rating from the informant rating (BRIEF-A Informant raw score – BRIEF-A Self raw score) and dividing the difference by the mean value of the two ratings [(BRIEF-A Informant + BRIEF-A Self)/2]. The possible range of corrected discrepancy scores is −1 to 1, with positive values indicating that self-rating is more positive than informant rating (taken to indicate an overestimation of executive functioning ability), and negative values indicating that self-rating is less positive than informant rating (taken to indicate an underestimation). Discrepancy scores close to 0 indicate close agreement. The discrepancy scores in the three study groups were compared using one-way analysis of variance (ANOVA) and Bonferroni post hoc analysis. BRIEF-A summary scores were compared using the Kruskal-Wallis test, and self- and informant ratings within each group were compared using the Wilcoxon Signed Ranks test.
Evaluative judgment of executive functioning versus EF test performance
Correlational analyses (Spearman's Rho) examined the relationship between performance on executive tests (TMT 4, CWI 3, and CWI 4; raw scores) and the overall judgment of executive functioning (BRIEF-A raw score), separately for PwPD with and without executive deficits.
Performance Monitoring Analysis
Awareness at the performance monitoring level was established by calculating Performance Ratios – the self-evaluation of test performance divided by the test performance band score, for each TMT and CWI condition.
Participants evaluated their performance on TMT and CWI immediately after a task was completed, using a 5-point scale: very poor, poor, alright, good, or very good (scored 1 to 5, respectively). PwPD and controls rated their performance on each TMT condition and PwPD also rated their performance on CWI. Test performance on each condition was classified into one of five bands, based on the age-scaled scores: impaired, below average, average, above average and superior. This formed a five-point scale (1 to 5) with lower scores indicating worse test performance (as reported in Martyr et al., Reference Martyr, Clare, Nelis, Marková, Roth, Woods and Morris2012).
A Performance Ratio score of 1 indicates perfect agreement between test performance and self-appraisal of that performance; values above 1 suggest overestimation of actual performance, and values below 1 indicate underestimation. As the interpretation of the scaled scores of EF tests in terms of the five self-appraisal categories is somewhat arbitrary, the actual values of the ratio need to be interpreted with caution. In contrast, the group comparison of the ratios provides an objective indication of whether PwPD are as accurate as healthy controls in self-appraising their task performance. Performance ratios were logarithmically transformed to ensure a more symmetrical distribution for statistical analysis (Trosset & Kaszniak, Reference Trosset and Kaszniak1996) and averaged to provide summary indices separately for TMT and CWI (Mean Performance Ratios). Performance ratios in the three study groups (PwPD with and without executive impairment, and the control group) were compared using one-way ANOVA and Bonferroni or Games-Howell post hoc analysis.
The relationships between the indicators of awareness (BRIEF-A Discrepancy Scores and Mean Performance Ratios) and other variables of interest were explored using correlational analyses (Spearman's Rho).
Results
Participants
Sixty-five PwPD and 43 controls were included in the study. One person was excluded from the PwPD group due to severe hearing difficulties and one control participant aged 94 was excluded as there are no normative data for the D-KEFS tests and BRIEF-A for people over 90. One-way ANOVA found no significant differences between controls and PwPD with regard to age (t(106) = −1.23; p = .220), years of education (t(106) = −1.91; p = .059), NART-estimated IQ (t(106) = 1.18; p = .240) or general cognition (as indicated by ACE-R, t(106) = 1.17; p = .243). See Table 1 for demographic characteristics.
Table 1 Demographic characteristics of the study groups
PwPD_EF- = PwPD with EF deficits; PwPD_EF+ = PwPD with normal EF; M = mean; SD = standard deviation; ACE-R = The Addenbrooke's Cognitive Examination—Revised; HADS = Hospital Anxiety and Depression Scale; GDS = Geriatric Depression Scale; 1n = 64 in PwPD and n = 41 in PwPD_EF+.
PwPD were allocated to one of two groups on the basis of their performance on EF tests: PwPD with normal performance on all three EF tests (TMT 4, CWI 3, and CWI 4) were allocated to the group with normal EF (PwPD_EF+), and PwPD who had impaired performance on one or more of the above tests (scaled score ≤ 5, 1.5 SD below the mean) were allocated to the group with EF deficits (PwPD_EF-). Two participants who did not complete CWI (due to color blindness) or TMT (due to difficulties with the alphabet) were allocated to PwPD_EF+ on the basis of their normal performance on the other tests.
One-way ANOVA found no significant differences between the control group, PwPD_EF+ and PwPD_EF- in age (F(2,105) = 3.11; p = .049, not significant in post hoc analysis), years of education (F(2,105) = 2.60; p = .079) or NART-estimated IQ (F(2,105) = 0.79; p = .457). There was a significant group difference in general cognition (ACE-R, F(2,105) = 6.19; p = .003; PwPD_EF-, controls < PwPD_EF+, significant at p < .05 in post hoc analysis). The comparison of ACE-R subscales indicated no group effect for Attention and orientation, Verbal fluency and Visuospatial abilities. There was a significant group effect for Memory (F(2,105) = 3.35; p = .039; not significant in post hoc analysis) and for Language (F(2,105) = 7.42; p = .001; PwPD_EF-, controls < PwPD_EF+ significant at p < .001 in post hoc analysis. PwPD_EF+ and PwPD_EF- were similar in terms of disease duration (t(33.99) = 1.07; p = .294) and the Total Daily Levodopa Equivalent Dose (t(62) = 1.11; p = .270), as indicated by an independent-samples t test. See Table 2 for detailed PD characteristics.
Table 2 Disease characteristics and EF test performance in PwPD groups
PwPD_EF- = PwPD with EF deficits; PwPD_EF+ = PwPD with normal EF; M = mean; SD = standard deviation; TMT = Trail Making Test (D-KEFS); CWI = Color Word Interference (D-KEFS); LED = Total Daily Levodopa Equivalent Dose; H&Y = Hoehn and Yahr stage; CBI = Caregiver Burden Inventory; Dopamine agonists = Non–ergot-derived dopamine-receptor agonists.
1Mean value of the time since first symptoms and the diagnosis, as reported by PwPD.
2Based on Tomlinson et al. (Reference Tomlinson, Stowe, Patel, Rick, Gray and Clarke2010).
3TMT4 in the control group: n = 42, M = 10.83, SD = 3.57, Range: 2–15.
4There was no rating available for 6 participants (9.2%).
Evaluative Judgment
BRIEF-A self-rating versus BRIEF-A informant rating
The largest Corrected Discrepancy scores were observed in PwPD_EF+ (PwPD_EF+ > PwPD_EF > Controls), but the differences were not statistically significant. See details of the one-way ANOVA in Table 3a and scaled scores in Figure 1.
Table 3 Comparisons of BRIEF-A self- and informant summary scales
Note. The corrected discrepancy scores were calculated for both scales by subtracting the self-rating from the informant rating and dividing the difference by the mean value of the two ratings (BRIEF-A Informant – BRIEF-A Self)/[(BRIEF-A Informant+BRIEF-A Self)/2] to correct for between-subject differences in actual level of scoring. Possible range of corrected discrepancy scores is −1 to 1.
Fig. 1 Mean age-scaled T scores of BRIEF-A (M = 50; SD = 10) in the three study groups.
Note. Informants in PwPD_EF- ni = 17; Informants in PwPD_EF+ ni = 30; Informants in Control ni = 39; PwPD_EF- – PwPD with EF deficits; PwPD_EF+ – PwPD with normal EF. PwPD = people diagnosed with Parkinson's disease; EF = executive functions.
There was no significant difference in how participants in the three study groups rated their own executive functioning (BRIEF-A Self), or in how informants rated the executive functioning of the participants (BRIEF-A Informant). See details of the Kruskal-Wallis tests in Table 3b.
Note. Prior to the analyses, the three BRIEF-A validity scales (for both self- and informant ratings) were inspected. Three participants in PwPD had a marginally elevated Inconsistency scale and one participant in PwPD had an elevated Infrequency scale. As directed in the BRIEF-A manual, BRIEF-A responses from all four participants were inspected, and as no further evidence for atypical or unreliable answers was found, these participants’ data were included in further analyses.
BRIEF-A_R = BRIEF-A raw score; PwPD_EF- = PwPD with EF deficits; PwPD_EF+ = PwPD with normal EF; M = mean; SD = standard deviation; CD = Corrected Discrepancy score.
1Informants in PwPD_EF- ni = 17.
2Informants in PwPD_EF+ ni = 30.
3Informants in Control ni = 39.
4Wilcoxon Signed Ranks test, based on positive ranks.
Self- and informant BRIEF-A ratings (compared within each study group) were similar in controls and PwPD_EF-, while in PwPD_EF+ participants reported significantly more problematic behaviors than did their informants. See details of the Wilcoxon Signed Ranks test in Table 3b.
Evaluative judgment of cognitive functioning versus objective test performance
The correlational analyses revealed that in PwPD_EF- BRIEF-A self-rating was negatively related to performance on CWI 3, with poorer performance on CWI 3 related to fewer difficulties reported. In PwPD_EF+ both self- and informant BRIEF-A ratings were positively related to performance on CWI 4, with poorer performance associated with more difficulties reported. See details of Spearman's Rho correlational analyses in Table 4.
Table 4 Bivariate correlations between ratings of executive functioning (BRIEF-A) and objective EF test performance
Note. Cases excluded pairwise in the event of missing data. No Bonferroni adjustment has been made in order to minimise the risk of Type II error (Bender & Lange, Reference Bender and Lange2001; Perneger, Reference Perneger1998).
PwPD_EF- = PwPD with EF deficits; PwPD_EF+ = PwPD with normal EF; rs = Spearman's correlation coefficient; BRIEF-A_R = BRIEF-A raw score; CWI 3_R = Color Word Interference (D-KEFS) raw score; TMT ratio = TMT4/TMT2.
* = significant at p < .05.
Performance Monitoring – TMT and CWI Performance Score versus Self-Ratings
The comparison of Mean Performance Ratios for TMT and CWI indicates that PwPD_EF- were significantly less accurate (more positive) in appraising their performance than other study groups. While mean test performance of PwPD_EF- was significantly worse than in other groups, their self-appraisals were comparable to those of other groups. See details of the one-way ANOVAs (TMT) and the t tests (CWI) in Table 5.
Table 5 Descriptive information on variables used for calculating Performance Ratios and comparison of Performance Ratios in the study groups
SS = Mean Scaled Score; TP band = Mean tests performance band score based on scaled score; Self-evaluation = Mean Self-evaluation of test performance score.
1F(2,69) = 514.54, PwPD_EF- < PwPD_EF+, Control.
2F(2,69) = 512.93, PwPD_EF- < PwPD_EF+, Control.
3F(2,71) = 50.25.
4t(32) = 524.39, PwPD_EF- < PwPD_EF+.
5t(32) = 524.52, PwPD_EF- < PwPD_EF+.
6t(31) = 520.44.
Note. The Performance Ratio scores were transformed using natural logarithms to improve distribution prior to comparison analyses.
PwPD_EF- = PwPD with EF deficits; PwPD_EF+ = PwPD with normal EF; M = mean; SD = standard deviation; PR = Performance Ratio; TMT = Trail Making Test (D-KEFS); CWI = Color Word Interference (D-KEFS); TMT MPR = Mean Performance Ratio for TMT subtests; CWI MPR = Mean Performance Ratio for CWI subtests.
1Games-Howell post-hoc correction for unequal variances.
Bivariate Correlations of Awareness Indicators and Other Variables of Interest
Table 6 shows Spearman's correlations between the awareness indicators and other variables in the PwPD group. Higher BRIEF-A Corrected Discrepancy (greater differences between self- and informant ratings) were associated with higher stress reported on the Caregiver Burden Inventory. Higher Mean Performance Ratios (greater discrepancies between self-appraisal and actual test performance) were associated with poorer general cognition (lower ACE-R).
Table 6 Spearmans's rho correlations of awareness indicators and other variables in PwPD
Note. Cases excluded pairwise in the event of missing data. The Performance Ratio scores were transformed using natural logarithms to improve distribution prior to analysis. No Bonferroni adjustment has been made in order to minimise the risk of Type II error (Bender & Lange, Reference Bender and Lange2001; Perneger, Reference Perneger1998).
BRIEF-A CD = Corrected Discrepancy for BRIEF-A BRI; BRIEF-A MI CD = Corrected Discrepancy for BRIEF-A MI; logTMT_MPR = logarithm of Mean Performance Ratio for TMT subtests; logCWI MPR = logarithm of Mean Performance Ratio for CWI subtests; CBI = Caregivers Burden Inventory; HADS-A = HADS Anxiety; HADS-D = HADS Depression; LED = Total Daily Levodopa Equivalent Dose.
**Significant at p < .01
Discussion
The present study investigated awareness of executive functioning in PwPD with and without EF deficits, and in healthy controls. At the evaluative judgment level, PwPD_EF- were found to be as accurate as PwPD_EF+ and healthy older people. At the performance monitoring level, PwPD_EF- were found to significantly overestimate their performance in comparison to PwPD_EF+ and healthy older people, which is a novel finding in PwPD. The overestimation was particularly profound in the more demanding tasks, and might be related to deficits in executive control processes. Larger BRIEF-A discrepancies were related to higher levels of caregiver burden, and higher performance ratios were related to poorer general cognition.
Evaluative Judgment Level
Self-rating versus informant rating
At the evaluative judgment level, awareness of executive functioning was operationalized with a discrepancy score between self- and informant ratings. The BRIEF-A Corrected Discrepancy scores were similar in all study groups, suggesting that PwPD (with and without EF deficits) are as accurate in self-appraisal of their executive functioning as healthy older people. To our knowledge, this is a new finding, as a discrepancy score approach has not been previously used to compare awareness in PwPD and healthy controls. The examination of the discrepancy scores is different from the direct comparison of self versus informant ratings, as it clarifies whether the level of agreement is similar across groups, regardless of whether the two ratings in a particular study group are comparable or not. Sitek et al. (Reference Sitek, Soltan, Wieczorek, Robowski and Slawek2011) used a discrepancy score approach to investigate memory awareness in PD, but they examined the score in relation to objective memory tests, and not to a control group. Clare et al. (Reference Clare, Whitaker and Nelis2010) compared memory awareness in people with Alzheimer's disease (AD) and healthy controls using a corrected discrepancy score approach and found that the discrepancies were significantly greater in the AD group than in controls, suggesting decreased awareness of memory impairment. At the same time, the AD participants did rate their memory significantly less positively than controls, suggesting some acknowledgment of their deficits. In the present study, all three participant groups reported on average the same number of EF-related difficulties, while it was expected that PwPD_EF- would report more EF-related difficulties. Given their impaired performance in EF tests, the absence of difference may suggest that PwPD_EF- acknowledge their difficulties only partially.
In the present study, all self-reports were higher than corresponding informant reports, which is consistent with studies comparing self- and informant ratings of executive functioning (Burgess, Alderman, Evans, Emslie, & Wilson, Reference Burgess, Alderman, Evans, Emslie and Wilson1998; Rabin et al., Reference Rabin, Roth, Isquith, Wishart, Nutter-Upham and Pare2006; Roth et al., Reference Roth, Isquith and Gioia2005). The difference was statistically significant only in PwPD_EF+, in line with the study of McKinlay and colleagues (2008), where PwPD with normal general cognition (MMSE score > 25, no EF tests included) self-reported more difficulties than their informants. PwPD_EF+ may in fact have some subtle deficits experienced internally as a change in cognitive processing, which is impossible for the carer to observe, hence resulting in discrepant appraisals (McKinlay et al., Reference McKinlay, Grace, Dalrymple-Alford, Anderson, Fink and Roger2008). In contrast, deficits in PwPD_EF- may be more substantial and, therefore, evident to the carers, resulting in more similar ratings.
Self- and informant ratings versus objective measures
The interpretation of discrepancies between self- and informant ratings in terms of degree of awareness is not straightforward, as it depends on the accuracy of informant ratings prone to influence by social and interpersonal factors or the degree of stress or burden experienced (Clare, Reference Clare2004), and may be affected by the impossibility of observing some aspects of internal cognitive processing (McKinlay et al., Reference McKinlay, Grace, Dalrymple-Alford, Anderson, Fink and Roger2008). It has been argued that comparing self-ratings to objective test performance provides a useful approach, as it eliminates that bias (Dalla Barba, Parlato, Iavarone, & Boller, Reference Dalla Barba, Parlato, Iavarone and Boller1995; McLoughlin, Cooney, Holmes, & Levy, Reference McLoughlin, Cooney, Holmes and Levy1996).
In PwPD_EF+, higher BRIEF-A ratings (both self- and informant) were related to poorer performance in CWI 4, while in PwPD_EF- there was a significant correlation between BRIEF-A self report and CWI 3. These correlations would suggest that the BRIEF-A ratings offer a degree of accuracy, and that inhibition and switching abilities assessed by CWI may overlap with some aspects of the executive difficulties elicited in the BRIEF-A. However, the pattern of correlations between BRIEF-A and EF tests was not consistent, as other correlations were non-significant. Non-significant relationships between EF tests and questionnaire-based ratings have previously been reported (Koerts et al., Reference Koerts, van Beilen, Leenders, Brouwer, Tucha and Tucha2012; Rabin et al., Reference Rabin, Roth, Isquith, Wishart, Nutter-Upham and Pare2006), and have been interpreted as a consequence of the generally low ecological reliability of EF tests and a lack of overlap between the difficulties assessed by EF tests and the kinds of cognitive failures listed in the BRIEF-A (Goldberg & Podell, Reference Goldberg and Podell2000; Manchester, Priestley, & Jackson, Reference Manchester, Priestley and Jackson2004).
Performance Monitoring Level
To our knowledge, this is the first study to report on the accuracy of self-appraisal of executive task performance in PwPD. PwPD_EF- overestimated their performance in EF tests significantly more than PwPD_EF+ and controls, particularly in more challenging tasks (TMT 4, CWI 3 & 4). While the exact values of the ratios might reflect the calculation method, the group comparison objectively demonstrates that PwPD_EF- were significantly less accurate. It has been argued that cognitive processes have greater impact on self-appraisal of performance on a given task than on general evaluation of cognitive functioning, as the former requires an EF-related ability to efficiently distribute attention between the task itself and self-appraisal (Clare, Marková, et al., Reference Clare, Marková, Roth and Morris2011). This notion is supported by the present study where poorer performance monitoring was associated with poorer general cognition, in line with studies reporting an association between overestimation of test performance and poorer general cognition and EF in people with dementia (Bettcher, Giovannetti, Macmullen, & Libon, Reference Bettcher, Giovannetti, Macmullen and Libon2008; Clare et al., Reference Clare, Whitaker and Nelis2010; Graham, Kunik, Doody, & Snow, Reference Graham, Kunik, Doody and Snow2005). Executive control has been previously reported as impaired in PwPD (Ridderinkhof, Van Den Wildenberg, Segalowitz, & Carter, Reference Ridderinkhof, Van Den Wildenberg, Segalowitz and Carter2004; West, Ergis, Winocur, & Saint-Cyr, Reference West, Ergis, Winocur and Saint-Cyr1998; Zgaljardic et al., Reference Zgaljardic, Borod, Foldi, Mattis, Gordon, Feigin and Eidelberg2006); it is, therefore, not surprising that PwPD_EF- exhibited difficulties in self-appraisal of task performance, even though they were allocated to the impaired group based on tasks which do not specifically assess task-monitoring abilities (Ridderinkhof et al., Reference Ridderinkhof, Van Den Wildenberg, Segalowitz and Carter2004; Zgaljardic et al., Reference Zgaljardic, Borod, Foldi, Mattis, Gordon, Feigin and Eidelberg2006).
Correlates of Decreased Awareness
Awareness is shaped by several factors, and might be prone to psychosocial influences, especially at the evaluative judgment and meta-representational levels (Clare, Marková, et al., Reference Clare, Marková, Roth and Morris2011). This notion was illustrated in the correlational analysis; while poorer performance monitoring was associated with poorer general cognition, lower level of agreement between ratings (greater discrepancy in BRIEF-A) was related to higher levels of caregiver burden. Caregiver burden has not been previously examined in relation to awareness in PD, but it might have a profound impact on informant ratings, as is consistently reported in dementia studies (Clare, Nelis, et al., 2011; Jorm et al., Reference Jorm, Christensen, Henderson, Korten, Mackinnon and Scott1994; Rymer et al., Reference Rymer, Salloway, Norton, Malloy, Correia and Monast2002). Discrepancy scores were not correlated with depression, which is different to findings from other PD studies (Koerts et al., Reference Koerts, van Beilen, Leenders, Brouwer, Tucha and Tucha2012; Sitek et al., Reference Sitek, Soltan, Wieczorek, Robowski and Slawek2011) and might result from the relatively low levels of depression in the study groups.
Study Limitations
Some limitations of the present study must be taken into account when interpreting the findings. The interpretation of EF test results is complicated as performance may reflect various lower and higher level cognitive functions. We included PwPD with normal global cognition and controlled for potential motor impairment on the TMT, which increases the reliability of test interpretation. However, the potential role of non-executive deficits for EF performance and awareness level in PwPD needs to be acknowledged, as the two groups distinguished on the basis of performance on EF tests may possibly differ from each other in respect to other non-executive abilities, such as language. Further research might be needed to clarify the relationship between awareness and non-executive abilities. Only two tests of EF were used to distinguish between participants with normal and impaired EF. TMT and CWI capture only some aspects of EF related to inhibitory control and mental switching, and including more tasks would facilitate more accurate identification of PwPD with EF deficits. Our sample of PwPD with EF impairments was relatively small, which limits the potential to generalize these findings. EF tests are different from tasks encountered by PwPD in daily life, and this might have added to the inaccuracy of self-appraisal. It would be interesting to examine the accuracy of performance monitoring in more ecologically valid tasks, and investigate how that accuracy relates to everyday functioning (e.g., driving, medication adherence).
Conclusions
Accurate self-appraisal and performance monitoring are crucial for independent and safe day-to-day functioning. This study demonstrates that while PwPD accurately acknowledge their deficits at the general level, they may lack capacity to recognize their limitations while performing specific tasks, which may have implications for functional abilities. Performance monitoring is a new approach in assessing awareness in PD, with the results supporting the view that awareness at the evaluative judgment level involves different processes than those required for accurate monitoring of one's own performance. Future studies could explore the potential consequences of inaccurate self-appraisal for everyday functioning and examine strategies to prevent possible excess disability associated with limited awareness of functioning and performance.
Acknowledgments
The authors report no conflicts of interest. A.K. was supported by the School of Psychology, Bangor University, and by a Bangor University Anniversary PhD Bursary. We are grateful to Anthony Martyr for contributions to collection and management of the control data, and would like to acknowledge the assistance of MSc students Naomi Bowers, Katy Jones, Laura Keeney and Olga Lee.
