Introduction
Patients with Parkinson's disease (PD) experience not only motor symptoms, such as bradykinesia and rigidity, but also non-motor symptoms among which depression is one of the most frequent and debilitating (Reijnders et al. Reference Reijnders, Ehrt, Weber, Aarsland and Leentjens2008). Despite such a high prevalence and impact, the mechanisms underlying depression in PD are unclear and accordingly, treatment is often suboptimal.
Depression has been associated with an imbalance in the impact of reward and/or punishment on learning, behaviour and cognition (Clark et al. Reference Clark, Chamberlain and Sahakian2009; Eshel & Roiser, Reference Eshel and Roiser2010; Der-Avakian & Markou, Reference Der-Avakian and Markou2012; Roiser et al. Reference Roiser, Elliott and Sahakian2012; Treadway & Zald, Reference Treadway and Zald2013; Whitton et al. Reference Whitton, Treadway and Pizzagalli2015). For example, patients with depression exhibit both enhanced impact of punishment as well as reduced impact of reward on learning (Murphy et al. Reference Murphy, Michael, Robbins and Sahakian2003; Taylor Tavares et al. Reference Taylor Tavares, Clark, Furey, Williams, Sahakian and Drevets2008; Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011). Notably, negative affective biases are also observed in individuals at risk for depression in several cognitive domains, including learning, putatively representing a vulnerability factor (Forbes et al. Reference Forbes, Shaw and Dahl2007; Robinson et al. Reference Robinson, Moeller and Fetterman2010a ; Roiser et al. Reference Roiser, Elliott and Sahakian2012). In this study, we asked whether similar biases in learning from reward v. punishment contribute to depression in PD.
This question is particularly relevant given evidence that PD is accompanied by dopamine-dependent changes in the balance between reward- v. punishment-based learning, which involves dopaminergic prediction error coding in the striatum (Schultz & Dickinson, Reference Schultz and Dickinson2000). Multiple studies have shown that dopaminergic medication in PD reduces punishment-based learning, but, if anything, enhances reward-based learning (Frank et al. Reference Frank, Seeberger and O'reilly2004; Cools et al. Reference Cools, Altamirano and D'esposito2006; Moustafa et al. Reference Moustafa, Cohen, Sherman and Frank2008; Bodi et al. Reference Bodi, Keri, Nagy, Moustafa, Myers, Daw, Dibo, Takats, Bereczki and Gluck2009; Palminteri et al. Reference Palminteri, Lebreton, Worbe, Grabli, Hartmann and Pessiglione2009; Rutledge et al. Reference Rutledge, Lazzaro, Lau, Myers, Gluck and Glimcher2009; Smittenaar et al. Reference Smittenaar, Chase, Aarts, Nusselein, Bloem and Cools2012). According to the current modelling work, these drug effects reflect dopamine-induced shifts in the balance between activity in the direct and indirect pathways of the basal ganglia (Maia & Frank, Reference Maia and Frank2011). Despite consistent medication effects, discrepancy exists between studies with regard to the pattern of performance on such tasks of PD patients OFF medication. While some studies report unaltered performance in the OFF state compared with healthy controls (Cools et al. Reference Cools, Altamirano and D'esposito2006; Moustafa et al. Reference Moustafa, Cohen, Sherman and Frank2008; Rutledge et al. Reference Rutledge, Lazzaro, Lau, Myers, Gluck and Glimcher2009; Smittenaar et al. Reference Smittenaar, Chase, Aarts, Nusselein, Bloem and Cools2012), other studies report impaired reward v. punishment-based learning (Frank et al. Reference Frank, Seeberger and O'reilly2004; Bodi et al. Reference Bodi, Keri, Nagy, Moustafa, Myers, Daw, Dibo, Takats, Bereczki and Gluck2009; Palminteri et al. Reference Palminteri, Lebreton, Worbe, Grabli, Hartmann and Pessiglione2009; Kobza et al. Reference Kobza, Ferrea, Schnitzler, Pollok, Sudmeyer and Bellebaum2012). The pattern of impaired reward v. punishment learning in PD patients OFF medication resembles that described above in depressed individuals (non-PD) (Clark et al. Reference Clark, Chamberlain and Sahakian2009; Eshel & Roiser, Reference Eshel and Roiser2010) and concurs generally with suggestions that striatal dopamine depletion contributes to depression in PD. For instance, nuclear neuroimaging studies revealed that depression in PD is accompanied by decreased dopamine transporter binding, especially in ventral striatal regions, compared with non-depressed patients (Remy et al. Reference Remy, Doder, Lees, Turjanski and Brooks2005; Weintraub et al. Reference Weintraub, Newberg, Cary, Siderowf, Moberg, Kleiner-Fisman, Duda, Stern, Mozley and Katz2005; Vriend et al. Reference Vriend, Raijmakers, VeltmaN, Van Dijk, Van Der Werf, Foncke, Smit, Berendse and Van Den Heuvel2013). Functional MRI studies in depressed individuals (non-PD) have shown attenuated ventral striatal functioning across various tasks (Epstein et al. Reference Epstein, Pan, Kocsis, Yang, Butler, Chusid, Hochberg, Murrough, Strohmayer, Stern and Silbersweig2006; Forbes et al. Reference Forbes, Hariri, Martin, Silk, Moyles, Fisher, Brown, Ryan, Birmaher, Axelson and Dahl2009; Pizzagalli et al. Reference Pizzagalli, Holmes, Dillon, Goetz, Birk, Bogdan, Dougherty, Iosifescu, Rauch and Fava2009), including reward-based learning (Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011). Based on this evidence, we hypothesized that the presence of impaired reward v. punishment learning in PD patients OFF medication depends on the presence of (a history of) depression and associated ventral striatal dysfunction.
Specifically, we predicted that depressed PD patients, OFF medication, would exhibit a greater imbalance between learning from reward v. punishment and greater abnormalities in ventral striatal BOLD signal than non-depressed PD patients. Moreover, this negative learning bias and associated ventral striatal dysfunction in depressed PD patients would be remedied by dopaminergic medication. Thus, we expected dopaminergic medication to normalize reward v. punishment learning and associated ventral striatal BOLD signal in depressed patients, while impairing punishment v. reward learning and associated ventral striatal BOLD signal in non-depressed patients [cf. Cools et al. (Reference Cools, Altamirano and D'esposito2006)].
To test these hypotheses, we investigated effects of dopaminergic medication withdrawal in PD patients with and without a depression (history), using pharmacological functional magnetic resonance imaging (fMRI) and a well-established reversal learning paradigm specifically designed to disentangle reward- from punishment-based reversal learning. Previous fMRI work with this paradigm has shown that both unexpected reward and unexpected punishment elicit a prediction error signal in the striatum (Robinson et al. Reference Robinson, Frank, Sahakian and Cools2010b ). Moreover, this paradigm has been shown to be sensitive to dopaminergic manipulation in healthy volunteers as well as PD (Cools et al. Reference Cools, Altamirano and D'esposito2006, Reference Cools, Frank, Gibbs, Miyakawa, Jagust and D'esposito2009; van der Schaaf et al. Reference Van Der Schaaf, Van Schouwenburg, Geurts, Schellekens, Buitelaar, Verkes and Cools2014; Janssen et al. Reference Janssen, Sescousse, Hashemi, Timmer, Ter Huurne, GEURTS and COOLS2015) and to depression (non-PD) (Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011). Here we build on this prior work to advance our understanding of the neurochemical and neurocognitive mechanisms of depression in PD.
Materials and methods
Participants and general procedure
Twenty-four depressed and 23 non-depressed PD patients were recruited. Data from five depressed patients and one non-depressed patient were excluded from the analysis. Two depressed patients failed to complete the study, leading to incomplete datasets. One depressed patient was claustrophobic and unable to perform the task inside the MRI scanner. Three PD patients (two depressed and one non-depressed) were outliers (mean error rates across the task as a whole >3 SD from the group mean). Therefore, results are based on datasets from 19 depressed patients and 22 non-depressed patients. We aimed for a sample size of 20 patients per group. This was based on general recommendations by Thirion et al. (Reference Thirion, Pinel, Meriaux, Roche, Dehaene and Poline2007), who suggest that 20 subjects per group is an appropriate sample size for cognitive fMRI studies with a between-group design, and on previous studies that have been done using the same task and drug manipulation [sample sizes varied between 10 and 15 subjects per group (Cools et al. Reference Cools, Altamirano and D'esposito2006; Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011)].
This study was part of a larger project investigating the neurobiological mechanisms of depression in PD. All participants gave informed consent as approved by the local research ethics committee (CMO region Arnhem – Nijmegen, The Netherlands, nr. 2012/43) and were compensated for participation. Patients were recruited from the Parkinson Center at the Radboud University Medical Center, Nijmegen, the Netherlands, and were diagnosed with idiopathic PD according to the UK Brain Bank criteria by a neurologist specialized in movement disorders (Professor B. R. Bloem, Dr R. A. J. Esselink, Dr B. Post). All patients used dopaminergic medication (non-depressed: levodopa n = 10, dopamine receptor agonists n = 2, combination of both n = 10; depressed: levodopa n = 14, dopamine receptor agonists n = 2, combination of both n = 3). Patient groups were matched for amounts of daily dopaminergic medication use [levodopa equivalent dose (Esselink et al. Reference Esselink, De Bie, De Haan, Lenders, Nijssen, Staal, Smeding, Schuurman, Bosch and Speelman2004), t (39) = 1.22, p = 0.23] as well as amounts of daily dopamine receptor agonist use [t (39) = 1.47, p = 0.15]. Six depressed patients used antidepressants (Paroxetine n = 2, Escitalopram n = 1, Citalopram n = 1 and Nortriptyline n = 2). All patients were on stable medication regimes during the course of the study, except for one patient who used Duloxetine for 4 weeks between the two testing days. The drug was prescribed to treat pain and discontinued 4 weeks before the second testing day.
Exclusion criteria were clinical dementia [Mini Mental State Examination <24 (Folstein et al. Reference Folstein, Folstein and Mchugh1975)], psychiatric disorders other than depression, neurological co-morbidity and hallucinations. Patients were assigned to the depressed group if they met the DSM-IV criteria, based on structured psychiatric interviews conducted during an intake session [MINI-plus (Sheehan et al. Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998)], for a major or minor depressive episode, dysthymic disorder or adjustment disorder with depressed mood within 5 years before PD diagnosis up until now. A depression history is significantly more common in PD patients compared with age-matched controls, with odds ratios varying between 1.5 and 3.1 (Ishihara & Brayne, Reference Ishihara and Brayne2006). The criterion of 5 years was based on a previous report suggesting that depression occurring within 5 years before PD diagnosis is more likely to be PD-related (Shiba et al. Reference Shiba, Bower, Maraganore, McDonnell, Peterson, Ahlskog, Schaid and Rocca2000). From here on, we refer to these patients as depressed patients, although it should be noted that this group consists of patients with current (n = 5), but mostly past depression (n = 14) (see online Supplementary Table S1 for more information about current and past psychiatric diagnoses). None of the patients in the non-depressed group had suffered from depression during their lifetime. Groups were matched for age, gender, IQ [Dutch version of the National Adult Reading Test (Schmand et al. Reference Schmand, Bakker, Saan and Louman1991)], disease severity [Unified Parkinson Disease Rating Scale part III (Goetz & Stebbins, Reference Goetz and Stebbins2004)] and amounts of dopaminergic medication [Levodopa Equivalent Dose (Esselink et al. Reference Esselink, De Bie, De Haan, Lenders, Nijssen, Staal, Smeding, Schuurman, Bosch and Speelman2004)] (Table 1).
Table 1. Patient characteristics
Values represent number of patients or mean (s.d.). NART, National Adult Reading Test; MMSE, Mini Mental State Examination; UPDRS, Unified Parkinson Disease Rating Scale; LED, Levodopa Equivalent Dose; BDI Beck Depression Inventory, averaged across the ON and OFF session in patients.
Patients were assessed on two occasions – once ON and once after withdrawal from their dopaminergic medication for at least 18 h (24 h for controlled-release dopamine receptor agonists) (OFF). Antidepressants were not withdrawn. The order of OFF and ON sessions was counterbalanced in each group (Table 1). Current depressive symptoms were measured using the Beck Depression Inventory (BDI). Testing days always started in the morning between 8:30 and 10:30 am.
Task
We used a deterministic reversal learning paradigm (Fig. 1) similar to that used in previous studies (Cools et al. Reference Cools, Altamirano and D'esposito2006; Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011; van der Schaaf et al. Reference Van Der Schaaf, Van Schouwenburg, Geurts, Schellekens, Buitelaar, Verkes and Cools2014). The task was presented on a screen visible via a mirror attached on the head coil in the MRI scanner. On each trial, participants were shown two simultaneously presented vertically adjacent stimuli, one scene and one face. One of these stimuli was associated with reward, the other with punishment. By trial and error, subjects had to learn these deterministic stimulus-outcome associations. Unlike classical instrumental reversal learning paradigms, subjects did not choose between stimuli, but had to predict whether the highlighted stimulus was associated with reward or punishment. Subjects indicated their prediction by pressing the reward or punishment button with their least affected hand. Response mappings were counterbalanced across subjects. Stimuli were presented until a response was made, after which the actual outcome was shown. If subjects did not respond in time, a ‘Too late’ message was presented. Stimulus-outcome contingencies reversed after four to six consecutive correct predictions. Reversals were signalled by either an unexpected reward (presented after a highlighted stimulus that was previously associated with punishment) or an unexpected punishment (presented after a highlighted stimulus that was previously associated with reward). Unexpected outcomes were only presented after a correct prediction was made according to the current contingency ruling-out the possibility of reversal anticipation. Moreover, participants were informed that reversal anticipation was not possible within the structure of this task. The same stimulus was always highlighted again on the first trial after an unexpected outcome to ensure that a contingency reversal would always be paired with a reversal in motor response. Patients were familiarized with the task during the intake session and performed a practice block on each testing day.
Fig. 1. Task overview. (a) Two stimuli (a face and a scene) were simultaneously presented. One of the stimuli was highlighted with a black border. Participants were asked to predict if the highlighted stimulus was followed by reward (green happy smiley and ‘+€100’ sign) or punishment (red sad smiley and ‘−€100’ sign). Following the participants’ prediction, the actual outcome was presented (100% deterministic). (b) Example sequence of trials. In this example the face stimulus was associated with expected reward (ER) and the scene stimulus was associated with expected punishment (EP). After a series of four to six consecutive correct responses, the stimulus-outcome associations reversed, signalled by either unexpected reward or unexpected punishment.
On each testing day, subjects completed two experimental blocks of 230 trials. Each experimental block contained a short break of 30 s. The number of reversals depended on task performance and thus varied across participants. The average number of reversal trials for reward and punishment was 29(±6) and 29(±5), respectively, across groups and did not differ between groups or drug sessions.
Behavioural analysis
Error rates and reaction times were analysed with a mixed ANOVA with GROUP as a between-subject factor and REVERSAL (reversal, non-reversal), VALENCE (reward, punishment) and DRUG (OFF and ON medication) as within-subject factors. Errors were defined as misses or incorrect predictions. Errors on reversal trials were defined as incorrect predictions on the trial immediately following an unexpected outcome. All other trials were defined as non-reversal trials, including trials that were followed by an unexpected outcome. Note that unexpected outcomes only followed a correct prediction. Error rates were arcsine transformed [2 × arcsin(√x)] as is appropriate when variance is proportionate to the mean (Howell, Reference Howell1997).
Image acquisition and analysis
A Siemens TIM-Trio 3-T MRI scanner with a 32-channel head-coil was used to acquire structural and functional MRI images. Functional images were acquired using a multi-echo echoplanar imaging sequence [38 axial slices, ascending slice acquisition order, voxel size = 3.3 × 3.3 × 2.5 mm3, matrix = 64 × 64, repetition time (TR) = 2.32 s, echo time (TE) = 9.0/19.3/30.0/40.0 ms, flip angle = 90°]. Multi-echo images were acquired in order to benefit from reduced susceptibility artefacts at low echo times (Poser et al. Reference Poser, Versluis, Hoogduin and Norris2006). Structural images were acquired using a T1-weighted MP-RAGE sequence (192 slices, voxel size = 1.0 × 1.0 × 1.0 mm3, matrix = 256 × 256, TR = 2.3 s, TE = 3.03 s, flip angle = 8°).
Images were preprocessed and analysed using SPM8 (Welcome Department of Cognitive Neurology, London). Images were realigned to the first volume using data from the shortest TE to estimate realignment parameters. After realignment, a weighted summation was performed to combine all four TEs into a single dataset (Poser et al. Reference Poser, Versluis, Hoogduin and Norris2006). To this aim, 30 ‘resting-state’ images, acquired before the start of the actual experiment, were used to estimate BOLD contrast-to-noise ratio maps for each TE. These maps were used to calculate an optimal voxel-wise weighting between the four echoes using in-house software, maximizing the contribution of each echo according to its contrast-to-noise ratio. Combined images were checked for spiking artefacts, slice-time corrected to the middle slice, coregistered to the structural image, normalized to the standard Montreal Neurological Institute template, re-sampled into 2.5 × 2.5 × 2.5 mm3 isotropic voxels and smoothed with an isotropic Gaussian kernel of 8 mm full-width at half-maximum.
A first-level general linear model (GLM) was estimated that incorporated separate regressors for each possible outcome [modelled as event at time of outcome presentation, convolved with a canonical hemodynamic response function (HRF)]: unexpected punishment, unexpected reward, correctly predicted expected punishment, correctly predicted expected reward, incorrectly predicted expected outcomes and misses. An additional epoch regressor modelled the 30 s break. Twenty-nine noise regressors were added to the GLM: 24 motion regressors [six derived from the realignment procedure, their first derivatives (n = 6) and those squared (n = 12)], three parameters to model global intensity changes (time series of the mean signal from white matter, cerebral spinal fluid and out-of brain segments) and two regressors to control for BOLD signal changes related to (changes in) tremor amplitude; an electromyography amplitude regressor and its first derivative both convolved with a canonical HRF (Helmich et al. Reference Helmich, Janssen, Oyen, Bloem and Toni2011). Time series were high-pass filtered (cut-off 128 s) to remove low-frequency signals and an AR(1) model was applied to adjust for serial correlations. The two experimental blocks from one session were modelled within one GLM. Preprocessing and estimation of the GLM was performed separately for each drug session.
Individual contrast maps were generated at the first level for each drug session. The main contrast of interest was (unexpected reward–unexpected punishment). We calculated individual ‘drug-difference maps’ (OFF–ON) and ‘drug-average maps’ [(OFF + ON)/2]. These contrast maps were taken to a second-level random-effects analysis. To compare drug-effects between depressed and non-depressed patients, we submitted individual ‘drug-difference maps’ to a second level two-sample t test. To assess the main effect of drug, we submitted individual ‘drug-difference maps’ to a second level one-sample t test and to assess the main effect of group, we submitted individual ‘drug-average maps’ to a second level two-sample t test. Response hand was added as a covariate of no-interest to control for differences in response hand between groups (Table 1).
Statistical inference was performed at the voxel level using a family-wise error (FWE)-corrected threshold of p < 0.05 within an a priori defined small-volume of interest corresponding to the bilateral striatum (p sv_fwe). To this end, we combined the bilateral caudate nucleus and putamen regions extracted from the AAL atlas into one single region of interest (Tzourio-Mazoyer et al. Reference Tzourio-Mazoyer, Landeau, Papathanassiou, Crivello, Etard, Delcroix, Mazoyer and Joliot2002). For additional whole brain analyses, statistical inference was performed at the cluster level using an FWE-corrected threshold of p < 0.05 across the whole-brain (p wb_fwe) combined with a cluster-forming threshold of p < 0.001 uncorrected. Marsbar software was used to extract mean parameter estimates and assess brain–behaviour correlations.
Results
Patient and disease characteristics
As expected, patient groups differed significantly in depressive symptoms [BDI averaged across the two drug sessions, F (1,39) = 13.22, p = 0.001, ηp 2 = 0.25] (Table 1), although BDI scores of the depressed patient group still fell within the normal range (mean 8.7 ± 5.0).
Behavioural results
Task performance in general was very good (Table 2). Comparison of error rates in non-depressed and depressed PD patients revealed a significant REVERSAL × VALENCE × GROUP interaction [F (1,39) = 4.17, p = 0.048, ηp 2 = 0.10]. Breakdown of this interaction revealed a significant REVERSAL × VALENCE interaction in depressed [F (1,39) = 8.55, p = 0.009, ηp 2 = 0.32], but not in non-depressed patients (p = 0.4). The significant interaction in depressed patients was driven by an effect of VALENCE on reversal trials [F (18) = 4.86, p = 0.041, ηp 2 = 0.21]. Depressed patients made more errors on reward compared with punishment reversal trials. There was also a significant effect of REVERSAL on reward trials [F (18) = 5.12, p = 0.036, ηp 2 = 0.22], indicating that depressed patients made more errors on reward reversal trials compared with reward non-reversal trials. There was no effect of VALENCE on non-reversal trials (p = 0.6). There were no other significant interactions with GROUP or DRUG and no significant main effects of GROUP, DRUG, REVERSAL or VALENCE (Fig. 2). There were no session order effects. Analyses of reaction times are reported in the supplement.
Fig. 2. Error rates on reversal trials (unexpected reward–unexpected punishment) (in black) and non-reversal trials (expected reward–expected punishment) (in grey) as a function of group (depressed and non-depressed PD patients). Error bars represent s.e. of the mean. *p < 0.05, ** p < 0.01, ***p < 0.001.
Table 2. Error rates
Median error rate (interquartile range) per group and drug session. EP, expected punishment; UP, unexpected punishment; ER, expected reward; UR, unexpected reward.
Dopamine receptor agonists
In contrast to previous studies [cf. Cools et al. (Reference Cools, Altamirano and D'esposito2006)] we did not observe valence-specific effects of dopaminergic medication on reversal learning. Because previous literature suggests that valence-specific drug effects might be driven by patients on dopamine receptor agonists (Cools et al. Reference Cools, Altamirano and D'esposito2006), we performed a supplementary analysis, including dopamine receptor agonist use (AGONIST) as an additional between-subject factor. However, this analysis revealed no significant interactions with GROUP, DRUG or AGONIST as factor(s) and no significant main effects of GROUP, DRUG or AGONIST.
Imaging results
We were primarily interested in valence-specific striatal BOLD signal changes during unexpected outcomes in depressed v. non-depressed PD patients. Supplementary analyses on outcome-general reversal-related brain signal changes and on valence-specific brain signal changes during expected outcomes are presented in the supplement (online Supplementary Figs S1 and S2). First, given the behavioural results, we assessed group differences using a two-sample t test on individual ‘drug-average maps’ contrasting unexpected reward and punishment. This analysis revealed a significant effect of GROUP on striatal BOLD signal elicited by unexpected reward v. unexpected punishment (right putamen, x = 30, y = −14, z = 12, T = 5.05, p sv_fwe = 0.008) (Fig. 3 a). Decomposing this interaction in each group separately revealed that unexpected reward induced significantly greater increases in striatal BOLD signal than unexpected punishment in non-depressed patients (right putamen, x = 30, y = −14, z = 12, T = 5.11, p sv_fwe = 0.037; left putamen, x = −28, y = −4, z = 12, T = 4.95, p sv_fwe = 0.049). This effect was not observed in depressed patients (Fig. 3 a). There were no differences in striatal BOLD signal elicited by either unexpected reward or unexpected punishment (contrasted against baseline) between depressed and non-depressed patients, indicating that the observed difference in valence-specific striatal BOLD signal during unexpected outcomes was driven by the difference between reward and punishment. Moreover, a supplementary analysis, for which we subtracted the response to expected rewards and punishments from that to unexpected rewards and punishments, revealed a similar result: a significant group effect on striatal BOLD signal elicited by [(unexpected reward–expected reward)–(unexpected punishment–expected punishment)] (right putamen, x = 30, y = −14, z = 12, T = 4.99, p sv_fwe = 0.009). The effect was restricted to the striatum: there were no other effects elsewhere in the brain as revealed by whole brain analysis. There was no GROUP × DRUG interaction nor a main effect of DRUG on striatal BOLD signal elicited by unexpected reward v. unexpected punishment, suggesting that the above reported effects did not differ between drug sessions.
Fig. 3. BOLD signal during reward- v. punishment-based reversal learning. (a) Valence-specific BOLD signal in the striatum during unexpected outcomes [unexpected reward (UR)–unexpected punishment (UP)] for the contrast (non-depressed–depressed patients) and for both groups separately (non-depressed patients and depressed patients). Data presented at p < 0.001 uncorrected (blue) and at p < 0.005 uncorrected (red). Note that peak activations in the putamen survive an FWE-corrected threshold of p < 0.05 in our anatomically defined striatal volume of interest. (b) Brain–behaviour relationship among depressed PD patients. This plot shows a significant correlation (ρ = −0.525, p = 0.021) between differential error rate and striatal BOLD signal for the contrast [unexpected reward (UR)–unexpected punishment (UP)]. Beta values were extracted from the striatal voxels showing a significant GROUP × VALENCE interaction in the voxel-wise analysis (i.e. nine voxels in the right putamen that survived the FWE-corrected threshold of p < 0.05 within our small-volume of interest).
In the depressed PD group, we performed brain–behaviour correlations. Specifically, we extracted individual β values from the striatal cluster (right putamen) showing a significant GROUP × VALENCE interaction in the voxel-wise analysis reported in Fig. 3 a. Behaviourally, error rates on punishment reversal trials were subtracted from error rates on reward reversal trials. We used non-parametric statistics (Spearman correlation) for this subgroup analysis, given the relatively low sample size (n = 19). There was a significant correlation between these two measurements (ρ = −0.525, p = 0.021) (Fig. 3 b). Patients who made more errors on reward v. punishment reversal trials also exhibited reduced striatal BOLD signal in response to unexpected reward v. unexpected punishment. In depressed PD patients, there was no significant correlation between BDI scores and impairments in valence-specific reversal learning (ρ = 0.135, p = 0.58) and no significant correlation between BDI scores and valence-specific BOLD signal changes in the striatum during unexpected outcomes (ρ = −0.025, p = 0.92).
Antidepressants
Six depressed PD patients used antidepressants. In order to rule out their potentially confounding effect, we performed an additional analysis excluding patients who used antidepressants (i.e. non-depressed group n = 22, depressed group n = 13). Analysis of error rates revealed a qualitatively similar although not significant REVERSAL × VALENCE × GROUP interaction [F (1,33) = 3.83, p = 0.059, ηp 2 = 0.10]. Decomposition of this 3-way interaction revealed a significant REVERSAL × VALENCE interaction in the depressed group [F (12) = 7.26, p = 0.020, ηp 2 = 0.38], but not in the non-depressed group [F (21) = 0.72, p = 0.41, ηp 2 = 0.03].
We also performed additional analyses of the imaging data after excluding the patients who used antidepressants. Comparing non-depressed patients with depressed patients revealed a similar result as reported above, i.e. a significant group effect on striatal BOLD signal elicited by unexpected reward v. unexpected punishment (right putamen, x = 30, y = −14, z = 12, T = 4.70, p sv_fwe = 0.028).
Discussion
In line with our hypothesis, we demonstrate that a depression (history) in PD is accompanied by impaired reward (v. punishment) reversal learning and an attenuation of the differential striatal response to unexpected reward v. unexpected punishment. Whereas unexpected reward induced significantly greater increases in striatal BOLD signal than unexpected punishment in non-depressed patients, this was not observed in depressed patients. However, in contrast to our other hypothesis, we did not observe an effect of dopaminergic medication on reversal learning or striatal BOLD signal.
In depression, impaired reward processing and attenuated striatal function has been shown previously across multiple facets of cognition (Epstein et al. Reference Epstein, Pan, Kocsis, Yang, Butler, Chusid, Hochberg, Murrough, Strohmayer, Stern and Silbersweig2006; Steele et al. Reference Steele, Kumar and Ebmeier2007; Forbes et al. Reference Forbes, Hariri, Martin, Silk, Moyles, Fisher, Brown, Ryan, Birmaher, Axelson and Dahl2009; Pizzagalli et al. Reference Pizzagalli, Holmes, Dillon, Goetz, Birk, Bogdan, Dougherty, Iosifescu, Rauch and Fava2009). The present effect concurs directly with a finding from previous work, using the same paradigm, showing reduced reward-based reversal learning and reduced striatal signalling (albeit in a slightly more anterior region) in depressed individuals (non-PD) (Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011). This is the first study demonstrating impaired reward (v. punishment) reversal learning and an attenuated differential striatal response to unexpected reward v. punishment in depressed v. non-depressed PD patients. It might be noted that the pattern of alteration observed at the behavioural level was partly different from that observed at the neural level. Whereas learning deficits in depressed PD patients were relatively selective for reward, the impairment observed at the striatal BOLD level concerned the differential response to unexpected reward v. punishment. Yet, we believe these two findings to be related. Indeed, in depressed patients, the degree of impairment in the differential striatal response to unexpected reward v. punishment correlated with the degree of impairment in learning from unexpected reward v. punishment. Together, these results provide evidence that abnormal signalling in the striatum, the key region affected by PD, also contributes to depression-related deficits in PD.
There is discrepancy in extant literature with respect to the integrity of reward and/or punishment learning in PD patients OFF medication. Some studies have reported OFF state performance to be unaltered compared with controls (Cools et al. Reference Cools, Altamirano and D'esposito2006; Moustafa et al. Reference Moustafa, Cohen, Sherman and Frank2008; Rutledge et al. Reference Rutledge, Lazzaro, Lau, Myers, Gluck and Glimcher2009; Smittenaar et al. Reference Smittenaar, Chase, Aarts, Nusselein, Bloem and Cools2012), whereas other studies have revealed impaired reward relative to punishment learning/performance (Frank et al. Reference Frank, Seeberger and O'reilly2004; Bodi et al. Reference Bodi, Keri, Nagy, Moustafa, Myers, Daw, Dibo, Takats, Bereczki and Gluck2009; Palminteri et al. Reference Palminteri, Lebreton, Worbe, Grabli, Hartmann and Pessiglione2009; Kobza et al. Reference Kobza, Ferrea, Schnitzler, Pollok, Sudmeyer and Bellebaum2012). The current data suggest that these discrepancies might reflect differences in the inclusion of patients with or without a depression (history). As such, our observations demonstrate that (striatal) reward learning deficits in PD depend on the presence of a depression (history) and highlight the importance of taking into account depression history in PD patients when investigating reward (v. punishment) learning.
The present study demonstrates attenuated brain responses to reward v. punishment in depressed PD patients in a posterior striatal region. This contrasts with some previous studies in depressed individuals (non-PD) showing blunted striatal responses in more anterior striatal regions (Epstein et al. Reference Epstein, Pan, Kocsis, Yang, Butler, Chusid, Hochberg, Murrough, Strohmayer, Stern and Silbersweig2006; Steele et al. Reference Steele, Kumar and Ebmeier2007; Robinson et al. Reference Robinson, Cools, Carlisi, Sahakian and Drevets2011). This discrepancy might reflect the effect of PD in our study. Critically, a similar posterior striatal locus of reward v. punishment prediction error coding has been previously shown using the same paradigm in healthy subjects (Robinson et al. Reference Robinson, Frank, Sahakian and Cools2010b ). This was argued to reflect recruitment of instrumental mechanisms in the context of reward (Robinson et al. Reference Robinson, Frank, Sahakian and Cools2010b ). Accordingly, the present effect might reflect an inability of depressed patients to recruit reward-guided instrumental actions (Henriques et al. Reference Henriques, Glowacki and Davidson1994; Pizzagalli et al. Reference Pizzagalli, Jahn and O'shea2005).
In contrast to our hypothesis, and contrary to previous studies (Frank et al. Reference Frank, Seeberger and O'reilly2004; Cools et al. Reference Cools, Altamirano and D'esposito2006; Moustafa et al. Reference Moustafa, Cohen, Sherman and Frank2008; Bodi et al. Reference Bodi, Keri, Nagy, Moustafa, Myers, Daw, Dibo, Takats, Bereczki and Gluck2009; Palminteri et al. Reference Palminteri, Lebreton, Worbe, Grabli, Hartmann and Pessiglione2009), we did not observe valence-specific drug effects. We are puzzled by this lack of effect and provide two possible accounts. First, valence-specific drug effects on (reversal) learning have been shown primarily with dopamine receptor agonists and antagonists (Cools et al. Reference Cools, Altamirano and D'esposito2006, Reference Cools, Frank, Gibbs, Miyakawa, Jagust and D'esposito2009; Moustafa et al. Reference Moustafa, Cohen, Sherman and Frank2008; Bodi et al. Reference Bodi, Keri, Nagy, Moustafa, Myers, Daw, Dibo, Takats, Bereczki and Gluck2009; van der Schaaf et al. Reference Van Der Schaaf, Van Schouwenburg, Geurts, Schellekens, Buitelaar, Verkes and Cools2014; Janssen et al. Reference Janssen, Sescousse, Hashemi, Timmer, Ter Huurne, GEURTS and COOLS2015). In contrast to previous studies, in our sample only less than half of the patients used dopamine receptor agonists (17/41). Moreover, most patients in our sample (15/17) used controlled-release dopamine receptor agonists for which one might argue that the withdrawal period was too short. However, the behavioural pattern (across both patient groups) observed in the current study was more akin to that seen in previous studies when patients were in an OFF rather than an ON state, suggesting that the effects of controlled-release dopamine receptor agonists on valence-specific (reversal) learning might not be comparable to those of regular dopamine receptor agonists. A second, not mutually exclusive possibility is that our failure to observe the predicted medication effect might reflect a ceiling effect: in the present study patients performed extremely well, and much better than did the patients in our previous study (Cools et al. Reference Cools, Altamirano and D'esposito2006). The median error rate OFF (across patients groups) for unexpected punishment was 0.06 and 0.08 for unexpected reward in the current study, while it was 0.12 for unexpected punishment and 0.20 for unexpected reward in our previous study (Cools et al. Reference Cools, Altamirano and D'esposito2006). Thus, it is possible that there was insufficient dynamic range for any medication-induced improvement in valence-specific learning to surface.
A potential caveat of the present study is the heterogeneous sample of depressed PD patients, which included patients with current as well as past depression. Although the sample sizes of both patients groups (n = 19 and 22) were large enough for a cognitive fMRI study with a between-group design (Thirion et al. Reference Thirion, Pinel, Meriaux, Roche, Dehaene and Poline2007), we lacked sufficient power for comparing PD patients with current (n = 5) v. past (n = 14) depression. Negative (learning) biases have been shown in never-depressed individuals at risk for depression (Forbes et al. Reference Forbes, Shaw and Dahl2007; Robinson et al. Reference Robinson, Moeller and Fetterman2010a ). Moreover, outside the domain of learning, it has been shown that negative affective biases can persist after remission of a depressive episode [see for review Roiser et al. (Reference Roiser, Elliott and Sahakian2012)]. However, the hypothesis that negative learning biases persist (or diminish) with remission of a depressive episode has never been investigated. The present results should therefore be validated in a follow-up study that includes a larger group of depressed PD patients enabling comparison of patients with past and current depression. In addition, six depressed patients used antidepressants. It is well known that other neurotransmitters than dopamine, such as serotonin, can influence reward v. punishment learning (Cools et al. Reference Cools, Robinson and Sahakian2008; Robinson et al. Reference Robinson, Cools and Sahakian2012). However, a supplementary analysis after excluding patients who used antidepressants revealed similar behavioural as well as neural findings, increasing our confidence in the results.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291717000769.
Acknowledgements
We would like to thank all participants for their cooperation in the study. Furthermore, we are grateful to Dr Rick Helmich and Michiel Dirkx for their help with analysing the data. This project was funded by a grant from the ‘Stichting Parkinson Fonds’, Hoofddorp, the Netherlands.
Declaration of interest
None.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
