Hostname: page-component-7b9c58cd5d-wdhn8 Total loading time: 0 Render date: 2025-03-15T22:21:19.910Z Has data issue: false hasContentIssue false

From laboratory to life: associating brain reward processing with real-life motivated behaviour and symptoms of depression in non-help-seeking young adults

Published online by Cambridge University Press:  29 November 2018

Jindra M. Bakker*
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands Department of Neuroscience, KU Leuven, Center for Contextual Psychiatry, Leuven, Belgium
Liesbet Goossens
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands
Poornima Kumar
Affiliation:
McLean Hospital, Center for Depression, Anxiety and Stress Research, Belmont, MA, USA Department of Psychiatry, Harvard Medical School, Boston, MA, USA
Iris M. J. Lange
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands
Stijn Michielse
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands
Koen Schruers
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands Department of Psychology, KU Leuven, Leuven, Belgium
Jojanneke A. Bastiaansen
Affiliation:
Department of Psychiatry (UCP), University of Groningen, University Medical Centre Groningen, Interdisciplinary Center Psychopathology and Emotion regulation (ICPE), Groningen, The Netherlands Friesland Mental Health Care Services, Leeuwarden, The Netherlands
Ritsaert Lieverse
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands
Machteld Marcelis
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands Institute for Mental Health Care Eindhoven (GGzE), Eindhoven, the Netherlands
Thérèse van Amelsvoort
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands
Jim van Os
Affiliation:
Department of Psychiatry and Psychology, Maastricht University, Maastricht University Medical Centre, School for Mental Health and Neuroscience, Maastricht, The Netherlands Department of Psychiatry, Utrecht University, University Medical Center, Brain Center Rudolf Magnus, Utrecht, The Netherlands Department of Psychosis Studies, King's College, King's Health Partners, Institute of Psychiatry, London, UK
Inez Myin-Germeys
Affiliation:
Department of Neuroscience, KU Leuven, Center for Contextual Psychiatry, Leuven, Belgium
Diego A. Pizzagalli
Affiliation:
McLean Hospital, Center for Depression, Anxiety and Stress Research, Belmont, MA, USA Department of Psychiatry, Harvard Medical School, Boston, MA, USA
Marieke Wichers
Affiliation:
Department of Psychiatry (UCP), University of Groningen, University Medical Centre Groningen, Interdisciplinary Center Psychopathology and Emotion regulation (ICPE), Groningen, The Netherlands
*
Author for correspondence: Jindra M. Bakker, E-mail: jindra.bakker@maastrichtuniversity.nl
Rights & Permissions [Opens in a new window]

Abstract

Background

Depression has been associated with abnormalities in neural underpinnings of Reward Learning (RL). However, inconsistencies have emerged, possibly owing to medication effects. Additionally, it remains unclear how neural RL signals relate to real-life behaviour. The current study, therefore, examined neural RL signals in young, mildly to moderately depressed – but non-help-seeking and unmedicated – individuals and how these signals are associated with depressive symptoms and real-life motivated behaviour.

Methods

Individuals with symptoms along the depression continuum (n = 87) were recruited from the community. They performed an RL task during functional Magnetic Resonance Imaging and were assessed with the Experience Sampling Method (ESM), completing short questionnaires on emotions and behaviours up to 10 times/day for 15 days. Q-learning model-derived Reward Prediction Errors (RPEs) were examined in striatal areas, and subsequently associated with depressive symptoms and an ESM measure capturing (non-linearly) how anticipation of reward experience corresponds to actual reward experience later on.

Results

Significant RPE signals were found in the striatum, insula, amygdala, hippocampus, frontal and occipital cortices. Region-of-interest analyses revealed a significant association between RPE signals and (a) self-reported depressive symptoms in the right nucleus accumbens (b = −0.017, p = 0.006) and putamen (b = −0.013, p = .012); and (b) the quadratic ESM variable in the left (b = 0.010, p = .010) and right (b = 0.026, p = 0.011) nucleus accumbens and right putamen (b = 0.047, p < 0.001).

Conclusions

Striatal RPE signals are disrupted along the depression continuum. Moreover, they are associated with reward-related behaviour in real-life, suggesting that real-life coupling of reward anticipation and engagement in rewarding activities might be a relevant target of psychological therapies for depression.

Type
Original Articles
Copyright
Copyright © Cambridge University Press 2018 

Introduction

Reduced approach behaviour has long been implicated in mood-related psychopathology (Lewinsohn and Graf, Reference Lewinsohn and Graf1973). A likely neurobiological substrate of such behaviour is dopaminergic (DA) projections in frontostriatal circuits (Schultz et al., Reference Schultz, Tremblay and Hollerman2000; Hamid et al., Reference Hamid, Pettibone, Mabrouk, Hetrick, Schmidt, Vander Weele, Kennedy, Aragona and Berke2016). A relevant DA signal is the reward prediction error (RPE) – an increase in neuronal firing after unexpected rewards (Bayer and Glimcher, Reference Bayer and Glimcher2005) – which is thought to regulate learning about which cues predict future rewards, leading to cue-motivated approach behaviour.

To measure this signal in humans, laboratory-based reward learning (RL) paradigms can be administered in which certain cues are probabilistically associated with rewards. The RPE signal can then be modelled using a computational approach (Sutton and Barto, Reference Sutton and Barto1998) to examine, with functional Magnetic Resonance Imaging (fMRI), wherein the brain activation correlates with this signal [model-based fMRI (O'Doherty et al., Reference O'Doherty, Hampton and Kim2007)]. Across many studies in healthy controls, RPE signals have been found in the ventral and dorsal striatum (Chase et al., Reference Chase, Kumar, Eickhoff and Dombrovski2015). Interestingly, studies by Kumar et al. (Reference Kumar, Waiter, Ahearn, Milders, Reid and Steele2008) and Gradin et al. (Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders, Reid, Hall and Steele2011) report this signal to be blunted in medicated individuals diagnosed with major depressive disorder (MDD). However, Rothkirch et al. (Reference Rothkirch, Tonn, Kohler and Sterzer2017) did not replicate this finding in unmedicated MDD patients. Hence, it remains unknown whether blunted striatal RPE signals underlie MDD, or whether previous findings were confounded by medication use.

A complementary approach to investigating reward-related behaviour comes from the Experience-Sampling Method (ESM) (Csikszentmihalyi and Larson, Reference Csikszentmihalyi and Larson1987). This method samples affect and behaviours frequently and in the context of one's daily life, thereby revealing how moment-to-moment experiences and behavioural patterns can interact differently at different stages of psychopathology (Myin-Germeys et al., Reference Myin-Germeys, Oorschot, Collip, Lataster, Delespau and van Os2009; Wichers, Reference Wichers2014). Interestingly, despite the finding that individuals with depressive symptoms reported a similar increase in Positive Affect (PA) following a pleasant event (Bylsma et al., Reference Bylsma, Taylor-Clift and Rottenberg2011; Thompson et al., Reference Thompson, Mata, Jaeggi, Buschkuehl, Jonides and Gotlib2012, van Roekel et al., Reference van Roekel, Bennik, Bastiaansen, Verhagen, Ormel, Engels and Oldehinkel2016), they reported relatively fewer pleasant events (van Roekel et al., Reference van Roekel, Bennik, Bastiaansen, Verhagen, Ormel, Engels and Oldehinkel2016), indicating reduced reward approach behaviour despite intact hedonic reactivity. This latter finding was extended by Bakker et al. (Reference Bakker, Goossens, Lange, Michielse, Schruers, Lieverse, Marcelis, van Amelsvoort, van Os, Myin-Germeys and Wichers2017), who found depressive symptoms to be associated with a reduced active behavioural response following reports of anticipating a pleasurable situation, despite similar PA responses to active behaviour.

Importantly, even though both neuroimaging and ESM literature refer to reward-related behaviour and its relevance in MDD research, the relationship between the two different approaches has rarely been investigated. Combining ESM and neuroimaging techniques can illuminate how neural abnormalities might be manifested in real-world behaviours as well as enrich the construct validity of ESM measures (Wilson et al., Reference Wilson, Smyth and MacLean2013). Heller et al. (Reference Heller, Fox, Wing, McQuisition, Vack and Davidson2015) showed that the duration of ventral striatal reactivity to winning money in a fMRI scanner was associated with the duration of PA increase in response to winning money in a task performed in the field, showing that neural processes – unfolding over seconds – and the experience of emotions – lasting minutes – might be traced back to common pathways. In the context of RL and real-world behaviour; a prior Positron Emission Tomography (PET) study in healthy controls found RL-induced striatal DA release to be associated with daily-life reward-oriented behaviour (Kasanova et al., Reference Kasanova, Ceccarini, Frank, Amelsvoort, Booij, Heinzel, Mottaghy and Myin-Germeys2017). However, the PET paradigm used in Kasanova et al. (Reference Kasanova, Ceccarini, Frank, Amelsvoort, Booij, Heinzel, Mottaghy and Myin-Germeys2017) could not separate DA responses to unexpected rewards (e.g. RPE) and reward-predicting stimuli. Hence, the current study will be the first to investigate how model-based striatal RPE responses relate to behaviour in a real-world context.

In doing so, we focused on an ESM measure hypothesized to capture a daily-life process related to RPE. Specifically, since the RPE involves a clear anticipatory component, it was examined how a momentary report of looking forward to a situation in the future (i.e. the anticipation of something pleasant) was related to the subsequent report of enjoyment of being engaged in an activity (~90 min later). A higher association between these time-lagged measures could be an indication that one is better able to predict a pleasurable activity sometime later, or that one is better able to change behaviour towards pleasant activities following the anticipation of reward. Both processes are possible results from intact stimulus-response learning processes, which is assumed to be driven by adequate RPE signals.

In sum, reduced Positive Valence System function, and specifically impaired model-based fMRI RPE signals, could be important in MDD. In this context, the current study aims to add insight into two unanswered questions; first, it is unclear how RPE-related neural activity might be disrupted along the continuum of depressive symptoms in the absence of antidepressant treatments. Second, little is known about how RPE brain signals translate into daily-life reward dynamics. Hence, the current study examined – in a sample of non-help-seeking and unmedicated individuals along the depression continuum (up to moderate severity) – the association between RPE brain activity and depressive symptoms as well as a relevant ESM measure.

Methods

Participants

A total of 132 young adults (age 16–25) were included between September 2013 and January 2017 as part of the SMARTSCAN study – a randomized controlled trial examining the effect of psychotherapeutic training on daily life functioning and neurobiological correlates, conducted at Maastricht University Medical Centre (Dutch Trial Register nr.: NTR3808). Since the current paper focuses on a small part of the SMARTSCAN baseline, only procedures relevant to the current analyses will be discussed.

Participants reporting symptoms along the depression continuum were recruited from the general population via advertisements in public places and social media. Individuals with sub-threshold symptoms of depression [Mild/Moderate Depression (m/m-D) group: n = 81] were oversampled by specifically including those with a Montgomery–Åsberg Depression Rating Scale [MADRS (Montgomery and Åsberg, Reference Montgomery and Åsberg1979)] total score of 10 or higher, unless they received current psychiatric/psychological treatment or had a significant need for care as assessed by a psychiatrist. Participants with a MADRS score below 10 [No/Low Depression (n/l-D) group: n = 51] were included, unless they reported current and/or lifetime psychological/psychiatric treatment or a psychiatric diagnosis [as assessed with the Mini International Neuropsychiatric Interview – MINI, (Overbeek et al., Reference Overbeek, Schruers and Griez1999)]Footnote Footnote 1. Overall, participants were excluded based on MRI contraindications (e.g. metal implants, pregnancy, claustrophobia, neurological disorders) or being left-handed.

The Medical Ethics Committee of Maastricht University Medical Centre approved all study procedures (protocol number: NL41929.068.12 / METC 12-2-072), and all participants provided written informed consent (additionally signed by their proxy if age<18).

Procedure

Study procedures are summarized in Fig. 1a. Depressive symptomatology was assessed with an interviewer-rated [MADRS (Montgomery and Åsberg, Reference Montgomery and Åsberg1979)] and a self-report [Inventory of Depressive Symptomatology (IDS-SR); Rush et al., Reference Rush, Gullion, Basco, Jarrett and Trivedi2009)] scale, since it has been established that each can measure unique aspects of depression (Uher et al., Reference Uher, Farmer, Maier, Rietschel, Hauser, Marusic, Mors, Elkin, Williamson, Schmael, Henigsberg, Perez, Mendlewicz, Janzing, Zobel, Skibinska, Kozel, Stamp, Bajs, Placentino, Barreto, McGuffin and Aitchison2007; Uher et al., Reference Uher, Perlis, Placentino, Dernovšek, Henigsberg, Mors, Maier, McGuffin and Farmer2012). MADRS score was measured both at the inclusion (first) and scan (second) sessions, and IDS-SR was collected only at the scanning session.

Fig. 1. Study and task procedures. (a) Study procedure. (b) Task procedure, adapted from Gold et al. (Reference Gold, Waltz, Matveeva, Kasanova, Strauss, Herbener, Collins and Frank2012). MADRS, Montgomery–Åsberg Depression Rating Scale (Montgomery and Åsberg, Reference Montgomery and Åsberg1979); MINI, Mini International Neuropsychiatric Interview (Overbeek et al., Reference Overbeek, Schruers and Griez1999); IDS-SR, Inventory of Depressive Symptomatology (Self-Report) (Rush et al., 2009); RL, Reinforcement Learning; fMRI, functional Magnetic Resonance Imaging; ESM, Experience Sampling Method, m/m-D, participants with moderate symptoms of depression; n/l-D, participants with low symptoms of depression. 1One participant only carried the device for 7 days since initially included in another arm of the SMARTSCAN study.

Reinforcement learning task

The task was adapted from Gold et al. (Reference Gold, Waltz, Matveeva, Kasanova, Strauss, Herbener, Collins and Frank2012, see Fig. 1b). On each trial, participants had to select one stimulus in one of three pairs of stimuli (gain, loss or neutral; 80 trials per condition). Participants were instructed to find out, within each pair, which stimulus was ‘better’ most of the time based on the feedback received after each selection (Gain:+€0.20 v. no gain; Loss: no loss v. −€0.20; Neutral: no monetary outcome but written feedback, ‘Correct’ v. ‘Incorrect’). Selection of the better picture (optimal choice) resulted in a better outcome in 70% of trials; the other stimulus of the same pair associated with the optimal outcome in 30% of trials. Stimuli were presented side by side; the position of the stimuli was counterbalanced across trials. Conditions were pseudo-randomly ordered within 4 blocks of 60 trials (i.e. 20 trials per condition per block, total duration: ~24 min). The current paper focuses only on gain trials, but the other two conditions were modelled in the imaging analyses (see below).

MRI data acquisition and processing

Data acquisition

MRI scans were acquired using a 3T Siemens Magnetom Prisma scanner at the Scannexus facilities, Maastricht, the Netherlands. See the Supplemental Materials for details.

Processing

For each individual, functional data were slice-time corrected, realigned to the mean image, co-registered to the anatomical image and normalized to 2 × 2 × 2 MNI space and smoothed with a 4 mm Gaussian kernel using the Statistical Mapping software package (SPM12). See Supplemental Materials for a more detailed description.

ESM procedure and measures

Procedure

Participants carried a dedicated device (the PsyMate©), which was programmed to emit a signal (‘beep’) at semi-random moments, within 90 min blocks, between 07:30 and 22:30 (i.e. 10 beeps a day). At each beep, participants completed a brief beep-questionnaire on the PsyMate© including reports on current (positive and negative) mood, (social and physical) context, daily events and activities, on a seven-point Likert-scale. Participants were able to fill in the questionnaire up to 10 min after the initial beep but were asked to fill in the questionnaires as soon as possible in order to reduce recall bias. All but one participant carried the device for 15 days (receiving 150 beeps); one participant who was initially included in another arm of SMARTSCAN but fulfilled criteria for the current sample, did so for 7 days (receiving 70 beeps). Following earlier work (Delespaul, Reference Delespaul1995), participants who filled in less than 30% of received beeps were excluded from analyses (excluded: 1 n/l-D, 1 m/m-D). In addition, subjects who filled in beep-questionnaires on less than 6 days were also excluded (excluded: 1 m/m-D).

Measures

Two ESM items (see below) were combined in a statistical modelFootnote 2 in order to form one ESM reward measure, by predicting the level of activity pleasantness at time t with the level of reward anticipation at time t-1, thereby indicating whether higher levels of reward anticipation were followed by higher levels of activity pleasantness, both actualized as follows:

Reward anticipation was assessed in two steps: (1) ‘Think about what you consider to be the most important situation in the next hour…’ and (2) ‘How much are you looking forward to this situation?’.

Activity pleasantness was assessed in two steps: (1) ‘Think about what you were doing right before the beep-signal…’ and (2) ‘I enjoy doing this’.

Analyses

RL task – behaviour

A standard Q-learning algorithm was used to calculate the expected value of, and prediction error after, each choice (Sutton and Barto, Reference Sutton and Barto1998). Additionally, it was tested how well the RL model fitted the observed data relative to chance (see Supplement for more detailed information).

Associations with depressive symptoms were examined by regressing total MADRS and IDS-SR scores with a model-fit measure (pseudoR2) and learning parameters.

RL task – imaging

First-level general linear models included 19 regressors which were convolved with a hemodynamic response function: cue and feedback onsets of gain, loss and neutral trials (6 regressors); parametric modulation of cue and feedback onset times (only gain and loss condition) by model-derived expected value and prediction error respectively (4 regressors); cue and feedback onsets of non-response trials (2 regressors); six motion realignment parameters and a constant term (7 regressors). A linear contrast of the regression coefficient of RPE was computed at the individual level and resulting contrast images were taken to conduct within-subject t test. Whole brain analyses across all subjects were run with a cluster-extent based threshold of p < 0.05 FWE with p < 0.001 as the initial cluster-forming threshold.

ROI analyses. Anatomically constrained bilateral putamen and nucleus accumbens (NAcc) were extracted from a recent meta-analysis of RPE studies in healthy controls (Chase et al., Reference Chase, Kumar, Eickhoff and Dombrovski2015; Fig. 2b). These anatomical masks were created from the FSL Harvard-Oxford subcortical atlas using 40% probability threshold. RPE parameter estimates from these four ROIs were derived using the REX toolbox (https://www.nitrc.org/projects/rex/).

Fig. 2. fMRI results and correlations between RPE and self-reported depressive symptoms. (a) Active clusters during Reward Prediction Error (average over the whole group). (b) Regions of Interest, extracted from Chase et al. (Reference Chase, Kumar, Eickhoff and Dombrovski2015). (c) Mean activation per Region of Interest for each participant. (d) Scatterplots of RPE beta weight and IDS-SR total score for the right Nucleus Accumbens and Putamen. L, left; R, right; NAcc, Nucleus Accumbens; RPE, reward prediction error; IDS-SR, Inventory of Depressive Symptomatology (Self-Report).

Association of Reward learning (RPE) signals with depressive symptoms and a measure of real-life motivated behaviour (measured using ESM).

Depressive symptoms. As our aim was to investigate RPE-related neural activity along the continuum of depressive symptoms, we adopted a dimensional approach. RPE signals from each ROI were, therefore, linearly regressed with total MADRS score and total IDS-SR score using the REGRESS command in STATA 13.1 (StataCorp, 2013).

ESM. It was examined whether RPE moderated the association between reward anticipation at time t-1 and activity pleasantness at time t. ESM data have a hierarchical structure (multiple observations – Level 1 – are clustered within participants – Level 2). Multilevel (mixed effects) linear regression analyses take the variability associated with each level of nesting into account (Snijders and Bosker, Reference Snijders and Bosker1999). For these analyses, the MIXED command in STATA 13.1 (StataCorp, 2013) was used. To control for potential time trends in the ESM data, analyses controlled also for sampling time (in days, including two decimals, starting from midnight of the first sampling day increasing up to the last completed beep questionnaire) as well as the time within a day (in minutes, starting from midnight that day). Since we were mostly interested in within-person effects, predictors of the analyses with lagged variables were person-mean-centred. These models additionally included the lagged version of the outcome variable in order to control the effect of the other lagged predictors for the autocorrelation in the dependent variable. The structure of the residual errors within the lowest-level groups was left to default (i.e. independent and identically distributed with one common variance)Footnote 3. Lastly, since not all models in which the variance-covariance matrix of the random effects was set to unstructured (thereby allowing estimation of all variances and covariances separately) converged, this structure was set to an identity matrix in which covariances were assumed to be 0. In addition to a standard maximum likelihood approach (using the observed information matrix to estimate the variance-covariance matrix of the fixed effects), we ran analyses using the robust Huber–White sandwich estimator (using subjects as the clustering variable), which is robust to some types of misspecification of the variance-covariance matrix (Rogers, Reference Rogers1993). For a complete description of the models that were tested, see Supplemental Materials.

All analyses controlled for age (in years) and gender. For each outcome, analyses were corrected for multiple testing of the four ROIs following the Holm method (Holm, Reference Holm1979; Aickin and Gensler, Reference Aickin and Gensler1996).

Results

Participants

Of the 132 included participants, 114 provided complete data. After quality checks of behavioural, imaging and ESM data, 87 participants were analyzed (16 excluded for RL task performance, 8 for MRI-related artifacts; 3 for ESM-related inconsistencies, see online Supplementary Fig. S1). There were no significant differences in age, gender, total MADRS score at inclusion and IDS-SR total scoreFootnote 4 between analyzed and not analyzed participants (all ps>0.2). Table 1a summarizes sociodemographic and clinical characteristics of the final sample (n = 87; n/l-D = 37 and m/m-D = 50). The groups did not differ significantly in age, gender or education (all ps>0.10). As expected, groups differed significantly in their depression severity as measured by interview (MADRS, both at inclusion (p < 0.001) and scan (p < 0.001) sessions) and self-report (IDS-SR, at scan session, p < 0.001), as well as average levels of daily-life reward anticipation (p < 0.001) and activity pleasantness (p < 0.001), with the m/m-D group reporting more symptoms and lower levels of the ESM items.

Table 1. Demographic and clinical descriptive information (a); Information regarding behavioural performance on the Reward Learning task (b); and computational modelling output (c)

s.d., standard deviation; s.d._b, between subjects standard deviation; s.d._w, within subjects standard deviation; n, number of participants; MADRS, Montgomery-Åsberg Depression Scale; IDS-SR, Inventory of Depressive Symptomatology – Self-Report; RL, Reward Learning; RT, reaction time, ms, miliseconds; m/m-D, group of participants included based on the report of moderate symptom severity; n/l-D,  group of participants included based on the report of low depressive symptom severity, CI, confidence interval, LLH, log-likelihood.

*p < 0.001.

a AIC: Akaike Information Criterion, calculated as −2 × LLH + 2 × k, where k is the number of parameters (k = 2).

b BIC: Bayesian Information Criterion, calculated as −2 × LLH + log(t) × k, where t is the number of trials (depends on how many responses a subject gave on the task) and k is the number of parameters (k = 2).

c PseudoR: calculated as -(LLHmodel-LLHchance)/LLHchance. LLHchance = log(0.5) × t, where t is the number of trials.

RL task performance and model fitting

Participants with optimal choice accuracy on the gain trials ⩽55% were excluded from further analyses since it could not be ascertained whether they were engaged in the task (excluded: 6 n/l-D, 10 m/m-D). Depressive symptoms (interview and self-report) were not associated with average reaction time, the number of non-responses, average optimal choice accuracy or the total amount of money won on gain trials (all ps>0.2; Table 1b).

Additionally, depressive symptoms (interview and self-report) were not associated with model-fit-measures nor learning parameters (all ps>0.2; Table 1c). See online Supplementary Fig. 2 for choice probabilities predicted by the computational model. For the fMRI analyses, a fixed alpha was chosen and a learning model was fitted with a single set of parameters (Pessiglione et al., Reference Pessiglione, Seymour, Flandin, Dolan and Frith2006; Daw, Reference Daw, Delgado, Phelps and Robbins2011). Specifically, we used averaged estimates of alpha calculated across all subjects during reward (alpha: 0.2).

fMRI

Replicating previous results, across all subjects significant RPE clusters were found in the dorsal and ventral striatum extending to the amygdala, cingulate gyrus, frontal cortex, inferior parietal lobule (Garrison et al., Reference Garrison, Erdeniz and Done2013; Chase et al., Reference Chase, Kumar, Eickhoff and Dombrovski2015) as well as in the hippocampus and precuneus (see Fig. 2a and online Supplementary Table S1).

ROI analyses indicated that average activity in all four regions (bilateral NAcc and putamen) was significantly correlated with model-based RPE signals (see online Supplementary Table S3 and Fig. 2c) across all subjects.

Associations of RPE from each ROI with depression symptoms

None of the associations between RPE signals in each of the ROIs and MADRS total score were significant (all ps>0.20). However, the association between RPE and IDS-SR total score was significant for the right putamen (b = −0.13, p = 0.009) and right NAcc (b = −0.017, p = 0.010), indicating decreased RPE-related striatal activity in participants with higher self-reported depression scores (see Table 2 and Fig. 2d).

Table 2. Associations between mean activation in R NAcc and Putamen and reward-related ESM measures (controlled for age and gender, and the ESM analyses also for overall time of measurement, time of measurement within the day)

All ESM predictors were person-mean-centered. Models with time-varying predictors include random slopes of those predictors (incl. quadratic variables). Models with lagged predictors include the lagged outcome as a predictor. L, left; R, right; NAcc, Nuccleus Accumbens; MADRS, Montgomery–Åsberg Depression Scale; IDS-SR, Inventory of Depressive Symptomatology – Self-Report; s.e., standard error; p, p value; s.e.ML, standard error estimated using standard log-likelihood procedure; s.e.rob, standard error estimated using the robust Huber–White sandwich estimation procedure; pML, p value of effect based on the log-likelihood standard error estimation; prob, p value of effect based on robust Huber–White sandwich estimation; qua, quadratic (referring to time-lagged quadratic effect of reward anticipation on activity pleasantness; lin, linear (referring to time-lagged linear effect of reward anticipation on activity pleasantness).

*Indicates a significant p value after correction for multiple testing following the Holm method (Holm, Reference Holm1979; Aickin and Gensler, Reference Aickin and Gensler1996), correction performed within each row of the table.

Associations of RPE from each ROI with the ESM reward measure

See Supplemental Materials for details on beep questionnaire response rates.

ROI RPEs did not significantly moderate the linear relation between reward anticipation and activity pleasantness (all ps>0.10). However, after visual inspection of the data, it seemed that a non-linear relation between reward anticipation and activity pleasantness might be a better fit. Hence, post-hoc analyses were performed examining the moderating role of ROI RPE on the quadratic association between reward anticipation and activity pleasantness, yielding significant results for the left NAcc (b = 0.031, p = 0.010), right NAcc (b = 0.026, p = 0.011) and right putamen (b = 0.047, p < 0.001). In other words, the association between reward anticipation (squared) at time t-1 and activity pleasantness at time t (on average 90 min later) depended on how well brain activity followed the model-derived RPE signal, with an increased association between these time-lagged ESM variables for higher brain RPE-related activity levels. These results remained significant when using a robust – but more rigorous – estimation of the standard errors of the effects (see Table 2 and Fig. 3).

Fig. 3. Simple slopes to illustrate significant moderating effects of RPE beta weight of three ROI's on the time-lagged association between Reward anticipation and Activity pleasantness, at three levels of the RPE beta weight (group average, and 2 standard deviations below/above this group average). R, right; L, left; NAcc, Nucleus Accumbens; EMM, estimated marginal mean; WP, within-person; s.d., standard deviation; RPE, Reward Prediction Error; ROI, Region of Interest

Exploratory analyses examined whether ROI RPE mean activations were associated with the individual ESM items that comprised the ESM measure of interest (reward anticipation and activity pleasantness). No significant associations were found for any of the ROIs (all ps>0.05, see Table 2).

Discussion

The current study found that RPE signals in the right putamen and nucleus accumbens correlated with self-reported depression severity suggesting that individuals with higher depression severity scores exhibit lower striatal RL signals. Second, although RPE-related brain activity in these areas did not moderate the lagged linear association between reward anticipation and activity pleasantness, quadratic effects emerged. Specifically, RPE-related brain activity in the putamen (right) and NAcc (left and right) moderated the lagged relation between quadratic reward anticipation and activity pleasantness. This suggests that a lower striatal RPE signal during a laboratory-based RL task was associated with increased decoupling between real-life enjoyment of activities and previous anticipation of pleasure (possibly akin to a real-life prediction error). Collectively, these findings reveal an association between how the brain processes RPEs under abstract and controlled circumstances and how much people change their behaviour towards pleasant activities following the anticipation of reward during day-to-day life.

Striatal RPE and depressive symptomatology

The current paper used a dimensional approach, with which we found significantly reduced striatal activation during unexpected rewards with increasing self-reported depression severity, which complements previous case-control analyses (Kumar et al., Reference Kumar, Waiter, Ahearn, Milders, Reid and Steele2008; Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders, Reid, Hall and Steele2011; Robinson et al., Reference Robinson, Cools, Carlisi, Sahakian and Drevets2012). This is of interest since there is an on-going debate regarding the discreteness of MDD (Hankin et al., Reference Hankin, Fraley, Lahey and Waldman2005; Prisciandaro and Roberts, Reference Prisciandaro and Roberts2009; Ruscio et al., Reference Ruscio, Brown and Meron Ruscio2009). Our findings indicate that striatal RPE is blunted with increasing depression severity, and is not a discontinuous, abrupt process emerging with an MDD diagnosis.

A recent study suggested that earlier findings regarding altered RPE signals in MDD might have been due to medication use (Rothkirch et al., Reference Rothkirch, Tonn, Kohler and Sterzer2017). However, this cannot explain the dimensional association with depressive symptoms we found in our non-help-seeking and unmedicated sample. Since MDD is a very heterogeneous disorder (Sharpley and Bitsika, Reference Sharpley and Bitsika2013; Zimmerman et al., Reference Zimmerman, Ellison, Young, Chelminski and Dalrymple2015) and different symptoms are present during different stages of the disorder (Rakofsky et al., Reference Rakofsky, Schettler, Kinkead, Frank, Judd, Kupfer, Rush, Thase, Yonkers and Rapaport2013), it is possible that clinical heterogeneity is partially responsible for inconsistent findings. We, therefore, advocate for better sample characterization and report sample averages per symptom in the supplementary materials.

It is important to note that we found a significant association between striatal RPE signal and depressive symptoms, only when the latter was assessed with a self-report measure (IDS-SR) and not with an interview-administered scale (MADRS). This discrepancy likely stems from the fact that self-report and interview assessments in general measure slightly different things (Uher et al., Reference Uher, Farmer, Maier, Rietschel, Hauser, Marusic, Mors, Elkin, Williamson, Schmael, Henigsberg, Perez, Mendlewicz, Janzing, Zobel, Skibinska, Kozel, Stamp, Bajs, Placentino, Barreto, McGuffin and Aitchison2007; Fantino and Moore, Reference Fantino and Moore2009; Uher et al., Reference Uher, Perlis, Placentino, Dernovšek, Henigsberg, Mors, Maier, McGuffin and Farmer2012). However, the scales also differ in their contents. For example, whereas the MADRS includes only two items related to the Positive Valence System (PVS; ‘Inability to feel’ and ‘Lassitude’), the IDS-SR assesses similar concepts in four items (‘Response of mood to good/desired events’, ‘General interest’, ‘Energy level’, ‘Capacity for pleasure/enjoyment’). The latter might, therefore, be more sensitive to variations in specific sub-domains of PVS functioning. Future research should include measures developed to specifically measure different PVS domains [e.g., the Temporal Experience of Pleasure Scale (Gard et al., Reference Gard, Gard, Kring and John2006)].

Lastly, the reduced striatal activation during unexpected rewards with increasing self-reported depression severity was not accompanied by a decreased average learning performance. That is, average optimal choice accuracy was not associated with depression severity. This discrepancy in associations of neural and behavioural data of the same task could be due to several reasons. One option we explored is that individuals with more symptoms of depression (and reduced neural PE-related striatal activation) learn slower, but that the task was long enough for these individuals to ‘catch up’. Indeed, a post-hoc analysis examining whether the cumulative choice accuracy (i.e. per trial, the percentage of optimal choices of all trials up to that trial) differs for different depression levels resulted in a significant quadratic effect (p < 0.001) characterized by a less steep (quadratic) learning curve for increasing depression severity (online Supplementary Fig. 4). Again, this effect was only visible for IDS-SR and not for MADRS scores.

Striatal RPE and a measure of real-life reward-related behaviour

The current study fits within the relatively recent approach to combine neuroimaging techniques with real-world psychological experiences. However, it is the first to examine a computational model-based brain signal in this context. Previous work on reward-processing in depression showed that striatal activation during anticipation and receipt of rewards was associated with a daily life measure of PA in a group of depressed adolescents (Forbes et al., Reference Forbes, Hariri, Martin, Silk, Moyles, Fisher, Brown, Ryan, Birmaher, Axelson and Dahl2009). This finding was replicated in healthy controls (Forbes et al., Reference Forbes, Ryan, Phillips, Manuck, Worthman, Moyles, Tarr, Sciarrillo and Dahl2010). These reports added significant ecological validity to other imaging studies reporting reduced striatal activity during reward anticipation (Smoski et al., Reference Smoski, Felder, Bizzell, Green, Ernst, Lynch and Dichter2009; Gotlib et al., Reference Gotlib, Hamilton, Cooney, Singh, Henry and Joormann2010; Olino et al., Reference Olino, McMakin, Morgan, Silk, Birmaher, Axelson, Williamson, Dahl, Ryan and Forbes2014) and reward receipt (McCabe et al., Reference McCabe, Cowen and Harmer2009; Pizzagalli et al., Reference Pizzagalli, Holmes, Dillon, Goetz, Birk, Ryan, Dougherty, Iosifescu, Rauch and Fava2009; McCabe et al., Reference McCabe, Woffindale, Harmer and Cowen2012; Sharp et al., Reference Sharp, Kim, Herman, Pane, Reuter and Strathearn2014) in individuals, and offspring of individuals, currently or previously diagnosed with MDD.

Recent research suggests that, in the context of mood-related psychopathology, neural RPE processing might be a relevant measure in addition to these more static reward-processing measures (Rutledge et al., Reference Rutledge, Skandali, Dayan and Dolan2014; Eldar et al., Reference Eldar, Rutledge, Dolan and Niv2016). By investigating a dynamic ESM measure, i.e., how two variables are related over time, we were able to gain insight into which real-life processes are related to the striatal RPE signal. Evolutionary theories pose that goals for behaviour are specified by reward and punishment evaluation (Rolls, Reference Rolls1999), hence forming the basis for flexible (approach and avoidance) behaviours. The assumption is that flexible approach behaviour requires learning to recognize stimuli signalling potential rewards, followed by a cost-benefit analysis, possibly leading to a change in behaviour. Our ESM data show that, on average, people do report taking more pleasure in their activity when earlier they reported looking forward to a future situation.

However, in our sample, this was more so the case in individuals whose striatal activation more closely followed the model-derived RPE signal. Interestingly, the relevant interaction was only found with a quadratic association between daily life reward anticipation and activity pleasantness. This was characterized by lower striatal PE-related activity being associated with more decoupling between the two constructs only at higher levels of reward anticipation. This could indicate that these individuals might be less able to change their behaviour following the recognition of a cue highly predictive of reward, which would be in line with experimental findings by Sherdell et al. (Reference Sherdell, Waugh and Gotlib2012) indicating that liking and motivation are dissociated in individuals with MDD. Another option for this quadratic effect could be that these individuals are limited by reduced reward sensitivity and thus, even though they might have changed their behaviour following the highly predictive cues, they were not able to like the activity as much as individuals with less blunted striatal PE errors. Since mood affects reward sensitivity (Eldar and Niv, Reference Eldar and Niv2015), the role of mood in the brain-daily life associations reported in the current paper will have to be investigated more closely in the future.

Gaining more insight into real-life dynamics could ultimately lead to the refinement of psychotherapeutic interventions aiming at normalizing PVS dysfunction, [e.g., Craske et al. (Reference Craske, Meuret, Ritz, Treanor and Dour2016)]. Our results suggest that a focus on increasing the coupling of reward anticipation and engagement in pleasing activities might be a relevant target, for example with the use of experience sampling interventions. On an individualized basis, individuals could be made aware of what disrupts this coupling by reviewing, with the help of a professional, their own data after a week of ESM (similar to Kramer et al., Reference Kramer, Simons, Hartmann, Menne-Lothmann, Viechtbauer, Peeters, Schruers, Bemmel, Myin-Germeys, Delespaul, Os and Wichers2014). More advanced technology could additionally be used to provide real-life prompts to teach people to reduce this decoupling, for example by providing motivating messages following the report of reward anticipation.

Limitations

The results should be interpreted in the context of certain limitations. First, the current ESM design could not ascertain whether anticipated (at t-1) and reported (at t) pleasure referred to the same activities/situations [as was done in (Wu et al., Reference Wu, Mata, Furman, Whitmer, Gotlib and Thompson2016)]. This was a conscious choice as otherwise we would influence the participant's mood and/or behaviour (asking whether one actually performed what one was anticipating could make one aware of the failure, or change behaviour in order to avoid this failure). As our goal was to investigate whether, in general, an increase in activity pleasantness is preceded by an increase in reward anticipation this issue did not affect our aims.

Second, even though studies show that depression might be dimensional in nature (Liu, Reference Liu2016), it cannot be assumed that our ESM-fMRI association results generalize to clinical samples. Additionally, sub-threshold symptoms are known to be diffuse (van Os, Reference van Os2013), hence sub-threshold comorbidities are a concern.

Third, the current probabilistic RL task was adapted from a purely behavioural version. It, therefore, has several shortcomings, including the fact that the timings of the response and feedback overlapped, possibly confounding the RPE signal with motor-related activity. However, the pattern of whole-brain RPE results was comparable with previous meta-analyses (Garrison et al., Reference Garrison, Erdeniz and Done2013; Chase et al., Reference Chase, Kumar, Eickhoff and Dombrovski2015), which provides reassurance. In addition, after a win or a loss the total amount of winnings was updated (so that people could keep track of their progress), but only at the start of the next trial. This prevented us from analysing the neural correlates of model-based expected value – which might be relevant in MDD (Gradin et al., Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders, Reid, Hall and Steele2011) – as these signals may be confounded by the effects of feedback.

Fourth, the cut-off for performing at chance level was set at 55%, which is an arbitrary limit. This cut-off was decided a priori to ensure the inclusion of compliant participants. Only three participants had accuracy scores between 50 and 55%, and their inclusion in the analyses did not affect the significance of the results. Additionally, the task was rather long (~24 min) and was part of a larger protocol including additional tasks. Hence, it is unclear whether participants performing below chance did so because of alterations in neural RPE mechanisms, fatigue or general reluctance to comply with instructions in order to finish more quickly. Owing to this design, we had to exclude a substantial number of participants due to potential task non-compliance. However, included and excluded participants did not differ in demographic or clinical variables. Future studies would benefit from including a post-scan questionnaire probing how motivated participants felt during the scan-session.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0033291718003446

Author ORCIDs

Jindra M. Bakker, Liesbet Goossens, Poornima Kumar, Iris M.J. Lange, Stijn Michielse, Jojanneke A. Bastiaansen, Jim van Os, Inez Myin-Germeys, Diego A. Pizzagalli, Marieke Wichers 0000-0002-5270-0581, 0000-0003-0368-2732, 0000-0002-9548-3281, 0000-0002-6251-8978, 0000-003-3930-8646, 0000-0002-7865-5694, 0000-0002-7245-1586, 0000-0002-3731-4930, 0000-0002-7772-1143,0000-0001-5024-9064.

Acknowledgements

This research was supported by grants from the Weijerhorst Foundation (J.v.O.); the Brain Foundation of the Netherlands (M.W., grant number 2012(1)-03); and a Boehringer Ingelheim Foundation travel grant (J.M.B., 2016). DAP was partially supported by R37 MH068376 from the National Institute of Mental Health (NIMH). PK was partially supported by a NARSAD Young Investigator award and by a R21MH105775 from NIMH.

Conflict of interest

Over the past 3 years, Dr Pizzagalli has received consulting fees from Akili Interactive Labs, BlackThorn Therapeutics, Boehringer Ingelheim, Pfizer and Posit Science, for activities unrelated to the current research. All other authors report no biomedical financial interests.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Footnotes

*

These authors contributed equally to this work.

The notes appear after the main text.

1 One participant developed depressive symptoms during participation in the study. The SMARTSCAN protocol had a mirrored design (two similar measurement moments). Hence, only data of this participant of the second-time point were included in the current analyses as part of the m/m-D group.

2 Activity_Pleasantness (t)ij = α 0i + β 1i Reward_Anticipation (t 1)ij + β 2i Activity_Pleasantness (t−1)ij + ε ij

3 Assuming and estimating an AR structure allowing for unequally spaced time values does not affect the significance of the results.

4 Since IDS-SR scores were collected at the second measurement, only data of 120 participants were available.

References

Aickin, M and Gensler, H (1996) Adjusting for multiple testing when reporting research results: the Bonferroni vs Holm methods. American Journal of Public Health 86, 726728.Google Scholar
Bakker, JM, Goossens, L, Lange, I, Michielse, S, Schruers, K, Lieverse, R, Marcelis, M, van Amelsvoort, T, van Os, J, Myin-Germeys, I and Wichers, M (2017) Real-life validation of reduced reward processing in emerging adults with depressive symptoms. Journal of Abnormal Psychology 126, 713725.Google Scholar
Bayer, HM and Glimcher, PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129141.Google Scholar
Bylsma, LM, Taylor-Clift, A and Rottenberg, J (2011) Emotional reactivity to daily events in major and minor depression. Journal of Abnormal Psychology 120, 155.Google Scholar
Chase, HW, Kumar, P, Eickhoff, SB and Dombrovski, AY (2015) Reinforcement learning models and their neural correlates: an activation likelihood estimation meta-analysis. Cognitive, Affective, & Behavioral Neuroscience 15, 435459.Google Scholar
Craske, MG, Meuret, AE, Ritz, T, Treanor, M and Dour, HJ (2016) Treatment for anhedonia: a neuroscience driven approach. Depression and Anxiety 33, 927938.Google Scholar
Csikszentmihalyi, M and Larson, R (1987) Validity and reliability of the Experience-Sampling Method. Journal of Nervous and Mental Disease 175, 526536.Google Scholar
Daw, ND (2011) Trial-by-trial data analysis using computational models. In Delgado, MR, Phelps, EA and Robbins, TW (eds), Decision Making, Affect, and Learning: Attention and Performance XXIII. New York: Oxford University Press, pp. 338.Google Scholar
Delespaul, PAEG (1995) Assessing Schizophrenia in Daily Life: The Experience Sampling Method. Maastricht, The Netherlands: Maastricht University Press.Google Scholar
Eldar, E and Niv, Y (2015) Interaction between emotional state and learning underlies mood instability. Nature Communications 6, 6149.Google Scholar
Eldar, E, Rutledge, RB, Dolan, RJ and Niv, Y (2016) Mood as representation of momentum. Trends in Cognitive Sciences 20, 1524.Google Scholar
Fantino, B and Moore, N (2009) The self-reported Montgomery-Åsberg depression rating scale is a useful evaluative tool in major depressive disorder. BMC Psychiatry 9, 26.Google Scholar
Forbes, EE, Hariri, AR, Martin, SL, Silk, JS, Moyles, DL, Fisher, PM, Brown, SM, Ryan, ND, Birmaher, B, Axelson, DA and Dahl, RE (2009) Altered striatal activation predicting real-world positive affect in adolescent major depressive disorder. The American Journal of Psychiatry 166, 6473.Google Scholar
Forbes, EE, Ryan, ND, Phillips, ML, Manuck, SB, Worthman, CM, Moyles, DL, Tarr, JA, Sciarrillo, SR and Dahl, RE (2010) Healthy adolescents’ neural response to reward: associations with puberty, positive affect, and depressive symptoms. Journal of the American Academy of Child & Adolescent Psychiatry 49, 162172.e5.Google Scholar
Gard, DE, Gard, MG, Kring, AM and John, OP (2006) Anticipatory and consummatory components of the experience of pleasure: a scale development study. Journal of Research in Personality 40, 10861102.Google Scholar
Garrison, J, Erdeniz, B and Done, J (2013) Prediction error in reinforcement learning: a meta-analysis of neuroimaging studies. Neuroscience and Biobehavioral Reviews 37, 12971310.Google Scholar
Gold, JM, Waltz, JA, Matveeva, TM, Kasanova, Z, Strauss, GP, Herbener, ES, Collins, AG and Frank, MJ (2012) Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. Archives of General Psychiatry 69, 129138.Google Scholar
Gotlib, IH, Hamilton, JP, Cooney, RE, Singh, MK, Henry, ML and Joormann, J (2010) Neural processing of reward and loss in girls at risk for major depression. Archives of General Psychiatry 67, 380387.Google Scholar
Gradin, VB, Kumar, P, Waiter, G, Ahearn, T, Stickle, C, Milders, M, Reid, I, Hall, J and Steele, JD (2011) Expected value and prediction error abnormalities in depression and schizophrenia. Brain 134, 17511764.Google Scholar
Hamid, AA, Pettibone, JR, Mabrouk, OS, Hetrick, VL, Schmidt, R, Vander Weele, CM, Kennedy, RT, Aragona, BJ and Berke, JD (2016) Mesolimbic dopamine signals the value of work. Nature Neuroscience 19, 117126.Google Scholar
Hankin, BL, Fraley, RC, Lahey, BB and Waldman, ID (2005) Is depression best viewed as a continuum or discrete category? A taxometric analysis of childhood and adolescent depression in a population-based sample. Journal of Abnormal Psychology 114, 96110.Google Scholar
Heller, AS, Fox, AS, Wing, EK, McQuisition, KM, Vack, NJ and Davidson, RJ (2015) The neurodynamics of affect in the laboratory predicts persistence of real-world emotional responses. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience 35, 1050310509.Google Scholar
Holm, S (1979) A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 6570.Google Scholar
Kasanova, Z, Ceccarini, J, Frank, MJ, Amelsvoort, TV, Booij, J, Heinzel, A, Mottaghy, F and Myin-Germeys, I (2017) Striatal dopaminergic modulation of reinforcement learning predicts reward-oriented behavior in daily life. Biological Psychology 127, 19.Google Scholar
Kramer, I, Simons, CJP, Hartmann, JA, Menne-Lothmann, C, Viechtbauer, W, Peeters, F, Schruers, K, Bemmel, AL, Myin-Germeys, I, Delespaul, P, Os, J and Wichers, M (2014) A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 13, 6877.Google Scholar
Kumar, P, Waiter, G, Ahearn, T, Milders, M, Reid, I and Steele, D (2008) Abnormal temporal difference reward-learning signals in major depression. Brain 131, 20842093.Google Scholar
Lewinsohn, PM and Graf, M (1973) Pleasant activities and depression. Journal of Consulting and Clinical Psychology 41, 261.Google Scholar
Liu, RT (2016) Taxometric evidence of a dimensional latent structure for depression in an epidemiological sample of children and adolescents. Psychological Medicine 46, 12651275.Google Scholar
McCabe, C, Cowen, PJ and Harmer, CJ (2009) Neural representation of reward in recovered depressed patients. Psychopharmacology 205, 667677.Google Scholar
McCabe, C, Woffindale, C, Harmer, CJ and Cowen, PJ (2012) Neural processing of reward and punishment in young people at increased familial risk of depression. Biological Psychiatry 72, 588594.Google Scholar
Montgomery, SA and Åsberg, M (1979) A new depression scale designed to be sensitive to change. The British Journal of Psychiatry 134, 382389.Google Scholar
Myin-Germeys, I, Oorschot, M, Collip, D, Lataster, J, Delespau, P and van Os, J (2009) Experience sampling research in psychopathology: opening the black box of daily life. Psychological Medicine 39, 15331547.Google Scholar
O'Doherty, JP, Hampton, A and Kim, H (2007) Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences 1104, 3553.Google Scholar
Olino, TM, McMakin, DL, Morgan, JK, Silk, JS, Birmaher, B, Axelson, DA, Williamson, DE, Dahl, RE, Ryan, ND and Forbes, EE (2014) Reduced reward anticipation in youth at high-risk for unipolar depression: a preliminary study. Developmental Cognitive Neuroscience 8, 5564.Google Scholar
Overbeek, T, Schruers, K and Griez, E (1999) Mini International Neuropsychiatric Interview, Nederlandse Versie 5.0.0. Maastricht, the Netherlands: Maastricht University.Google Scholar
Pessiglione, M, Seymour, B, Flandin, G, Dolan, RJ and Frith, CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 10421045.Google Scholar
Pizzagalli, DA, Holmes, A, Dillon, DG, Goetz, EL, Birk, JL, Ryan, B, Dougherty, DD, Iosifescu, DV, Rauch, SL and Fava, M (2009) Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. The American Journal of Psychiatry 166, 702710.Google Scholar
Prisciandaro, JJ and Roberts, JE (2009) A comparison of the predictive abilities of dimensional and categorical models of unipolar depression in the National Comorbidity Survey. Psychological Medicine 39, 10871096.Google Scholar
Rakofsky, JJ, Schettler, PJ, Kinkead, BL, Frank, E, Judd, LL, Kupfer, DJ, Rush, AJ, Thase, ME, Yonkers, KA and Rapaport, MH (2013) The prevalence and severity of depressive symptoms along the spectrum of unipolar depressive disorders: a post hoc analysis. Journal of Clinical Psychiatry 74, 10841091.Google Scholar
Robinson, OJ, Cools, R, Carlisi, CO, Sahakian, BJ and Drevets, WC (2012) Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive disorder. American Journal of Psychiatry 169, 152159.Google Scholar
Rogers, WH (1993) Sg17: Regression standard errors in clustered samples. Stata Technical Bulletin 13, pp. 1923.Google Scholar
Rolls, ET (1999) The Brain and Emotion. New York, NY: Oxford University Press.Google Scholar
Rothkirch, M, Tonn, J, Kohler, S and Sterzer, P (2017) Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder. Brain 140, 11471157.Google Scholar
Ruscio, J, Brown, TA and Meron Ruscio, A (2009) A taxometric investigation of DSM-IV major depression in a large outpatient sample: interpretable structural results depend on the mode of assessment. Assessment 16, 127144.Google Scholar
Rush, AJ, Gullion, CM, Basco, MR, Jarrett, RB and Trivedi, MH (2009) The Inventory of Depressive Symptomatology (IDS): psychometric properties. Psychological Medicine 26, 477486.Google Scholar
Rutledge, RB, Skandali, N, Dayan, P and Dolan, RJ (2014) A computational and neural model of momentary subjective well-being. Proceedings of the National Academy of Sciences 111, 1225212257.Google Scholar
Schultz, W, Tremblay, L and Hollerman, JR (2000) Reward processing in primate orbitofrontal cortex and basal ganglia. Cerebral Cortex 10, 272283.Google Scholar
Sharp, C, Kim, S, Herman, L, Pane, H, Reuter, T and Strathearn, L (2014) Major depression in mothers predicts reduced ventral striatum activation in adolescent female offspring with and without depression. Journal of Abnormal Psychology 123, 298309.Google Scholar
Sharpley, CF and Bitsika, V (2013) Differences in neurobiological pathways of four “clinical content” subtypes of depression. Behavioural Brain Research 256, 368376.Google Scholar
Sherdell, L, Waugh, CE and Gotlib, IH (2012) Anticipatory pleasure predicts motivation for reward in major depression. Journal of Abnormal Psychology 121, 5160.Google Scholar
Smoski, MJ, Felder, J, Bizzell, J, Green, SR, Ernst, M, Lynch, TR and Dichter, GS (2009) fMRI of alterations in reward selection, anticipation, and feedback in major depressive disorder. Journal of Affective Disorders 118, 6978.Google Scholar
Snijders, T and Bosker, R (1999) Multilevel Analysis: An introduction to Basis and Advanced Multilevel Modeling. London, UK: Sage.Google Scholar
StataCorp (2013). Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.Google Scholar
Sutton, RS and Barto, AG (1998) Reinforcement Learning: An Introduction. Cambridge, MA: MIT press.Google Scholar
Thompson, RJ, Mata, J, Jaeggi, SM, Buschkuehl, M, Jonides, J and Gotlib, IH (2012) The everyday emotional experience of adults with major depressive disorder: examining emotional instability, inertia, and reactivity. Journal of Abnormal Psychology 121, 819829.Google Scholar
Uher, R, Farmer, A, Maier, W, Rietschel, M, Hauser, J, Marusic, A, Mors, O, Elkin, A, Williamson, RJ, Schmael, C, Henigsberg, N, Perez, J, Mendlewicz, J, Janzing, JGE, Zobel, A, Skibinska, M, Kozel, D, Stamp, AS, Bajs, M, Placentino, A, Barreto, M, McGuffin, P and Aitchison, KJ (2007) Measuring depression: comparison and integration of three scales in the GENDEP study. Psychological Medicine 38, 289300.Google Scholar
Uher, R, Perlis, RH, Placentino, A, Dernovšek, MZ, Henigsberg, N, Mors, O, Maier, W, McGuffin, P and Farmer, A (2012) Self-report and clinician-rated measures of depression severity: can one replace the other? Depression and Anxiety 29, 10431049.Google Scholar
van Os, J (2013) The dynamics of subthreshold psychopathology: implications for diagnosis and treatment. American Journal of Psychiatry 170, 695698.Google Scholar
van Roekel, E, Bennik, EC, Bastiaansen, JA, Verhagen, M, Ormel, J, Engels, RCME and Oldehinkel, AJ (2016) Depressive symptoms and the experience of pleasure in daily life: an exploration of associations in early and late adolescence. Journal of Abnormal Child Psychology 44, 9991009.Google Scholar
Wichers, M (2014) The dynamic nature of depression: a new micro-level perspective of mental disorder that meets current challenges. Psychological Medicine 44, 13491360.Google Scholar
Wilson, SJ, Smyth, JM and MacLean, RR (2013) Integrating ecological momentary assessment and functional brain imaging methods: new avenues for studying and treating tobacco dependence. Nicotine & Tobacco Research 16, S102S110.Google Scholar
Wu, H, Mata, J, Furman, DJ, Whitmer, AJ, Gotlib, IH and Thompson, RJ (2016) Anticipatory and consummatory pleasure and displeasure in major depressive disorder: an experience sampling study. Journal of Abnormal Psychology 126, 149159.Google Scholar
Zimmerman, M, Ellison, W, Young, D, Chelminski, I and Dalrymple, K (2015) How many different ways do patients meet the diagnostic criteria for major depressive disorder? Comprehensive Psychiatry 56, 2934.Google Scholar
Figure 0

Fig. 1. Study and task procedures. (a) Study procedure. (b) Task procedure, adapted from Gold et al. (2012). MADRS, Montgomery–Åsberg Depression Rating Scale (Montgomery and Åsberg, 1979); MINI, Mini International Neuropsychiatric Interview (Overbeek et al., 1999); IDS-SR, Inventory of Depressive Symptomatology (Self-Report) (Rush et al., 2009); RL, Reinforcement Learning; fMRI, functional Magnetic Resonance Imaging; ESM, Experience Sampling Method, m/m-D, participants with moderate symptoms of depression; n/l-D, participants with low symptoms of depression. 1One participant only carried the device for 7 days since initially included in another arm of the SMARTSCAN study.

Figure 1

Fig. 2. fMRI results and correlations between RPE and self-reported depressive symptoms. (a) Active clusters during Reward Prediction Error (average over the whole group). (b) Regions of Interest, extracted from Chase et al. (2015). (c) Mean activation per Region of Interest for each participant. (d) Scatterplots of RPE beta weight and IDS-SR total score for the right Nucleus Accumbens and Putamen. L, left; R, right; NAcc, Nucleus Accumbens; RPE, reward prediction error; IDS-SR, Inventory of Depressive Symptomatology (Self-Report).

Figure 2

Table 1. Demographic and clinical descriptive information (a); Information regarding behavioural performance on the Reward Learning task (b); and computational modelling output (c)

Figure 3

Table 2. Associations between mean activation in R NAcc and Putamen and reward-related ESM measures (controlled for age and gender, and the ESM analyses also for overall time of measurement, time of measurement within the day)

Figure 4

Fig. 3. Simple slopes to illustrate significant moderating effects of RPE beta weight of three ROI's on the time-lagged association between Reward anticipation and Activity pleasantness, at three levels of the RPE beta weight (group average, and 2 standard deviations below/above this group average). R, right; L, left; NAcc, Nucleus Accumbens; EMM, estimated marginal mean; WP, within-person; s.d., standard deviation; RPE, Reward Prediction Error; ROI, Region of Interest

Supplementary material: File

Bakker et al. supplementary material

Bakker et al. supplementary material
Download Bakker et al. supplementary material(File)
File 12 MB