Introduction
One cardinal feature of depression is weakened or distorted behavioral effectiveness of reinforcement (reviewed in Eshel & Roiser, Reference Eshel and Roiser2010). Learning theory suggests that the problem may lie in representing unexpected rewards that normally drive appetitive behavior (Bush & Mosteller, Reference Bush and Mosteller1951; Rescorla & Wagner, Reference Rescorla, Wagner, Black and Prokasy1972). A reward's behavioral effectiveness depends not only on its magnitude, but also on the amount of surprise, since surprise results in learning. This surprise is quantified by the reward prediction error – the difference between the actually received and expected reward. In every episode of learning (trial, t), expected reward is updated from its previous value by the prediction error [equation (1), simplified for illustration]:
$$\left. \eqalign{& {\rm expected \; rewar}{{\rm d}_t} = {\rm expected \; rewar}{{\rm d}_t}_{-1} \cr & + {\rm \alpha} \times {\rm reward \; prediction \; erro}{{\rm r}_t}, \cr & {\rm reward \; prediction \; erro}{{\rm r}_t} = {\rm actual \; rewar}{{\rm d}_t} \cr & \; -{\rm expected \; rewar}{{\rm d}_t}_{-1}, }\right\}$$
where α = learning rate.
Reward expectancy, in turn, controls approach behavior. Variants of this simple learning rule underlie all reinforcement learning models (Sutton & Barto, Reference Sutton and Barto1998). Equation (1) helps understand the asymptotic learning curves observed in operant conditioning (Skinner, Reference Skinner1938; Bush & Mosteller, Reference Bush and Mosteller1951). Early in learning, a rat rewarded for lever presses has low reward expectancy. Early rewards generate large prediction errors, driving a rapid increase in the response rate. Late in learning, reward expectancy becomes high, prediction errors wane, and the response rate stabilizes.
Neural reward prediction error signals are thought to originate in the mesostriatal dopamine pathway (Schultz et al. Reference Schultz, Dayan and Montague1997). They are believed to shape cognitive processes and behavior through synaptic modification of corticostriatothalamic circuits (see Glimcher, Reference Glimcher2011; Chase et al. unpublished data, for a theoretical review and meta-analysis of human imaging studies). Specifically, trial-by-trial learning signals are thought to train sustained, multi-trial prefrontal representations of the reward contingency (Pasupathy & Miller, Reference Pasupathy and Miller2005; Histed et al. Reference Histed, Pasupathy and Miller2009). In human imaging, the degree of co-variation between model-estimated reward prediction errors and blood oxygen-level dependent (BOLD) responses is typically assumed to reflect individual differences in corticostriatothalamic prediction error signals (Kumar et al. Reference Kumar, Waiter, Ahearn, Milders, Reid and Steele2008; Gradin et al. Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders, Reid, Hall and Steele2011; Dombrovski et al. Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). In depression, corticostriatothalamic circuits display altered responses to rewards and punishments in general (Eshel & Roiser, Reference Eshel and Roiser2010) and diminished reward prediction error signals in particular (Kumar et al. Reference Kumar, Waiter, Ahearn, Milders, Reid and Steele2008; Gradin et al. Reference Gradin, Kumar, Waiter, Ahearn, Stickle, Milders, Reid, Hall and Steele2011). Using fMRI, we have recently extended these observations to late-life depression (Dombrovski et al. Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). We also observed that, in depressed individuals, oversensitivity to punishments (Murphy et al. Reference Murphy, Michael, Robbins and Sahakian2003) was inversely related to the strength of corticostriatothalamic reward prediction error signals (Dombrovski et al. Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013).
These observations reveal certain neural correlates of altered reward-guided behavior in depression, yet its specific neural mechanisms remain unclear. One particularly influential account of behavioral disturbance in depression emphasizes failures of top-down control, broadly linked to prefrontal dysfunction (Robbins, Reference Robbins2007). Executive control is impaired in depression (Snyder, Reference Snyder2013), and the co-occurrence of depression and poor executive control forms the basis for the vascular depression hypothesis (Alexopoulos et al. Reference Alexopoulos, Meyers, Young, Campbell, Silbersweig and Charlson1997; Taylor et al. Reference Taylor, Aizenstein and Alexopoulos2013). It attributes depression and poor executive control in old age to prefrontal and cingulate disconnection from subcortical structures by ischemic white-matter lesions (Alexopoulos et al. Reference Alexopoulos, Kiosses, Choi, Murphy and Lim2002, Reference Alexopoulos, Hoptman, Kanellopoulos, Murphy, Lim and Gunning2012; Sneed et al. Reference Sneed, Roose, Keilp, Krishnan, Alexopoulos and Sackeim2007; Aizenstein et al. Reference Aizenstein, Andreescu, Edelman, Cochran, Price, Butters, Karp, Patel and Reynolds2011). This hypothesis is supported by findings of poor treatment response in patients with poor executive control and white-matter hyperintensities (WMH) (Baldwin et al. Reference Baldwin, Jeffries, Jackson, Sutcliffe, Thacker, Scott and Burns2004; Sneed et al. Reference Sneed, Roose, Keilp, Krishnan, Alexopoulos and Sackeim2007; Barch et al. Reference Barch, D'Angelo, Pieper, Wilkins, Welsh-Bohmer, Taylor, Garcia, Gersing, Doraiswamy and Sheline2012). Can the vascular depression hypothesis explain alterations in reward-guided behavior and reward signals seen in late-life depression? Our fMRI study of reward-guided behavior tested the prediction that both depression and poor executive control are related to a disconnection of the prefrontal and cingulate cortices from the striatum and thalamus.
To establish that we possess the appropriate index of corticostriatothalamic integrity for testing the predictions of the vascular depression hypothesis, we assessed whether individual variation in prediction error signals scaled with executive control ability in patients with late-life depression. We were then able to test whether depression and executive control accounted for shared variance in corticostriatothalamic integrity (indexed by prediction error signals), as would be the case if they were both caused by disconnection. Then, to obtain more direct evidence of functional striato-cortical disconnection, we tested the effects of depression and poor executive control on striato-cortical functional connectivity. Next, to examine the role of vascular lesions, we tested whether (i) WMH disrupted corticostriatothalamic responses and striato-cortical functional connectivity and (ii) whether they explained the relationship of these indices with depression and executive control. All analyses were repeated in a subsample with vascular conditions. Finally, we tested whether late-life depression and poor executive control had similar or distinct effects on reward- and punishment-guided behavior in two non-overlapping samples.
Method
Participants and diagnosis of depression (Table 1)
Between 1 January 1 2008 and 31 August 2011, we recruited 47 participants aged ⩾60 years: 31 with major depressive disorder (MDD) and 16 psychiatrically healthy controls. MDD was diagnosed by Structured Clinical Interview for DSM-IV Axis I Disorders (First, Reference First, Gibbon and Williams1995; APA, 2000) (SCID/DSM-IV). Participants with MDD had symptoms of varying severity (Table 1; examined in Sensitivity analyses). To exclude individuals with dementia, all were required to have a MMSE score of ⩾24. Our reported sample is exclusive of elderly with sensory disorders that precluded cognitive testing, limited English, mental retardation, delirium, neurologic disorders including strokes, bipolar disorder, schizophrenia, schizoaffective disorder, exposure to electroconvulsive therapy in the previous 6 months, and conditions precluding an fMRI assessment. This is a subsample of participants described in our earlier report (Dombrovski et al. Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013), exclusive of six participants missing assessments of executive control.
Table 1. Demographic, clinical and cognitive characteristics
a Wechsler Test of Adult Reasoning, standard score.
b Cumulative Illness Rating Scale, adapted for geriatrics.
c Hamilton Rating Scale for Depression, 17-item.
d Antidepressant Treatment History Form, cumulative score.
All participants provided written informed consent. The University of Pittsburgh Institutional Review Board approved the study.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helinski Declaration of 1975, revised in 2008.
Assessments
Cognitive and clinical characterization
Current global cognitive function was assessed with the Dementia Rating Scale (DRS; Mattis, Reference Mattis1988). Depression severity was measured with the 17-item Hamilton Rating Scale for Depression (HAMD; Hamilton, Reference Hamilton1960). Burden of physical illness was assessed with the Cumulative Illness Rating Scale adapted for Geriatrics (CIRS-G; Miller et al. Reference Miller, Paradis, Houck, Mazumdar, Stack, Rifai, Mulsant and Reynolds1992). We measured antidepressant exposure in the current episode of depression using the Antidepressant Treatment History Form (Sackeim, Reference Sackeim2001), based on antidepressant trial duration, dose, and use of augmenting agents. We additionally assessed exposure to sedatives/hypnotics, anticholinergics, and opioids.
Executive control
We assessed executive control with the Executive Interview (EXIT; range 0–50; Royall et al. Reference Royall, Mahurin and Gray1992). The 25 items are administered in rapid succession with minimal instructions to elicit automatic behaviors. Items include modifications of well-known ‘frontal lobe’ tests (number/letter sequencing, Stroop, fluency, go/no-go, and Luria's hand sequences). Higher EXIT scores indicate greater impairment. Crucially, unlike in the probabilistic reversal learning task, no external reinforcement is given.
Probabilistic reversal learning fMRI task (Fig. 1)
On each of 300 trials, participants choose between two stimuli using button presses. In the first 25-trial block, one stimulus has a high probability of reward when chosen (varied between 80% and 87%), and the other, a low probability of reward (13–20%). Thus, even when correctly choosing the high-reward probability stimulus, participants receive occasional misleading or ‘probabilistic’ negative feedback. After every 25 trials, this contingency is reversed, such that the high-reward probability stimulus is assigned a low reward probability and vice versa. There are a total of 12 reversals. On this task, one needs to first learn the identity of the best stimulus and then trade off ‘staying’ with the previously reinforced stimulus despite occasional misleading feedback and ‘switching’ when a true reversal occurs. The tendency to ‘stay’ too long after reversal while ignoring negative feedback leads to perseverative errors. Conversely, the tendency to ‘switch’ after a single misleading punishment results in lose-switches (also known as probabilistic switches), previously linked to depression (Murphy et al. Reference Murphy, Michael, Robbins and Sahakian2003). Spontaneous or win-switch errors occur when the participant abandons a choice rewarded on a preceding trial.
Fig. 1. Probabilistic Learning Task. (a) On each trial, the participant chooses between two stimuli. Following the choice, feedback is displayed: green frame for reward (as shown), red frame for reward omission. A fixation screen follows. Scanning is synchronized with trial onset. (b) One stimulus initially has a high probability of reward and the other, a low probability of reward. This contingency is reversed unbeknownst to the participant every 25 trials, a total of 12 reversals. (c) The reinforcement learning model estimates expected reward and reward prediction error for each trial (shown for 50 trials), using the participant's reinforcement history and choices. (Simplified learning rule provided for illustration; the model is described in full in Dombrovski et al. Reference Dombrovski, Clark, Siegle, Butters, Ichikawa, Sahakian and Szanto2013.)
Imaging
Thirty-two 3-mm slices were acquired parallel to the AC–PC line using a reverse EPI pulse sequence [one of two 3 T Siemens Trio magnets, T2*-weighted images depicting BOLD contrast (activity and connectivity indices did not vary between scanners: t 51 < 0.73, p > 0.47); TR = 1670 ms, TE = 29 ms, FOV = 20 cm, flip = 75], yielding 954 whole-brain volumes per participant. Following slice-time correction, motion correction (AFNI 3dVolReg) and linear detrending to eliminate scanner drift, data were converted to percent-change, temporally smoothed, cross-registered to the Colin-27 Montreal Neurological Institute template (AIR's 32-parameter nonlinear warp), and spatially smoothed (6 mm FWHM).
Estimation of prediction error signals using reinforcement learning
We estimated prediction error signals from participants’ reinforcement history and behavior on the probabilistic reversal learning task, using a modified Rescorla–Wagner reinforcement learning (RL) model (Dombrovski et al. Reference Dombrovski, Clark, Siegle, Butters, Ichikawa, Sahakian and Szanto2010, Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013) (Fig. 1).
Components of BOLD response tracking prediction errors
We estimated the empirical hemodynamic response function (HRF) in the control group using the independent switch v. stay contrast in the ventrolateral prefrontal cortex (vlPFC). It was then convolved with the positive prediction error estimates from the RL model for each subject. Voxelwise BOLD signal was regressed on these estimates using AFNI's 3dDeconvolve. We used AFNI's 3dTtest to map prediction error responses in the group of healthy controls. We applied a voxelwise threshold of p < 0.001 and a whole-brain cluster-size threshold (Forman et al. Reference Forman, Cohen, Fitzgerald, Eddy, Mintun and Noll1995; Cox, Reference Cox1996). The resulting thresholded group map served as a functional mask for extracting mean responses to positive prediction error in each region for each participant.
WMH
T2-weighted FLAIR data (TR = 9002 ms, TEeffective = 56 ms, TI = 2200 ms; N excitations = 1) were obtained on 27/47 participants, demographically and clinically similar to the rest of the group (p < 0.19). WMH volumes were estimated with an automated localization and segmentation method (Wu et al. Reference Wu, Rosano, Butters, Whyte, Nable, Crooks, Meltzer, Reynolds and Aizenstein2006) and normalized to each participant's overall brain volume (Aizenstein et al. Reference Aizenstein, Andreescu, Edelman, Cochran, Price, Butters, Karp, Patel and Reynolds2011).
Replication of behavioral effects in a second sample
We used data from our earlier behavioral study of late-life depression (Dombrovski et al. Reference Dombrovski, Clark, Siegle, Butters, Ichikawa, Sahakian and Szanto2010), which used identical recruitment/assessment procedures and an 80-trial probabilistic reversal learning task with a single reversal. To ensure independence, we excluded participants who subsequently took part in the imaging protocol, leaving 39 with major depression and 12 non-psychiatric controls. The groups were similar in race (white: controls, 12/12; depressed, 32/39) and education [controls: mean(s.d.) = 15.6 (2.7) years; depressed: 14.1(3.0) years], but the depressed participants were older [controls: 65.7 (5.3) years; depressed: 70.9 (8.1) years, p = 0.045] and more likely to be female (controls: 2/12; depressed: 23/39). Thus, in our analysis, we co-varied for age and gender as well as history of suicide attempts, which was previously shown to predict task performance (Dombrovski et al. Reference Dombrovski, Clark, Siegle, Butters, Ichikawa, Sahakian and Szanto2010).
Statistical analysis
We performed five separate families of tests: (1) magnitude of neural responses to unpredicted rewards – positive prediction errors – and effects of executive control and depression on these responses; (2) corticostriatal functional connectivity modulated by unexpected rewards for each participant and examined the effects of depression and executive control on connectivity indices; (3) effects of WMH on corticostriatal responses to unpredicted rewards; (4) effects of executive control and depression on behavior in the present study; (5) replication of behavioral findings using data from our earlier study.
Effects of cognitive control and depression on BOLD response tracking prediction errors
Applying positive prediction error group maps from non-psychiatric controls as a functional mask, we extracted BOLD responses to positive prediction error for each subject and region.
Responses to positive prediction error were strongly inter-correlated across regions of the corticostriatothalamic circuit, revealing a single underlying factor (Dombrovski et al. Reference Dombrovski, Szanto, Clark, Reynolds and Siegle2013). Thus, to reduce dimensionality and control type I error, we examined summary activations across the entire network responsive to prediction error instead of activations in single regions. We followed up by examining separate clusters, to check whether the effects of executive dysfunction were spatially uniform. Our corticostriatothalamic network mask was defined in healthy controls, and to pre-empt concerns about circularity, we first examined the effect of executive control on prediction error signals in the independent group of depressed patients. After independently establishing this effect, we examined effects of depression and executive control in the combined group. We included age, gender, and education as covariates in our main analyses in order to de-noise the relationship between the independent (executive dysfunction, depression) and dependent variables (BOLD). To ensure that this relationship was not an artifact of partialing out common variance between independent variables and covariates (Miller & Chapman, Reference Miller and Chapman2001), we verified that the results were similar without covariates.
Striatocortical functional connectivity in the context of unexpected rewards
We aimed to test the effects of poor executive control and depression on the training of sustained prefrontal representations by trial-by-trial striatal signals (Pasupathy & Miller, Reference Pasupathy and Miller2005). We used the framework of generalized psychophysiological interaction analysis (McLaren et al. Reference McLaren, Ries, Xu and Johnson2012) to estimate striatocortical functional connectivity specifically associated with unexpected rewards as opposed to any surprising outcomes. We defined the striatal seed as the region responsive to positive prediction errors in the entire sample within the Talairach Daemon striatum (p voxelwise < 0.001). Using AFNI, we extracted the time-course for the striatal seed and deconvolved the HRF from it. We then computed an interaction between the z-scored ‘neural’ striatal time course and the positive and negative prediction (PE+ and PE-) error regressors. The single-subject model, implemented using AFNI's 3dDeconvolve included the PE+*striatum and PE-*striatum interactions, as well as the following nuisance regressors: main effects of PE+ and PE-, striatal seed time-course, motor action (left or right), trial stages (choice, feedback, wait), and the six motion regressors. We then computed a map of the difference between the regression coefficients for the PE+*striatum and PE-*striatum interactions, which was taken to the group analyses.
Effects of WMH
We examined effects of WMH burden on BOLD responses and functional connectivity. We also tested whether WMH burden explained the effects of depression and executive control, as the vascular depression hypothesis would predict.
Effects of cognitive control and depression on behavior
We used linear models with EXIT scores and depression group status as independent variables, age, gender and education as covariates, and the following behavioral indices as dependent variables: lose-switches, spontaneous switches, and perseverative errors. When necessary, we tested the robustness of results to deviations from normality using a generalized linear model.
Replication of behavioral effects
We tested the effect of executive control on reversal learning performance, including depression, gender, and history of suicide attempt as factors and age and education as covariates.
Results
Executive control and neural response to unpredicted rewards (Fig. 2)
In depressed patients, poor executive control predicted blunted responses to unpredicted rewards in the network independently identified in controls, encompassing a frontoparietal circuit, a cingulo-opercular circuit, dorsomedial prefrontal cortex, thalamus, and striatum (F 1,26 = 13.4, p = 0.001, η 2 p = 0.34, controlling for age, gender, and education). In the combined group of patients and controls, lower neural response to unexpected rewards in this network was predicted by poorer executive control (F 1,41 = 15.7, p < 0.001, η 2 p = 0.29) and depression (F 1,42 = 17.4, p < 0.001, η 2 p = 0.31), controlling for age (η 2 p = 0.02), gender (η 2 p = 0.08), and education (η 2 p < 0.01). Contrary to the prediction of the vascular hypothesis, effects of executive control and depression were not shared, but rather were simply additive [model including depression (η 2 p = 0.34), age, gender, education: R 2 = 0.35; model including executive control (η 2 p = 0.29), depression (η 2 p = 0.31), age, gender, education: R 2 = 0.57]. Results were similar without covariates (Fig. 1). The disruptive effects of depression and poor executive control on responses to unpredicted rewards were not anatomically selective, with no significant differences between prefrontal, thalamostriatal, paralimbic (precuneus), parietal, and temporal regions (p > 0.33, η 2 p ⩽ .11).
Fig. 2. Corticostriatothalamic responses to positive prediction error and executive control. (a) Positive prediction error functional network masks were independently derived in controls. (b) In depressed participants, weak corticostriatothalamic response to prediction errors was related to poor executive control as captured by the Executive Interview.
Effects on striatocortical connectivity modulated by unpredicted rewards (Fig. 3)
In healthy controls, unpredicted rewards, contrasted with unpredicted punishments, triggered an increase in functional connectivity between striatum and left operculoinsular cortex, right vlPFC, bilateral superior temporal gyrus, bilateral thalamus, and posterior cingulate cortex (Fig. 2a ). In a ROI analysis, functional connectivity in this circuit was less strongly modulated by unpredicted rewards in patients with depression than in controls (F 1,42 = 10.4, p = 0.002, η 2 p = 0.20), but was not affected by poor executive control (p = 0.85, η 2 p < 0.01; Fig. 2b ). Since the ROIs were identified in controls, we also performed an unbiased whole-brain group contrast. Similarly, an effect of depression was seen for striatal connectivity with the left operculoinsular cortex (Fig. 2c ), with no significant effects of poor executive control.
Fig. 3. Corticostriatal functional connectivity in the context of unexpected rewards. (a) Positive v. negative prediction errors differentially enhanced functional connectivity between the striatum and left operculoinsular, right ventrolateral prefrontal cortex, bilateral superior temporal gyrus, and bilateral thalamus. (b) In a region-of-interest analysis, this cortico-thalamic network was less strongly modulated in depressed patients compared to controls. There were no effects of executive control. (c) An independent whole-brain contrast mapped the effect of depression to the left operculoinsular cortex. There were no significant effects of executive control.
Effects of depression and executive control in a subsample with vascular conditions
To test the predictions of the vascular depression hypothesis in a group at highest risk for neurovascular injury, we limited the sample to those with clinically significant vascular disease. These 37 participants [25 depressed, 12 controls; 22 female; mean (s.d.) age 68 (7) years] scored ⩾2 on the CIRS-G, Vascular subscale [examples: one symptom of atherosclerotic disease (angina, claudication, bruit, amaurosis fugax, absent pedal pulses) or need for daily antihypertensives or aortic aneurysm <4 cm (Miller et al. Reference Miller, Paradis, Houck, Mazumdar, Stack, Rifai, Mulsant and Reynolds1992)]. The results remained qualitatively unchanged. Weaker corticostriatothalamic response to unexpected rewards was predicted by poorer executive control (F 1,30 = 20.6, p < 0.001, η 2 p = 0.41) and depression (F 1,30 = 14.3, p < 0.001, η 2 p = 0.32; age: η 2 p = 0.04; gender: η 2 p = 0.08; education: η 2 p = 0.08). Further, striatocortical functional connectivity was weaker in patients with depression than in controls (F 1,30 = 5.8, p = 0.022, η 2 p = 0.16), but was unaffected by poor executive control (p = 0.61, η 2 p < 0.01).
WMH
WMH burden predicted blunted corticostriatal responses to unpredicted rewards (F 1,22 = 8.7, p = 0.007, η 2 p = 0.28), after controlling for predictors reported above. WMH burden failed to explain the effects of depression (effect of depression controlling for WMH: η 2 p = 0.51; without WMH: η p 2 = 0.40) or poor executive control (controlling for WMH: η 2 p = 0.35; without WMH: η 2 p = 0.40). WMH burden explained no significant variance in striatocortical connectivity (p > 0.29, η 2 p < 0.05). Finally, in the subsample of participants with vascular disease, WMH burden predicted a weaker neurohemodynamic response to unexpected rewards (F 1,19 = 5.5, p = 0.030, η 2 p = 0.23), but did not explain the effects of executive control (F 1,19 = 10.0, p = 0.005, η 2 p = 0.35) or depression (F 1,19 = 11.0, p = 0.004, η 2 p = 0.37).
WMH location
We verified that WMH located in tracts terminating in prefrontal and cingulate cortex (cingulum bundle, superior longitudinal fasciculus, uncinate fasciculus, and anterior thalamic radiation) exerted similar effects on corticostriatothalamic prediction error signals as the total WMH burden. WMH in each of these tracts had disruptive effects on prediction error signals, but did not account for effects of depression and executive dysfunction, which remained undiminished (data available upon request).
Behavioral signatures of poor executive control and depression
Individuals with poor executive control failed to persist with rewarded actions. Meanwhile, participants with depression over-reacted to single misleading punishments. Specifically, poor executive control predicted spontaneous switches (EXIT: F 1,41 = 10.1, p = 0.003, η 2 p = 0.20; depression: F 1,41 = 0.5, p = 0.50, η 2 p = 0.01; controlling for age, gender, and education). Meanwhile, depression predicted lose-switches (depression: F 1,41 = 6.5, p = 0.014, η 2 p = 0.14; EXIT: F 1,41 = 1.8, p = 0.19, η 2 p = 0.04; controlling for age, gender, and education). Neither executive control nor depression affected perseverative errors (p > 0.31, η 2 p < 0.03). A repeated-measures analysis confirmed a double dissociation of these effects: poor executive control was selectively associated with spontaneous switches (EXIT*error type: F 1,41 = 7.9, p = 0.007, η 2 p = 0.16) and depression was selectively associated with lose-switches (depression*error type: F 1,41 = 4.2, p = 0.048, η 2 p = 0.09, both controlling for age, gender, and education). Only executive control (F 1,40 = 4.4, p = 0.043, η 2 p = 0.10) but not depression (F 1,41 = 1.8, p = 0.18, η 2 p = 0.04) explained significant variance in the overall number of correct responses. Finally, WMH burden did not explain any additional variance in behavioral indices (p > 0.24, η 2 p < 0.06).
Replication of behavioral effects
In the replication sample, participants with poor cognitive control similarly failed to persist with rewarded actions on a single-reversal task. Poor executive control predicted spontaneous switches during acquisition (F 1,40 = 6.5, p = 0.014, η 2 p = 0.14; controlling for the effects of age, gender, education, and history of depression and attempted suicide), but not lose-switches or perseverative errors (p > 0.39, η 2 p < 0.02). There were not enough spontaneous switches in the post-reversal phase for analysis. As we have reported, depression was not reliably related to lose-switches in that study (Dombrovski et al. Reference Dombrovski, Clark, Siegle, Butters, Ichikawa, Sahakian and Szanto2010). As in the fMRI study, poor executive control selectively predicted the encoding of rewards (spontaneous switches) rather than sensitivity to single misleading punishments (lose-switches; EXIT*error type: F 1,47 = 6.2, p = 0.016, η 2 p = 0.12).
Sensitivity analyses
Co-morbid conditions
Depressed participants with lifetime history of substance use disorders tended to display blunted corticostriatothalamic positive prediction error responses (F 1,20 = 4.1, p = 0.058, η 2 p = 0.17). This effect was additive and did not modify that of executive dysfunction (F 1,20 = 7.4, p = 0.013, η 2 p = 0.27). Lifetime history of anxiety disorders had no significant impact (F 1,20 = 0.2, p = 0.69; all controlling for depression, age, gender, and education).
Burden of physical illness
While depressed participants suffered from a higher burden of physical illness (Table 1), it had no effect on corticostriatothalamic positive prediction error responses (F 1,39 = 0.3, p = 0.57, η 2 p < 0.01), while the effects of executive control (F 1,39 = 21.9, p < 0.001, η p 2 = 0.36) and depression remained unchanged (F 1,39 = 22.0, p < 0.001, η 2 p = 0.36).
Age at first episode of depression
One might argue that prefrontal disconnection associated with poor executive control better accounts for cases of late-onset depression. However, later age of onset was not associated with poorer executive control in our sample (r = 0.05, p = 0.78) and did not modify the effects of executive control on prediction error signals (EXIT: F 1,24 = 13.0, p = 0.001, η 2 p = 0.35; age at first episode: p = 0.81, η 2 p < 0.01) or on spontaneous switch errors (EXIT: F 1,24 = 5.1, p = 0.034, η 2 p = 0.17; age at first episode: p = 0.72, η 2 p < 0.01). At the same time, patients with earlier onset of depression demonstrated a greater sensitivity to single misleading punishments indexed by lose-switches, while there were still no significant effects of executive control (age at first episode: F 1,24 = 5.0, p = 0.035, η 2 p = 0.17; EXIT: p = 0.27, η p 2 = 0.05).
Medication exposure
The relationship between executive dysfunction and corticostriatothalamic positive prediction error responses was unaffected by exposure to opioids, sedatives or anticholinergics (EXIT: F 1,23 = 12.1, p = 0.002, η 2 p = 0.35; exposure to any medication class: p > 0.58, η 2 p < 0.02). The same was true for the cumulative strength of antidepressant exposure measured by the Antidepressant Treatment History Form (EXIT: F 1,17 = 9.0, p = 0.008, η 2 p = 0.35; ATHF: p = 0.73 η 2 p < 0.01).
Potential experimental and analytic confounds
It was important to verify that the relationship between executive control and corticostriatothalamic prediction error signals was not driven by participants who were not on task or whose behavior could not be fit with the reinforcement learning model. Neither the median response time nor its variation coefficient had an impact on corticostriatothalamic positive prediction error signals (p > 0.12, η 2 p < 0.06) and did not diminish the effect of executive control. Participants with poor executive control had significantly poorer fits of the reinforcement learning model (F 1,40 = 7.8, p = 0.008, η 2 p = 0.16). However, controlling for model fits (F 1,39 = 1.0, p = 0.33, η 2 p = 0.03) did not diminish the effect of executive control (F 1,39 = 13.7, p = 0.001, η 2 p = 0.26).
Distribution of spontaneous switch errors
The distribution of spontaneous switch errors was zero-inflated. However, an analysis using a generalized linear model with a negative binomial log-link confirmed that the relationship between poor executive control and spontaneous switch errors was robust to this violation of normality (EXIT: Wald χ2 = 8.1, p = 0.005). Distributions of other dependent variables did not significantly deviate from the Gaussian.
Conclusions
We tested a prediction of the vascular depression hypothesis that prefrontal/cingulate disconnection from subcortical nuclei would account for neural correlates of depression and poor executive control during reward-guided behavior. Overall, this prediction was not confirmed. First, poor executive control was associated with disrupted corticostriatothalamic responses to unpredicted rewards independently of the effects of depression. Second, depression, but not poor executive control, was associated with disrupted striatocortical differential connectivity to unpredicted rewards v. unpredicted punishments. Third, while WMH predicted blunted corticostriatothalamic responses, they failed to account for the effects of depression or executive control. Fourth, poor executive control and depression had distinct behavioral signatures. Poor executive control predicted spontaneous switches, presumably reflecting a basic disruption in contingency encoding. Depression predicted oversensitivity to misleading punishments, presumably because preceding rewards were not robustly encoded. Below, we briefly explicate the neurobiological framework for interpreting the results. We then discuss implications for the vascular depression hypothesis and alternative models of depression.
Corticostriatothalamic mechanisms of reward learning and the vascular depression hypothesis
Reward prediction errors were theoretically predicted (Montague et al. Reference Montague, Dayan and Sejnowski1996) and empirically shown (Schultz et al. Reference Schultz, Dayan and Montague1997) to originate in the mesostriatal dopaminergic pathway. Prediction errors are thought to influence behavior through synaptic modification of corticostriatothalamic circuits. Prediction error signals during Pavlovian learning were first described in the human ventral striatum (O'Doherty et al. Reference O'Doherty, Dayan, Friston, Critchley and Dolan2003), which forms reentrant loops with the medial orbitofrontal and anterior cingulate cortices (Middleton & Strick, Reference Middleton and Strick2000) classically implicated in reward-guided behavior. During the learning of action-reward (instrumental) contingencies, prediction error signals are found in the dorsal striatum connected to associative prefrontal cortices (O'Doherty et al. Reference O'Doherty, Dayan, Schultz, Deichmann, Friston and Dolan2004).
In this study, we used the strength of instrumental prediction error signals in the striatum, thalamus, cingulo-opercular and frontoparietal networks as an indicator of their functional integrity. Based on what we know about the corticostriatothalamic reward systems from lesion, electrophysiological, and imaging studies, our functional connectivity and behavioral findings suggest that the two syndromes are paralleled by disruptions at different levels of corticostriatothalamic circuits. Alterations in reward signals associated with executive dysfunction, not surprisingly, fit the pattern of prefrontal dysfunction and, possibly, disconnection. The behavior of individuals with impaired executive control bears a close resemblance to that of patients with bilateral prefrontal lesions (Hornak et al. Reference Hornak, O'Doherty, Bramham, Rolls, Morris, Bullock and Polkey2004) and, interestingly, to that of primates with lesions to white-matter prefrontal connections (Rudebeck et al. Reference Rudebeck, Saunders, Prescott, Chau and Murray2013). Our findings parallel those of a study showing that fluid intelligence (a proxy for executive control) was correlated with striatal prediction error signals independently of dopamine synthesis capacity (Schlagenhauf et al. Reference Schlagenhauf, Rapp, Huys, Beck, Wustenberg, Deserno, Buchholz, Kalbitzer, Buchert, Bauer, Kienast, Cumming, Plotkin, Kumakura, Grace, Dolan and Heinz2012). Both executive control and fluid intelligence depend on the functional integrity and intact outputs of the frontoparietal network (Cole et al. Reference Cole, Yarkoni, Repovs, Anticevic and Braver2012). Meanwhile, in depression, it is the ascending transmission of mesostriatal reward prediction errors that appears to be affected. This is indicated by diminished functional striatocortical connectivity in response to positive v. negative prediction errors. These data are consistent with a disruption of the mesostriatal output through the pallidum and thalamus to the associative prefrontal neocortex. This interpretation is in agreement with electrophysiological evidence that, as contingencies are encoded, striatal responses to reinforcement emerge and asymptote before the prefrontal responses, which mediate behavioral improvement (Pasupathy & Miller, Reference Pasupathy and Miller2005). Thus, reduced functional striatocortical connectivity in depression presumably reflects disrupted training of tonic prefrontal representations by phasic striatal signals. Supporting this idea, the oversensitivity of depressed patients to misleading punishments resembles the behavior of individuals with a genetically reduced expression of striatal D2 receptors, which conceivably attenuates mesostriatal prediction error signals (Jocham et al. Reference Jocham, Klein, Neumann, von Cramon, Reuter and Ullsperger2009). The notion that ascending dopaminergic signals are deficient in depression is further supported by the recent finding of an increased burden of brainstem Lewy bodies and tangles and lower density of ventral tegmental area dopaminergic neurons in older people with depressive symptoms (Wilson et al. Reference Wilson, Nag, Boyle, Hizel, Yu, Buchman, Shah, Schneider, Arnold and Bennett2013).
The high co-occurrence of depression and poor cognitive control nevertheless demands an alternative explanation. We would argue that impaired executive control prevents people from finding and exploiting reward contingencies in their environment. This inability to learn reward contingencies is depressogenic: a potentially controllable environment becomes uncontrollable, akin to that found in learned helplessness experiments (Seligman & Maier, Reference Seligman and Maier1967). Support for this account is provided by evidence that cognitive decline rather than lesion location predicts depressive symptoms after a stroke (Nys et al. Reference Nys, van Zandvoort, van der Worp, de Haan, de Kort, Jansen and Kappelle2006) and that cardiovascular risk factors predict depressive symptoms only if executive control is impaired (Mast et al. Reference Mast, Yochim, MacNeill and Lichtenberg2004).
Limitations
The cross-sectional, case-control design of our study precludes causal inferences. It also raises questions about possible confounders, although many were ruled out in our sensitivity analyses. The fact that many patients were partially recovered from depression can be seen as a limitation. Another limitation is the availability of FLAIR data on only a subsample (58%), albeit large enough to detect a strong relationship between the WMH burden and BOLD response. The EXIT is a broad screening measure of executive control, and it is possible that its specific domains would show a stronger overlap with effects of depression. However, a recent meta-analysis demonstrated a broad, non-selective impairment across executive function domains in depression (Snyder, Reference Snyder2013). Further, our results are strengthened by absence of external reinforcement on the EXIT. Finally, a modest sample size warrants caution.
In summary, in a sample of older adults, depression and impaired executive control were linked to dissociable disruptions in the corticostriatothalamic encoding of reinforcement. Neither of these disruptions was explained by WMH. We conclude that ascending striatocortical reward signals appear to be disrupted in late-life depression, while deficits associated with poor executive control more closely resemble effects of lateral prefrontal lesions or prefrontal disconnection.
Acknowledgements
The authors thank Amanda Collier, B.S., for her work on data collection and processing, Jan Kalkus, B.S., for help with data processing and manuscript preparation, and Swathi Gujral, B.S., Natalie Truty, B.S., and Cori Shollenberger, B.S., for their hard work on recruitment and assessments. Meryl A. Butters, Ph.D., supervised neuropsychological assessments and shared data collected in her study. Ariel G. Gildengers, M.D., also shared data collected in his study. The authors also thank George Alexopoulos, MD, for his thoughtful comments on the manuscript. Grant support: NIMH (K23MH086620, K23MH070471, R01MH085651, R01MH080240, R01MH084921) and the American Foundation for Suicide Prevention.
Declaration of Interest
Dr Reynolds reports receiving pharmaceutical support for NIH-sponsored research studies from Bristol-Myers Squibb, Forest, Pfizer, and Lilly; receiving grants from the National Institute of Mental Health, National Institute on Aging, National Center for Minority Health Disparities, National Heart Lung and Blood Institute, Center for Medicare and Medicaid Services (CMS), Patient Centered Outcomes Research Institute (PCORI),the Commonwealth of Pennsylvania, the John A Hartford Foundation, National Palliative Care Research Center (NPCRC), Clinical and Translational Science Institute (CTSI), and the American Foundation for Suicide Prevention; and serving on the American Association for Geriatric Psychiatry editorial review board. He has received an honorarium as a speaker from MedScape/WEB MD. He is the co-inventor (Licensed Intellectual Property) of Psychometric analysis of the Pittsburgh Sleep Quality Index (PSQI) PRO10050447 (PI: Buysse) and is supported by the National Institutes of Health through grant numbers P60 MD000207; P30 MH090333; UL1RR024153, UL1TR000005; and the UPMC Endowment in Geriatric Psychiatry. None of the other authors have any conflict of interest to report.
