Mather and colleagues have developed an impressive theoretical model linking arousal-mediated changes in cognition to the local signaling dynamics at the axon terminals of noradrenergic locus coeruleus (LC) projections. The authors provide compelling evidence for how this link between biology and cognition is made; however, they leave open a key question: Why should the brain undergo fluctuations in arousal that influence information processing? Answering this question is important not only for appreciating the intricate biological design proposed by Mather and colleagues, but also for understanding the contexts in which such a design would be maladaptive. Here we explore the why question and speculate that healthy aging may constitute one such maladaptive context.
So, if the brain has a good system for prioritizing sources of information, why should it ever be turned down? One possible answer is that prioritized sources of information are often imperfect predictors of behaviorally relevant variables. In such cases, inferring the variable of interest (e.g., the best foraging location) requires combining probabilistic sensory information (e.g., the color of the berry bushes on a distant hill) with currently held beliefs (how many berries are expected based on past experience). The relative contributions of these factors should be determined by the precision with which they predict the behaviorally relevant variable; thus, sensory information should be discounted when prior expectations are strong (e.g., the forager has recently counted the berries on the bush) or when sensory evidence is weak (e.g., the color of the bush is a bad predictor of caloric yield). In either circumstance, amplifying the prioritized sensory information would disrupt inference by allowing poor sensory cues to overwhelm precise internal expectations. There is some evidence to suggest that arousal levels are decremented under such conditions. Pupil diameter, a marker for arousal, is large during periods of uncertainty and constricts as expectations become more reliable (Lavín et al. Reference Lavín, San Martín and Rosales Jubal2014; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Nieuwenhuis et al. Reference Nieuwenhuis, De Geus and Aston-Jones2011; Preuschoff Reference Preuschoff2011). Information that is inconsistent with expectations drives sharp increases in arousal that appear to affect the relative influence of new observations on behavior (Lavín et al. Reference Lavín, San Martín and Rosales Jubal2014; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Nieuwenhuis et al. Reference Nieuwenhuis, De Geus and Aston-Jones2011; Preuschoff Reference Preuschoff2011). These data suggest that decrements in arousal, likely implemented through reductions in the firing of LC neurons, may serve an important role in optimal inference (Joshi et al. Reference Joshi, Li, Kalwani and Gold2016). In particular, one role of low arousal levels might be to protect strong internal predictions from prioritized but potentially distracting information.
Another normative justification for reducing the influence of priority maps is that under some conditions, it is useful to explore alternatives to the current course of action that might provide better long-run returns. For example, information about a known source of reward (e.g., the berry bush on the hill) might be prioritized over other potential sources of reward that are yet to be discovered (e.g., an apple tree that only recently began to bear fruit). Thus, reducing the influence of priority maps could provide an incentive to explore non-prioritized inputs. Exploring potential alternative reward streams becomes important as the known source of reward is depleted, particularly if there is sufficient time to capitalize on any knowledge gained in the exploratory process (Wilson et al. Reference Wilson, Geana, White, Ludvig and Cohen2014). There is some evidence to suggest that shifts from exploiting a known source of reward to exploring alternative options are accompanied by a shift from a phasic (stimulus evoked) mode of LC firing to a high tonic mode (Aston-Jones & Cohen Reference Aston-Jones and Cohen2005; Jepma & Nieuwenhuis Reference Jepma and Nieuwenhuis2011). Thus, fluctuations in arousal might allow for an effective navigation of the trade-off between exploitation and exploration in addition to optimizing inference.
The biological mechanism proposed by Mather and colleagues suggests that the optimization of inference and exploration through fluctuations in arousal may be highly sensitive to the state of the LC–norepinephrine (NE) system and its biophysical components. There is some evidence for dysregulation of this system over the course of healthy aging. Findings from histologic post-mortem studies point to a substantial cell loss in the LC with age (Chan-Palay & Asan Reference Chan-Palay and Asan1989; Grudzien et al. Reference Grudzien, Shaw, Weintraub, Bigio, Mash and Mesulam2007; Manaye et al. Reference Manaye, McIntire, Mann and German1995). Moreover, neuronal density in LC is strongly related to cognitive decline in the period before death (Wilson et al. Reference Wilson, Nag, Boyle, Hizel, Yu, Buchman, Schneider and Bennett2013). These findings seem to line up with recent in vivo structural magnetic resonance imaging findings that point to neuromelanin-related magnetic resonance signal loss with age (Shibata et al. Reference Shibata, Sasaki, Tohyama, Kanbara, Otsuka, Ehara and Sakai2006). Taken together, these findings suggest that aging is associated with substantial structural changes in the LC, which are associated with cognitive decline.
One potential cause of age-related cognitive decline could be that this pattern of changes in the NE system disrupts optimal inference. In particular, lower levels of NE could prevent the positive feedback of glutamate on NE release from achieving high enough NE levels to activate low-affinity β-adrenoceptors proposed by Mather and colleagues. This could lead to a suppression of high-priority signals, even at high arousal levels associated with uncertainty, when new information should be highly influential on behaviorally relevant beliefs. Consistent with this notion, older adults show selective behavioral impairments at learning under conditions of uncertainty, the same conditions that typically drive increased arousal and increased influence of new information on learning in younger participants (Eppinger et al. Reference Eppinger, Kray, Mock and Mecklinger2008; Reference Eppinger, Hämmerer and Li2011; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Reference Nassar, Bruckner, Gold, Li, Heekeren and Eppinger2016). The cause of this learning impairment has not been directly linked to the function of the LC–NE system to date, but in light of the biological link provided by Mather and colleagues, it should be explored in the very near future.
Changes in NE functioning with age may also affect the ability to mediate exploration and exploitation in older adults. In particular, reduced NE levels may prevent phasic signals from activating β-adrenoceptors, even in regions where signals are prioritized through association with an exploitative action. This would reduce the contrast between exploitative and exploratory action representations and shift behavior toward a more exploratory regime. Interestingly, increased choice variability, which can be used as a strategy for random exploration, is enhanced in older adults across a wide range of tasks (Garrett et al. Reference Garrett, Samanez-Larkin, MacDonald, Lindenberger, Mcintosh and Grady2013). Future work should focus on animal models where the mechanisms for age-related changes can be explored to the level of detail specified by Mather and colleagues.
Mather and colleagues have developed an impressive theoretical model linking arousal-mediated changes in cognition to the local signaling dynamics at the axon terminals of noradrenergic locus coeruleus (LC) projections. The authors provide compelling evidence for how this link between biology and cognition is made; however, they leave open a key question: Why should the brain undergo fluctuations in arousal that influence information processing? Answering this question is important not only for appreciating the intricate biological design proposed by Mather and colleagues, but also for understanding the contexts in which such a design would be maladaptive. Here we explore the why question and speculate that healthy aging may constitute one such maladaptive context.
So, if the brain has a good system for prioritizing sources of information, why should it ever be turned down? One possible answer is that prioritized sources of information are often imperfect predictors of behaviorally relevant variables. In such cases, inferring the variable of interest (e.g., the best foraging location) requires combining probabilistic sensory information (e.g., the color of the berry bushes on a distant hill) with currently held beliefs (how many berries are expected based on past experience). The relative contributions of these factors should be determined by the precision with which they predict the behaviorally relevant variable; thus, sensory information should be discounted when prior expectations are strong (e.g., the forager has recently counted the berries on the bush) or when sensory evidence is weak (e.g., the color of the bush is a bad predictor of caloric yield). In either circumstance, amplifying the prioritized sensory information would disrupt inference by allowing poor sensory cues to overwhelm precise internal expectations. There is some evidence to suggest that arousal levels are decremented under such conditions. Pupil diameter, a marker for arousal, is large during periods of uncertainty and constricts as expectations become more reliable (Lavín et al. Reference Lavín, San Martín and Rosales Jubal2014; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Nieuwenhuis et al. Reference Nieuwenhuis, De Geus and Aston-Jones2011; Preuschoff Reference Preuschoff2011). Information that is inconsistent with expectations drives sharp increases in arousal that appear to affect the relative influence of new observations on behavior (Lavín et al. Reference Lavín, San Martín and Rosales Jubal2014; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Nieuwenhuis et al. Reference Nieuwenhuis, De Geus and Aston-Jones2011; Preuschoff Reference Preuschoff2011). These data suggest that decrements in arousal, likely implemented through reductions in the firing of LC neurons, may serve an important role in optimal inference (Joshi et al. Reference Joshi, Li, Kalwani and Gold2016). In particular, one role of low arousal levels might be to protect strong internal predictions from prioritized but potentially distracting information.
Another normative justification for reducing the influence of priority maps is that under some conditions, it is useful to explore alternatives to the current course of action that might provide better long-run returns. For example, information about a known source of reward (e.g., the berry bush on the hill) might be prioritized over other potential sources of reward that are yet to be discovered (e.g., an apple tree that only recently began to bear fruit). Thus, reducing the influence of priority maps could provide an incentive to explore non-prioritized inputs. Exploring potential alternative reward streams becomes important as the known source of reward is depleted, particularly if there is sufficient time to capitalize on any knowledge gained in the exploratory process (Wilson et al. Reference Wilson, Geana, White, Ludvig and Cohen2014). There is some evidence to suggest that shifts from exploiting a known source of reward to exploring alternative options are accompanied by a shift from a phasic (stimulus evoked) mode of LC firing to a high tonic mode (Aston-Jones & Cohen Reference Aston-Jones and Cohen2005; Jepma & Nieuwenhuis Reference Jepma and Nieuwenhuis2011). Thus, fluctuations in arousal might allow for an effective navigation of the trade-off between exploitation and exploration in addition to optimizing inference.
The biological mechanism proposed by Mather and colleagues suggests that the optimization of inference and exploration through fluctuations in arousal may be highly sensitive to the state of the LC–norepinephrine (NE) system and its biophysical components. There is some evidence for dysregulation of this system over the course of healthy aging. Findings from histologic post-mortem studies point to a substantial cell loss in the LC with age (Chan-Palay & Asan Reference Chan-Palay and Asan1989; Grudzien et al. Reference Grudzien, Shaw, Weintraub, Bigio, Mash and Mesulam2007; Manaye et al. Reference Manaye, McIntire, Mann and German1995). Moreover, neuronal density in LC is strongly related to cognitive decline in the period before death (Wilson et al. Reference Wilson, Nag, Boyle, Hizel, Yu, Buchman, Schneider and Bennett2013). These findings seem to line up with recent in vivo structural magnetic resonance imaging findings that point to neuromelanin-related magnetic resonance signal loss with age (Shibata et al. Reference Shibata, Sasaki, Tohyama, Kanbara, Otsuka, Ehara and Sakai2006). Taken together, these findings suggest that aging is associated with substantial structural changes in the LC, which are associated with cognitive decline.
One potential cause of age-related cognitive decline could be that this pattern of changes in the NE system disrupts optimal inference. In particular, lower levels of NE could prevent the positive feedback of glutamate on NE release from achieving high enough NE levels to activate low-affinity β-adrenoceptors proposed by Mather and colleagues. This could lead to a suppression of high-priority signals, even at high arousal levels associated with uncertainty, when new information should be highly influential on behaviorally relevant beliefs. Consistent with this notion, older adults show selective behavioral impairments at learning under conditions of uncertainty, the same conditions that typically drive increased arousal and increased influence of new information on learning in younger participants (Eppinger et al. Reference Eppinger, Kray, Mock and Mecklinger2008; Reference Eppinger, Hämmerer and Li2011; Nassar et al. Reference Nassar, Rumsey, Wilson, Parikh, Heasly and Gold2012; Reference Nassar, Bruckner, Gold, Li, Heekeren and Eppinger2016). The cause of this learning impairment has not been directly linked to the function of the LC–NE system to date, but in light of the biological link provided by Mather and colleagues, it should be explored in the very near future.
Changes in NE functioning with age may also affect the ability to mediate exploration and exploitation in older adults. In particular, reduced NE levels may prevent phasic signals from activating β-adrenoceptors, even in regions where signals are prioritized through association with an exploitative action. This would reduce the contrast between exploitative and exploratory action representations and shift behavior toward a more exploratory regime. Interestingly, increased choice variability, which can be used as a strategy for random exploration, is enhanced in older adults across a wide range of tasks (Garrett et al. Reference Garrett, Samanez-Larkin, MacDonald, Lindenberger, Mcintosh and Grady2013). Future work should focus on animal models where the mechanisms for age-related changes can be explored to the level of detail specified by Mather and colleagues.