1. Subjective value gleaned from vigor

The concept of subjective value is central to economics, as well as cognitive neuroscience. Economists have quantified subjective value so that governments may produce greater good via public policy. Cognitive neuroscientists have estimated subjective value to understand the neural basis of decision-making. Both have relied on a simple methodology: choice. When you select one option over another, you indicate that you value the chosen option more. However, your choice indicates your order of preference, not your degree of preference (Samuelson, Reference Samuelson1938). For example, if I noticed that from the dessert cart you picked caramel flan, not the bowl of fruit, I would infer that you preferred the flan to the fruit. However, I could not assign a numeric scale that reflected how much more you preferred the chosen item to the one left behind. Is there a way to measure your degree of preference?

In the last decade, neuroscientists have added a new tool with which to measure subjective value: vigor. They noticed that during decision-making, as people and other animals deliberated about their options, their movements reflected not just their choice, but also the subjective value of that choice (Haith, Reppert, & Shadmehr, Reference Haith, Reppert and Shadmehr2012; Reppert, Lempert, Glimcher, & Shadmehr, Reference Reppert, Lempert, Glimcher and Shadmehr2015; Reppert et al., Reference Reppert, Rigas, Herzfeld, Sedaghat-Nejad, Komogortsev and Shadmehr2018; Yoon, Geary, Ahmed, & Shadmehr, Reference Yoon, Geary, Ahmed and Shadmehr2018; Yoon, Jaleel, Ahmed, & Shadmehr, Reference Yoon, Jaleel, Ahmed and Shadmehr2020). Thus, if the restaurant used a camera to monitor your eye movements as you considered your dessert options, their analysis might reveal that when you shifted your gaze among the options, your saccadic eye movements exhibited something interesting: The velocity of the eyes was somewhat greater when you shifted your gaze from the fruit to the flan, than from the flan to the fruit (Yoon et al., Reference Yoon, Jaleel, Ahmed and Shadmehr2020). That is, before you verbalized your decision to the waiter, as you deliberated and gazed back and forth between the options, your saccade vigor was greater toward the item that you eventually chose. Your choice of flan over fruit indicated your preference, but the restaurant via its measurement gained an additional piece of information: The eye velocities that shifted your gaze toward the flan or the fruit described a proxy for how much more you preferred one to the other. That would certainly be a useful bit of information for setting prices on the dessert menu.

2. The mathematical link between vigor and subjective value

Why should the way we move toward a goal be affected by how we value the destination? After all, we could imagine a scenario in which the brain assigns value to the various stimuli, picks the one that has the greatest subjective value, and then passes on the chosen action to the motor system, which robotically executes a movement to acquire that stimulus. Indeed, this is the traditional framework for motor control; decision-making circuits make choices, whereas the motor circuits produce the actions needed to acquire that choice.

This robotic view of motor control, divorced from the decision-making process, is illustrated in the language that is still commonly used to describe saccadic eye movements. The relationship between saccade amplitude and velocity is imagined to be invariant in healthy people, unaffected by their affective state, or the reward at stake, and is referred to as the “main sequence,” a term that motor control borrowed from astronomy (Bahill, Clark, & Stark, Reference Bahill, Clark and Stark1975). In astronomy, the term refers to a plot of star color versus brightness, showing that during the hydrogen burning stage of a star's lifetime, it follows a specific two-dimensional trajectory. In motor control, main sequence refers to the relationship between peak saccade velocity versus amplitude, and deviations from “normal” can be interpreted as a pathological condition. However, recent study has shown that this relationship is far from invariant. Rather, peak velocity increases when gazing toward something that the subject associates with reward (Manohar, Muhammed, Fallon, & Husain, Reference Manohar, Muhammed, Fallon and Husain2019, Manohar et al., Reference Manohar, Chong, Apps, Batla, Stamelou, Jarman and Husain2015; Reppert et al., Reference Reppert, Lempert, Glimcher and Shadmehr2015; Seideman, Stanford, & Salinas, Reference Seideman, Stanford and Salinas2018; Takikawa, Kawagoe, & Hikosaka, Reference Takikawa, Kawagoe and Hikosaka2002; Yoon et al., Reference Yoon, Jaleel, Ahmed and Shadmehr2020), decreases when the subject is fatigued (Golla et al., Reference Golla, Tziridis, Haarmeier, Catz, Barash and Thier2008; Straube, Fuchs, Usher, & Robinson, Reference Straube, Fuchs, Usher and Robinson1997), and is even modulated by the recent history of reward and effort experienced by the subject (Yoon et al., Reference Yoon, Geary, Ahmed and Shadmehr2018).

In a similar vein, motor control has historically overlooked the decision-making process when considering how fast we walk and how fast we reach. For example, consider speed of walking. Unlike the invariance presumed in saccades, walking speed is thought to be determined by the energetic cost of the movement. This energetic cost is high for slow speeds of walking, and also high for fast speeds, but exhibits a minimum at an intermediate speed (Ralston, Reference Ralston1958). The energetically optimal speed is shifted to slower speeds on inclines and when carrying loads, and, indeed, gait speeds in animals are slower under these conditions (Wickler, Hoyt, Cogger, & Hall, Reference Wickler, Hoyt, Cogger and Hall2001; Wickler, Hoyt, Cogger, & Hirschbein, Reference Wickler, Hoyt, Cogger and Hirschbein2000). Similarly, energetic cost of reaching also exhibits a minimum around an optimal speed (Shadmehr, Huang, & Ahmed, Reference Shadmehr, Huang and Ahmed2016), and we reach slower when the effort cost of the movement is higher (Gordon, Ghilardi, & Ghez, Reference Gordon, Ghilardi and Ghez1994). Thus, it has commonly been assumed that the speed with which we choose to walk or to reach is selected in such a way as to minimize the energetic cost of the movement.

However, people do not always move in the energetically optimal manner. When walking down an incline, people prefer a gait that incurs a greater energetic cost than a more relaxed gait (Hunter, Hendrix, & Dean, Reference Hunter, Hendrix and Dean2010). There is also the intriguing observation that when effort costs are higher, not only do we make slower movements, but we also take longer to start those moments, even though the reaction time has no influence on the movement's energetic cost (Reppert et al., Reference Reppert, Rigas, Herzfeld, Sedaghat-Nejad, Komogortsev and Shadmehr2018). Thus, when it comes to setting movement speed, energetic optimality is not the sole concern of the brain.

As another example, consider the curious finding that average walking speed in a city correlates the strength of a city's economy and the economic well-being of its average citizen (Levine & Norenzayan, Reference Levine and Norenzayan1999). The better off a city's residents are, the faster they walk (Shadmehr & Ahmed, Reference Shadmehr and Ahmed2020). Not only do people walk faster, but they also perform everyday transactions faster. This suggests that the history of an individual's experience, reflected in their well-being, influences the vigor of their movements. Similar effects have been observed in eye movements. The speed with which the eyes move toward a target depends not just on the value of that target, but also the history of reward and effort experienced by the subject (Yoon et al., Reference Yoon, Geary, Ahmed and Shadmehr2018).

The link between decision-making and motor control is illustrated by the fact that as the promised reward increases, animals are more likely to choose that option, but they also react earlier to the stimulus, and move with greater velocity toward it (Berret, Castanier, Bastide, & Deroche, Reference Berret, Castanier, Bastide and Deroche2018; Kawagoe, Takikawa, & Hikosaka, Reference Kawagoe, Takikawa and Hikosaka1998; Milstein & Dorris, Reference Milstein and Dorris2007; Sackaloo, Strouse, & Rice, Reference Sackaloo, Strouse and Rice2015; Seideman et al., Reference Seideman, Stanford and Salinas2018; Summerside, Shadmehr, & Ahmed, Reference Summerside, Shadmehr and Ahmed2018; Thura, Cos, Trung, & Cisek, Reference Thura, Cos, Trung and Cisek2014; Xu-Wilson, Zee, & Shadmehr, Reference Xu-Wilson, Zee and Shadmehr2009). In contrast, as the effort required to acquire reward increases, animals are less likely to choose that option, but if chosen, movements take longer to start, and longer to conclude (Gordon et al., Reference Gordon, Ghilardi and Ghez1994; Ivry, Reference Ivry1986; Reppert et al., Reference Reppert, Rigas, Herzfeld, Sedaghat-Nejad, Komogortsev and Shadmehr2018; Rosenbaum, Reference Rosenbaum1980; Shadmehr et al., Reference Shadmehr, Huang and Ahmed2016; Stelmach & Worringham, Reference Stelmach and Worringham1988; Wickler et al., Reference Wickler, Hoyt, Cogger and Hall2001). These observations are true regardless of whether the action involves movements of our limbs, or movements of our eyes.

For example, when presented with a candy bar that we like, the latency to start the reach is earlier, and the reach velocity is higher. Similarly, when we are presented with a small image of that candy bar on a video screen, the saccadic eye movement toward that image tends to have a shorter latency, as well as higher peak velocity. Thus, subjective value, that elusive variable that is critical to decision-making, leaves its impression on our actions via the latency to begin movements, and the velocity with which to make that movement.

But why should the way we react (latency) and the way we move (velocity) be influenced by the subjective value that we assign to our destination? To answer this question, in this book (Shadmehr & Ahmed, Reference Shadmehr and Ahmed2020) we imagine the problem from the point of view of an ecologist: Is there a common currency that the brain is trying to optimize via its choices and movements?

In the natural environment, animals appear to make choices based on the desire to maximize a specific currency: the global capture rate, defined as the sum of all rewards acquired minus all efforts expended, divided by time (Bautista, Tinbergen, & Kacelnik, Reference Bautista, Tinbergen and Kacelnik2001; Richardson & Verbeek, Reference Richardson and Verbeek1986). For example, crows that live along the beaches of the pacific northwest of the United States rely on clams for their food. The clams hide in the sand, and the crows look for them, spending time and effort to dig them up. However, once a clam has been uncovered, the crow faces a critical decision: should it spend the time and energy needed to open the clam (grab it, fly over to a rocky shore, drop it a few times, etc.), or abandon this clam and look for another one? This decision depends on the size of the clam, and optimal foraging theory provides a framework to consider this decision. Once the energetic cost of searching, flying, and harvesting is considered, along with the time it takes to perform these actions, the choice of what to invest in and what to abandon becomes one of maximizing the global capture rate. Indeed, the global capture rate, which is roughly the energetic sum of reward and effort, divided by time, appears to play a fundamental role in the longevity and fecundity of animals (Lemon, Reference Lemon1991), suggesting that living one's life in a way that increases the capture rate has evolutionary advantages.

Until recently, optimal foraging has been viewed as a framework to consider patterns of decision-making, not patterns of movement. In this book, we extend this theory and show that it also predicts vigor modulation as a function of reward and effort. The key idea is that movements dictate expenditure of effort (which costs energy), as well as expenditure of time (Yoon et al., Reference Yoon, Geary, Ahmed and Shadmehr2018). Moving faster gets you to the reward sooner, but requires greater energetic expenditure (Shadmehr et al., Reference Shadmehr, Huang and Ahmed2016). Thus, if we wish to maximize the global capture rate over the long run, then we must find policies that are informed by both the effort of making movements, and the benefit of acquiring reward.

As a result, the mathematical link between decisions and movements arises because both influence the global capture rate. To optimize this currency, we cannot simply make good choices; we must also move with vigor that is consistent with those choices.

3. Neuromodulators and their influence on decision-making and vigor

If vigor and decision-making are indeed working in alliance, then are there neural correlates to support this? A good example of the neural link between systems that assign value to stimuli, thus directing our decisions, and systems that control our movements is in Parkinson's disease. In Parkinson's disease, there is deterioration of the dopaminergic system of the basal ganglia, and its cardinal symptom is bradykinesia (slowness of movements). However, it is not the case that the patients are unable to move rapidly. Rather, it appears that the Parkinsonian brain is unwilling to expend the required effort to acquire the available reward (Manohar et al., Reference Manohar, Chong, Apps, Batla, Stamelou, Jarman and Husain2015; Mazzoni, Hristova, & Krakauer, Reference Mazzoni, Hristova and Krakauer2007). That is, the movement disorder may be a result of a dysfunction in the economic evaluation of reward and effort, which in turn is shared by the circuitry that controls movements.

This dysfunction of economic evaluation is illustrated in monkeys that do not show reward-dependent modulation of vigor (Kawagoe, Takikawa, & Hikosaka, Reference Kawagoe, Takikawa and Hikosaka2004). In these subjects, there appears to be a critical problem: Dopamine release in the caudate nucleus of the basal ganglia appears insensitive to reward. Curiously, these monkeys do not exhibit lower latency and higher velocity saccades toward the more rewarding stimulus.

How does dopamine influence vigor? The magnitude of the inhibition imposed by the basal ganglia upon the superior colliculus depends on the expected reward from the stimulus (Sato & Hikosaka, Reference Sato and Hikosaka2002; Yasuda, Yamamoto, & Hikosaka, Reference Yasuda, Yamamoto and Hikosaka2012). The expected reward information is transmitted from the input stage of the basal ganglia (striatum) to the output stage (substantia nigra reticulata, SNr) (Kim, Amita, & Hikosaka, Reference Kim, Amita and Hikosaka2017; Kim & Hikosaka, Reference Kim and Hikosaka2013; Lauwereyns, Watanabe, Coe, & Hikosaka, Reference Lauwereyns, Watanabe, Coe and Hikosaka2002). The striatal cells receive excitatory inputs from the cerebral cortex, but their output to the SNr via the direct and indirect pathways depends on the amount of dopamine that is present in the striatum. That is, dopamine influences how the striatal cells respond to their cortical inputs, impeding or encouraging a transition to an active state, a state in which the striatal cells are more responsive to their excitatory cortical inputs. Dopamine activity, in turn, is influenced by serotonin. Together, dopamine and serotonin, these ancient neurotransmitters, influence both the decision-making process, and its vigor. As a result, dopamine release before movement onset is sufficient to increase vigor of that movement (da Silva, Tecuapetla, Paixao, & Costa, Reference da Silva, Tecuapetla, Paixao and Costa2018).

Although dopamine neurons fire in response to stimuli that promise reward, acting as a teacher for learning subjective value, dopamine concentration in the striatum is modulated by the effort required to acquire that reward (Schelp et al., Reference Schelp, Pultorak, Rakowski, Gomez, Krzystyniak, Das and Oleson2017, p. 2017; Syed et al., Reference Syed, Grima, Magill, Bogacz, Brown and Walton2016). That is, dopamine spiking activity at the time of stimulus presentation appears to encode reward prediction error, whereas during production of effort, dopamine concentrations appear to support production of vigorous movements. Indeed, reward prediction error, and not reward alone, is the dominant modulator of vigor (Sedaghat-Nejad, Herzfeld, & Shadmehr, Reference Sedaghat-Nejad, Herzfeld and Shadmehr2019). In this way, dopamine plays the role of Janus the Roman god: one face looking forward to reward, the other looking toward the effort needed to acquire that reward.

However, dopamine firing rates appear insensitive to one of the critical variables necessary to control vigor: the global capture rate (i.e., history of reward and effort) (Cohen, Amoroso, & Uchida, Reference Cohen, Amoroso and Uchida2015). If firing rates of dopamine neurons do not provide a signal that reflects the global capture rate, then how is it that the brain controls vigor as a function of reward history? (Yoon et al., Reference Yoon, Geary, Ahmed and Shadmehr2018). One possible answer is serotonin (Cohen et al., Reference Cohen, Amoroso and Uchida2015), which is sensitive to reward history, and in many ways appears to act as an antagonist to dopamine, modulating the willingness to wait (Eagle, Bari, & Robbins, Reference Eagle, Bari and Robbins2008; Miyazaki et al., Reference Miyazaki, Miyazaki, Tanaka, Yamanaka, Takahashi, Tabuchi and Doya2014), increasing the harvest duration (Lottem et al., Reference Lottem, Banerjee, Vertechi, Sarra, Lohuis and Mainen2018), and encouraging sloth (Correia et al., Reference Correia, Lottem, Banerjee, Machado, Carey and Mainen2017).

The elevated presence of serotonin in the brain under normal conditions coincides with reduced movement vigor, and a reluctance to produce effort in exchange for reward (Bailey et al., Reference Bailey, Goldman, Bello, Chohan, Jeong, Winiger and Simpson2018). In addition, serotonin increases the tendency to linger and harvest for a longer period, encouraging persistence. In some of these cases, the effect of serotonin on behavior is mediated via modulation of dopamine.

4. Moving forward

An exciting implication of the research on vigor is that movements can provide a proxy for subjective value. This may be of importance to two disparate fields: economics and neuroscience. Although economists have strived for decades to estimate subjective value based on preferences that people have expressed via their choices, vigor may provide an implicit measure of this elusive variable. On the contrary, because neurobiology of vigor is based on many of the same neurotransmitters that malfunction in disease, such as Parkinson's disease and depression, tracking vigor may provide a real-time proxy for the state of these chemicals, thus aiding administration of interventions and providing an objective measure of treatment efficacy.

Overall, the book Vigor attempts to synthesize the mathematical, behavioral, and neurophysiological results gathered in the past decade regarding the link between control of movements, and control of decisions. From a scientific perspective, the results imply that by studying vigor, we may discover a new way with which to measure individual preferences and thus provide economists a behavioral tool that can objectively estimate subjective value. From a clinical perspective, vigor may act as a proxy for our current affective state. And from a technological perspective, with the increasing power of smart phones and presence of surveillance cameras, machines may measure our movements and gather vigor-based estimates of our personal preferences, even when we are not overtly making a choice, thereby unwittingly revealing one of our secrets; how much we value the thing we are moving toward.