The brain mechanisms supporting the achievement of human-level goals are sophistications of the neuronal architectures used by our evolutionary ancestors for situated interaction. These simple organisms had to select adaptive action rapidly; and for this purpose, a brain design based on a staged perception-cognition-action pipeline was probably too slow. A better design could be a basic (but robust) control-theoretic loop with several operations deploying in parallel and influencing each other; for example, an “affordance competition” architecture that specifies and selects multiple potential action plans in parallel under the biasing influence of current goal and motivation contexts (Cisek & Kalaska Reference Cisek and Kalaska2010).
This embodied view encourages seeing all cognitive processes through the lens of action and goal achievement. Here, the main role of perception is signaling opportunities for achieving valuable goals through action, not providing an objective representation of the external environment (Gibson Reference Gibson1979; Proffitt Reference Proffitt2006). Memory, too, is in the service of action and goals: it integrates patterns of past interaction with current perception to provide context for goal selection and achievement (Glenberg Reference Glenberg1997; Verschure Reference Verschure2012).
From this perspective, the brains of simple organisms are already well configured for goal achievement in situated interactions. More complex cognitive architectures (including human) might be elaborations of this initial brain design, but each would retain the initial design's essential aspects (Pezzulo & Castelfranchi Reference Pezzulo and Castelfranchi2007; Reference Pezzulo and Castelfranchi2009). However, this view leaves unexplained how higher animals achieve increasingly more complex goals (e.g., distal, abstract) and with higher behavioral flexibility.
One hypothesis is that increasingly more complex forms of goal-directedness might result from the progressive improvement of predictive abilities, which permitted incorporating future events in decision and action control (Pezzulo Reference Pezzulo2011). Here, I elaborate on that idea and argue that goals “configure” cognitive processing by controlling the predictive dynamics of the brain.
The notions of prediction and prediction error are ubiquitous in computational neuroscience theories, including those for perceptual processing (e.g., predictive coding), motivation and reward (e.g., reinforcement learning), and action control (e.g., internal modeling) and its dysfunctions (Frith et al. Reference Frith, Blakemore and Wolpert2000). Ultimately, the brain could be seen as a prediction machine (Bar Reference Bar2007; Friston Reference Friston2010; Pezzulo Reference Pezzulo2008); still it cannot predict everything.
I propose that the current goals define the relevant dimensions along which predictions are generated and prediction errors are monitored and corrected. For example, goals determine which expectations have value and should be fulfilled, which events should be predicted and which errors monitored, how prediction errors should be evaluated (e.g., as good or bad), and what should be learned from them. In sum, goals can bias all cognitive processes by channeling predictions toward goal-relevant events. Below I discuss representative theories assigning goals a role in “(re)configuring” cognition and link them within a common prediction-based framework.
In an early cybernetic model of goal-directedness, the TOTE, the discrepancy between desired (goal) and actual (sensed) state triggers a goal-directed action (Miller et al. Reference Miller, Galanter and Pribram1960; Pezzulo et al. Reference Pezzulo, Baldassarre, Butz, Castelfranchi, Hoffmann, Butz, Sigaud, Pezzulo and Baldassarre2007). Thus, goals determine what errors are monitored and what actions minimize them; but they do not directly modulate predictive and perceptual processes.
Ideomotor theories recognize those links and argue that actions are controlled by goals, which are coded as distal action effects (Hommel et al. Reference Hommel, Musseler, Aschersleben and Prinz2001). When a goal is selected, goal-related feature dimensions are “intentionally weighted” and have a stronger impact on perceptual processing and response selection (Memelink & Hommel Reference Memelink and Hommel2013). “Reconfiguration” happens because goal selection enhances the salience of effect-related (goal-related) sensory events and modulates attention to these events. Similar hypothesis are advanced in theories of selective attention and top-down control (Desimone & Duncan Reference Desimone and Duncan1995; Miller & Cohen Reference Miller and Cohen2001).
Goals have also motivational aspects that influence perception and action. A recent theory emphasizes that perception can combine external (sensory) and interoceptive (drive- and goal-related) signals, so that a mismatch between an internal need (e.g., hunger) and sensory stimuli (e.g., no food) modulates the importance of the visual signals (Montague & King-Casas Reference Montague and King-Casas2007; Pezzulo Reference Pezzulo2013), and attention can be deployed to goal-relevant events (Mysore & Knudsen Reference Mysore and Knudsen2011). Motivational factors influence action selection, too, using a mechanism that minimizes reward prediction errors through reinforcement learning (Dayan & Balleine Reference Dayan and Balleine2002) and during planning (Pezzulo et al. Reference Pezzulo, Rigoli and Chersi2013; Solway & Botvinick Reference Solway and Botvinick2012).
All of those seemingly disconnected ideas can be reconciled within the prediction-based framework of active inference (Friston Reference Friston2005; Reference Friston2010). In this framework, goals correspond to probabilistic (Bayesian) priors at high hierarchical layers of the brain and are achieved by steering actions that minimize prediction errors between the priors (goals) and the current state. Brain hierarchies encode internal models at multiple timescales that give flexibility and context for goal achievement. Perception compares competing perceptual hypotheses by generating predictions at higher hierarchical levels, comparing the predictions with sensory stimuli, and using the prediction errors for hypothesis selection. Doing so permits reducing uncertainty in the sensorium, which is necessary for accurate action. Performing a complete perceptual processing is too costly; however, it can also be unnecessary as only few perceptual hypotheses correspond to highly valued goals. Thus, during perceptual inference, goals control which aspects of the sensorium should be made less uncertain and bias perception toward the states of affairs that realize the agent goals with higher probability (and as a side effect, they produce an “optimism bias”, Sharot et al. Reference Sharot, Korn and Dolan2011). In other words, goals can modulate perception and behavior by regulating attention and affordance selection; in the active inference framework, this corresponds to optimizing the precision (inverse variance) of the relevant parts of the sensory and proprioceptive data (Feldman & Friston Reference Feldman and Friston2010). From this perspective, perception signals where value (i.e., the opportunity to achieve a goal) is and permits picking up affordances and achieving goals.
A challenge for this framework is explaining increasingly more complex, human-level forms of goal-directedness, often linked to prefrontal function and cognitive control. An initial hypothesis is that cognitive control might essentially replicate the active inference scheme, but in a “covert” (simulated) form (Pezzulo Reference Pezzulo2012). The covert process allows generating goals that need not be achieved immediately but are retained in prospective memory until there is an opportunity to achieve them. The goals generated through this process might use the aforementioned precision-weighting mechanisms to influence cognitive processing over the long timescales required to achieve distal objectives; at least, unless another “selfish” goal takes control.
The brain mechanisms supporting the achievement of human-level goals are sophistications of the neuronal architectures used by our evolutionary ancestors for situated interaction. These simple organisms had to select adaptive action rapidly; and for this purpose, a brain design based on a staged perception-cognition-action pipeline was probably too slow. A better design could be a basic (but robust) control-theoretic loop with several operations deploying in parallel and influencing each other; for example, an “affordance competition” architecture that specifies and selects multiple potential action plans in parallel under the biasing influence of current goal and motivation contexts (Cisek & Kalaska Reference Cisek and Kalaska2010).
This embodied view encourages seeing all cognitive processes through the lens of action and goal achievement. Here, the main role of perception is signaling opportunities for achieving valuable goals through action, not providing an objective representation of the external environment (Gibson Reference Gibson1979; Proffitt Reference Proffitt2006). Memory, too, is in the service of action and goals: it integrates patterns of past interaction with current perception to provide context for goal selection and achievement (Glenberg Reference Glenberg1997; Verschure Reference Verschure2012).
From this perspective, the brains of simple organisms are already well configured for goal achievement in situated interactions. More complex cognitive architectures (including human) might be elaborations of this initial brain design, but each would retain the initial design's essential aspects (Pezzulo & Castelfranchi Reference Pezzulo and Castelfranchi2007; Reference Pezzulo and Castelfranchi2009). However, this view leaves unexplained how higher animals achieve increasingly more complex goals (e.g., distal, abstract) and with higher behavioral flexibility.
One hypothesis is that increasingly more complex forms of goal-directedness might result from the progressive improvement of predictive abilities, which permitted incorporating future events in decision and action control (Pezzulo Reference Pezzulo2011). Here, I elaborate on that idea and argue that goals “configure” cognitive processing by controlling the predictive dynamics of the brain.
The notions of prediction and prediction error are ubiquitous in computational neuroscience theories, including those for perceptual processing (e.g., predictive coding), motivation and reward (e.g., reinforcement learning), and action control (e.g., internal modeling) and its dysfunctions (Frith et al. Reference Frith, Blakemore and Wolpert2000). Ultimately, the brain could be seen as a prediction machine (Bar Reference Bar2007; Friston Reference Friston2010; Pezzulo Reference Pezzulo2008); still it cannot predict everything.
I propose that the current goals define the relevant dimensions along which predictions are generated and prediction errors are monitored and corrected. For example, goals determine which expectations have value and should be fulfilled, which events should be predicted and which errors monitored, how prediction errors should be evaluated (e.g., as good or bad), and what should be learned from them. In sum, goals can bias all cognitive processes by channeling predictions toward goal-relevant events. Below I discuss representative theories assigning goals a role in “(re)configuring” cognition and link them within a common prediction-based framework.
In an early cybernetic model of goal-directedness, the TOTE, the discrepancy between desired (goal) and actual (sensed) state triggers a goal-directed action (Miller et al. Reference Miller, Galanter and Pribram1960; Pezzulo et al. Reference Pezzulo, Baldassarre, Butz, Castelfranchi, Hoffmann, Butz, Sigaud, Pezzulo and Baldassarre2007). Thus, goals determine what errors are monitored and what actions minimize them; but they do not directly modulate predictive and perceptual processes.
Ideomotor theories recognize those links and argue that actions are controlled by goals, which are coded as distal action effects (Hommel et al. Reference Hommel, Musseler, Aschersleben and Prinz2001). When a goal is selected, goal-related feature dimensions are “intentionally weighted” and have a stronger impact on perceptual processing and response selection (Memelink & Hommel Reference Memelink and Hommel2013). “Reconfiguration” happens because goal selection enhances the salience of effect-related (goal-related) sensory events and modulates attention to these events. Similar hypothesis are advanced in theories of selective attention and top-down control (Desimone & Duncan Reference Desimone and Duncan1995; Miller & Cohen Reference Miller and Cohen2001).
Goals have also motivational aspects that influence perception and action. A recent theory emphasizes that perception can combine external (sensory) and interoceptive (drive- and goal-related) signals, so that a mismatch between an internal need (e.g., hunger) and sensory stimuli (e.g., no food) modulates the importance of the visual signals (Montague & King-Casas Reference Montague and King-Casas2007; Pezzulo Reference Pezzulo2013), and attention can be deployed to goal-relevant events (Mysore & Knudsen Reference Mysore and Knudsen2011). Motivational factors influence action selection, too, using a mechanism that minimizes reward prediction errors through reinforcement learning (Dayan & Balleine Reference Dayan and Balleine2002) and during planning (Pezzulo et al. Reference Pezzulo, Rigoli and Chersi2013; Solway & Botvinick Reference Solway and Botvinick2012).
All of those seemingly disconnected ideas can be reconciled within the prediction-based framework of active inference (Friston Reference Friston2005; Reference Friston2010). In this framework, goals correspond to probabilistic (Bayesian) priors at high hierarchical layers of the brain and are achieved by steering actions that minimize prediction errors between the priors (goals) and the current state. Brain hierarchies encode internal models at multiple timescales that give flexibility and context for goal achievement. Perception compares competing perceptual hypotheses by generating predictions at higher hierarchical levels, comparing the predictions with sensory stimuli, and using the prediction errors for hypothesis selection. Doing so permits reducing uncertainty in the sensorium, which is necessary for accurate action. Performing a complete perceptual processing is too costly; however, it can also be unnecessary as only few perceptual hypotheses correspond to highly valued goals. Thus, during perceptual inference, goals control which aspects of the sensorium should be made less uncertain and bias perception toward the states of affairs that realize the agent goals with higher probability (and as a side effect, they produce an “optimism bias”, Sharot et al. Reference Sharot, Korn and Dolan2011). In other words, goals can modulate perception and behavior by regulating attention and affordance selection; in the active inference framework, this corresponds to optimizing the precision (inverse variance) of the relevant parts of the sensory and proprioceptive data (Feldman & Friston Reference Feldman and Friston2010). From this perspective, perception signals where value (i.e., the opportunity to achieve a goal) is and permits picking up affordances and achieving goals.
A challenge for this framework is explaining increasingly more complex, human-level forms of goal-directedness, often linked to prefrontal function and cognitive control. An initial hypothesis is that cognitive control might essentially replicate the active inference scheme, but in a “covert” (simulated) form (Pezzulo Reference Pezzulo2012). The covert process allows generating goals that need not be achieved immediately but are retained in prospective memory until there is an opportunity to achieve them. The goals generated through this process might use the aforementioned precision-weighting mechanisms to influence cognitive processing over the long timescales required to achieve distal objectives; at least, unless another “selfish” goal takes control.
ACKNOWLEDGMENT
Research funded by the EU's FP7 under grant agreement no. FP7-ICT-270108 (Goal-Leaders).