In his target article, Clark paints a diverse picture of how prediction is a ubiquitous part of brain and behaviour interactions. Taking heavy cues from Friston's “free energy principle,” his target article summarises ideas at the neural level, suggesting that the critical variable for sensory coding and motor control is the deviation from the expected signal, rather than the sensory or motor processing per se. In the field of sensorimotor control, this Bayesian approach is a popular one (e.g., Körding & Wolpert Reference Körding and Wolpert2004). Many researchers have built their careers showing that, in a wide range of contexts, an individual's motor behaviour can be modeled as the approximately optimal combination of the “undiluted” sensory input and the prior probability of that sensory event occurring, thus biasing the response one way or the other. Similarly, a wide range of psychophysical experiments have demonstrated that our conscious perception of events in the world represents not veridical sensory input, but the integration of multiple sources of evidence from our sensory system and our prior experience, rather than the veridical (and noisy) sensory input itself (Gregory Reference Gregory1998). An especially compelling case for this Bayesian standpoint can be made from the study of perceptual illusions, and several classic visual illusions can be explained with this optimal integration strategy (Geisler & Kersten Reference Geisler and Kersten2002; Weiss et al. Reference Weiss, Simoncelli and Adelson2002). In these contexts, this integration is thought to overcome the noise in the system of our sensory organs, maximising the likelihood of perceptual or motor “success.”
Despite the apparent descriptive power of optimally combining sensory prediction with sensory input, there are common situations where conscious perception is clearly not a product of Bayesian-style optimal integration. In fact, when we lift an object and experience its weight, our conscious perception of how heavy it feels is almost exactly the opposite of what might be expected if a perceiver integrates perpetual priors with sensory input. This incongruence is easily demonstrated with the famous size–weight illusion (SWI), first described in 1891 by Augustin Charpentier (translation by Murray et al. Reference Murray, Ellis, Bandomir and Ross1999). The SWI occurs when small and large objects, that otherwise look similar to one another, are adjusted to have identical weights. When individuals lift these objects, the small one feels substantially heavier than the (equally-weighted) larger one – an effect that is persistent and apparently cognitively impenetrable. The mechanism that underpins this illusion is still something of a mystery. It has long been contended (in a rather vague way) that the illusion is caused by the violation of an individual's expectations about how heavy each object will be – namely, the expectation that the large objects will outweigh the small objects (Ross Reference Ross1969). It is not difficult to imagine how this prior is built up, given the consistency of the relationship between size and weight outside of the laboratory setting. It is repeatedly encountering this positive size/weight relationship throughout our entire lives that presumably serves to establish a very powerful prior for our perceptions of heaviness (Flanagan et al. Reference Flanagan, Bittner and Johansson2008). Crucially, however, this prior is not integrated into the lifter's percept of how heavy the objects feel, as one might predict from a Bayesian optimal integration standpoint. Instead, the lifter's conscious perception of heaviness contrasts the prior expectation, leading some authors to label the effect as “anti-Bayesian” (Brayanov & Smith Reference Brayanov and Smith2010). Variants of the SWI can even manifest in a single, unchanging, object, which can be made to feel different weights by simply manipulating an individual's expectations of what they are about to lift (Buckingham & Goodale Reference Buckingham and Goodale2010).
The functional significance of this contrastive effect has been the source of great (and largely unresolved) debate – why would our perceptual system be so stricken with errors? Extending the conclusions of a recent study by Baugh and colleagues (Baugh et al. Reference Baugh, Kao, Johansson and Flanagan2012), it could be proposed that the SWI is a product of a perceptual system specialised for the detection and subsequent flagging of outliers in the statistics of the environment. Thus, conscious weight perception can be framed as an example of a task where it is important to emphasise the unexpected nature of the stimuli, in a system which presumably favours more efficient coding of information.
As lifting behaviour is a largely predictive process, our fingertip forces are driven by our expectations of how heavy something looks. And, in a more conventional Bayesian fashion, the weighting of these priors is rapidly adjusted (or rapidly ignored) by the presence of lifting errors. This provides the sensorimotor system with the best of both worlds – lifting behaviour that is flexible enough to rapidly adapt to constantly changing environments (e.g., a bottle of water which is being emptied by a thirsty drinker), but will automatically “snap back” to the (generally correct) lifting forces when the context of the lift is altered (so that the next time a fresh bottle of water is grasped, the sensorimotor prediction will have a good chance of being accurate). Thus, when lifting SWI-inducing cubes for the first time, lifters will apply excess force to the large cube and apply insufficient force to the small cube the first time they lift them, but will lift these two identically-weighted cubes with appropriately identical forces after only a few experiences with them (Flanagan & Beltzner Reference Flanagan and Beltzner2000). Clearly, this adaptive behaviour is a consequence of a complex interaction between short-term and long-term priors (Flanagan et al. Reference Flanagan, Bittner and Johansson2008) – a process that looks far more like the Bayesian processes outlined by Clark in his target article (Brayanov & Smith Reference Brayanov and Smith2010). It is tempting to ascribe a causal relationship between the force errors and the perceptual ones. Remarkably, however, the two kinds of errors appear to be completely isolated from one another: The magnitude of the SWI remains constant from one trial to the next, even in the face of the rapid trial-to-trial adaptation of the gripping and lifting forces. This complicates the situation even further by suggesting that there must be independent sets of priors for motor control and perceptual/cognitive judgements, which ultimately serve quite different functions.
In conclusion, we have outlined how the deceptively simple SWI paradigm can uncover the operation of multiple priors operating simultaneously, with different weightings and different goals. It is worth noting, however, that while the predictive brain makes sense in a post-hoc way, providing a computationally plausible parameter for both the perceptual and lifting effects (Brayanov & Smith Reference Brayanov and Smith2010), it is still very much a black-box explanation – and, to date, the term “prior” seems to serve only as a convenient placeholder in lieu of any tangible mechanism linking expectations to the perceptual or motor effects they appear to entail.
In his target article, Clark paints a diverse picture of how prediction is a ubiquitous part of brain and behaviour interactions. Taking heavy cues from Friston's “free energy principle,” his target article summarises ideas at the neural level, suggesting that the critical variable for sensory coding and motor control is the deviation from the expected signal, rather than the sensory or motor processing per se. In the field of sensorimotor control, this Bayesian approach is a popular one (e.g., Körding & Wolpert Reference Körding and Wolpert2004). Many researchers have built their careers showing that, in a wide range of contexts, an individual's motor behaviour can be modeled as the approximately optimal combination of the “undiluted” sensory input and the prior probability of that sensory event occurring, thus biasing the response one way or the other. Similarly, a wide range of psychophysical experiments have demonstrated that our conscious perception of events in the world represents not veridical sensory input, but the integration of multiple sources of evidence from our sensory system and our prior experience, rather than the veridical (and noisy) sensory input itself (Gregory Reference Gregory1998). An especially compelling case for this Bayesian standpoint can be made from the study of perceptual illusions, and several classic visual illusions can be explained with this optimal integration strategy (Geisler & Kersten Reference Geisler and Kersten2002; Weiss et al. Reference Weiss, Simoncelli and Adelson2002). In these contexts, this integration is thought to overcome the noise in the system of our sensory organs, maximising the likelihood of perceptual or motor “success.”
Despite the apparent descriptive power of optimally combining sensory prediction with sensory input, there are common situations where conscious perception is clearly not a product of Bayesian-style optimal integration. In fact, when we lift an object and experience its weight, our conscious perception of how heavy it feels is almost exactly the opposite of what might be expected if a perceiver integrates perpetual priors with sensory input. This incongruence is easily demonstrated with the famous size–weight illusion (SWI), first described in 1891 by Augustin Charpentier (translation by Murray et al. Reference Murray, Ellis, Bandomir and Ross1999). The SWI occurs when small and large objects, that otherwise look similar to one another, are adjusted to have identical weights. When individuals lift these objects, the small one feels substantially heavier than the (equally-weighted) larger one – an effect that is persistent and apparently cognitively impenetrable. The mechanism that underpins this illusion is still something of a mystery. It has long been contended (in a rather vague way) that the illusion is caused by the violation of an individual's expectations about how heavy each object will be – namely, the expectation that the large objects will outweigh the small objects (Ross Reference Ross1969). It is not difficult to imagine how this prior is built up, given the consistency of the relationship between size and weight outside of the laboratory setting. It is repeatedly encountering this positive size/weight relationship throughout our entire lives that presumably serves to establish a very powerful prior for our perceptions of heaviness (Flanagan et al. Reference Flanagan, Bittner and Johansson2008). Crucially, however, this prior is not integrated into the lifter's percept of how heavy the objects feel, as one might predict from a Bayesian optimal integration standpoint. Instead, the lifter's conscious perception of heaviness contrasts the prior expectation, leading some authors to label the effect as “anti-Bayesian” (Brayanov & Smith Reference Brayanov and Smith2010). Variants of the SWI can even manifest in a single, unchanging, object, which can be made to feel different weights by simply manipulating an individual's expectations of what they are about to lift (Buckingham & Goodale Reference Buckingham and Goodale2010).
The functional significance of this contrastive effect has been the source of great (and largely unresolved) debate – why would our perceptual system be so stricken with errors? Extending the conclusions of a recent study by Baugh and colleagues (Baugh et al. Reference Baugh, Kao, Johansson and Flanagan2012), it could be proposed that the SWI is a product of a perceptual system specialised for the detection and subsequent flagging of outliers in the statistics of the environment. Thus, conscious weight perception can be framed as an example of a task where it is important to emphasise the unexpected nature of the stimuli, in a system which presumably favours more efficient coding of information.
As lifting behaviour is a largely predictive process, our fingertip forces are driven by our expectations of how heavy something looks. And, in a more conventional Bayesian fashion, the weighting of these priors is rapidly adjusted (or rapidly ignored) by the presence of lifting errors. This provides the sensorimotor system with the best of both worlds – lifting behaviour that is flexible enough to rapidly adapt to constantly changing environments (e.g., a bottle of water which is being emptied by a thirsty drinker), but will automatically “snap back” to the (generally correct) lifting forces when the context of the lift is altered (so that the next time a fresh bottle of water is grasped, the sensorimotor prediction will have a good chance of being accurate). Thus, when lifting SWI-inducing cubes for the first time, lifters will apply excess force to the large cube and apply insufficient force to the small cube the first time they lift them, but will lift these two identically-weighted cubes with appropriately identical forces after only a few experiences with them (Flanagan & Beltzner Reference Flanagan and Beltzner2000). Clearly, this adaptive behaviour is a consequence of a complex interaction between short-term and long-term priors (Flanagan et al. Reference Flanagan, Bittner and Johansson2008) – a process that looks far more like the Bayesian processes outlined by Clark in his target article (Brayanov & Smith Reference Brayanov and Smith2010). It is tempting to ascribe a causal relationship between the force errors and the perceptual ones. Remarkably, however, the two kinds of errors appear to be completely isolated from one another: The magnitude of the SWI remains constant from one trial to the next, even in the face of the rapid trial-to-trial adaptation of the gripping and lifting forces. This complicates the situation even further by suggesting that there must be independent sets of priors for motor control and perceptual/cognitive judgements, which ultimately serve quite different functions.
In conclusion, we have outlined how the deceptively simple SWI paradigm can uncover the operation of multiple priors operating simultaneously, with different weightings and different goals. It is worth noting, however, that while the predictive brain makes sense in a post-hoc way, providing a computationally plausible parameter for both the perceptual and lifting effects (Brayanov & Smith Reference Brayanov and Smith2010), it is still very much a black-box explanation – and, to date, the term “prior” seems to serve only as a convenient placeholder in lieu of any tangible mechanism linking expectations to the perceptual or motor effects they appear to entail.