When observers are asked to adjust an object with a typical colour (e.g., a yellow banana) to grey in an achromatic adjustment task, they adjust it slightly to the colour opposite to the typical colour (e.g., blue). This result implies that observers still perceive remnants of the typical colour of the object when the object is shown at a chromaticity that would be considered grey otherwise. And that shows that the knowledge about the typical colour of an object influences the perceived colour of that object (Hansen et al. Reference Hansen, Olkkonen, Walter and Gegenfurtner2006; Olkkonen et al. Reference Olkkonen, Hansen and Gegenfurtner2008; Witzel et al. Reference Witzel, Valkova, Hansen and Gegenfurtner2011).

In contrast to earlier work on memory colour, including Duncker (Reference Duncker1939) and Bruner et al. (Reference Bruner, Postman and Rodrigues1951), we particularly designed our achromatic adjustment method to circumvent problems related to judgement, memory, and response biases. It is important to note that Firestone & Scholl (F&S) did not correctly state our methods and findings. The banana was not “judged to be more than 20% yellow” (sect. 4.4.1, para. 3) at the neutral point; instead, observers needed to adjust the banana 20% in the “blue” direction to make it appear neutral. Yellow judgments would naturally be prone to judgement biases, whereas our nulling method is not, because participants are not asked to implicitly or explicitly rate the object colours. Instead, the achromatic adjustment task involves a genuinely perceptual comparison between the colour of the objects and the grey background to which the observers were adapted (Pitfall 2, “perception versus judgment,” and Pitfall 6, “memory and recognition”).

To avoid response biases, we presented the images in random colours at the beginning of each trial (Pitfall 3, “demand and response bias”). Doing so prevented a strategy of merely overshooting in the opposite colour direction, thus producing a spurious memory colour effect (Witzel & Hansen Reference Witzel, Hansen, Elliot, Fairchild and Franklin2015). Even with this precaution, the observed effects went specifically in the opposite direction of the typical memory colours.

We carefully controlled our stimuli in their low-level, sensory characteristics (Pitfall 4, “low-level differences”). In contrast to F&S's general critique about the lack of control in luminance (sect. 4.4.1, para. 3), stimuli in the memory colour experiments were matched in average luminance (Hansen et al. Reference Hansen, Olkkonen, Walter and Gegenfurtner2006; Olkkonen et al. Reference Olkkonen, Hansen and Gegenfurtner2008; Witzel et al. Reference Witzel, Valkova, Hansen and Gegenfurtner2011). Moreover, the control stimuli used to establish observer's grey adjustments independent of memory colour effects were matched in spatial and chromatic low-level properties with the colour-diagnostic images.

We also carefully explored the conditions under which the memory colour effect does not occur, providing “uniquely disconfirmatory predictions” (Pitfall 1, “an overly confirmatory research strategy,” sect. 4.1). Objects without a memory colour and objects with achromatic (greyscale) memory colours, such as a striped sock and a white golf ball, do not produce any shift in grey adjustments (Witzel & Hansen, Reference Witzel, Hansen, Elliot, Fairchild and Franklin2015; Witzel et al. Reference Witzel, Valkova, Hansen and Gegenfurtner2011). Moreover, the effect lessens when decreasing characteristic features of the objects, such as in uniformly painted objects and outline shapes (Olkkonen et al. Reference Olkkonen, Hansen and Gegenfurtner2008; see also Fig. 1 in Witzel et al. Reference Witzel, Valkova, Hansen and Gegenfurtner2011).

Finally, the task required observers to pay attention to the image in order to complete the grey adjustment, independent of whether the image showed a colour-diagnostic object or a control object (Pitfall 5, “peripheral attentional effects”). Apart from that, there is no reason a priori to assume that shifts of attention away from the stimulus would produce spurious memory colour effects.

We are left to explain why the greyscale image of the banana in the target article's Figure 2K does not appear yellow. The sensory signal coming from that figure unambiguously establishes that the colour difference between the leftmost and the rightmost banana is a difference between grey and yellow. The memory colour effect is more subtle and cannot compete with the unambiguous sensory information in Figure 2K (cf. our Fig. 1A). Contrary to Figure 2K, our method allows for detecting the small but systematic deviations of the grey perceived for example on a banana from the grey perceived on a control stimulus. These systematic deviations towards blue show that the recognition of the object as being a banana provides additional evidence for it being yellow that is combined with sensory evidence about the contrast between the adjusted colour and the grey background.

Figure 1. (A) Illustration of the memory colour effect: the banana from Hansen et al. (2006) when it has the same chromaticity as the background (left) and when it has the average chromaticity that observers adjusted to make it appear grey (right). (B) Bayesian model of the memory colour effect. Hypothetical reliability of the sensory signal (blue line) and memory reliability (red line) for the typical yellow of a banana. The Bayesian combination of the two sources of information (grey line) predicts a shift in the perception of grey (at zero) towards yellow that corresponds to the memory colour effect. The observers compensate for this yellow shift in the percept (dotted vertical line) by adjusting the image towards blue.

In vision science, combining different types of evidence is most elegantly considered in a Bayesian framework (Maloney & Mamassian Reference Maloney and Mamassian2009). Consider our Figure 1b: When the images are achromatic, the sensory signal (blue curve) indicates greyness with a certain level of reliability. At the same time, prior knowledge about the typical colour of the object suggests that the object is likely to be coloured in its typical colour (red curve). Because sensory signals always contain uncertainty, combining sensory evidence with prior knowledge is a useful strategy to constrain perceptual estimates. As a result of the combination of sensory signals and prior knowledge in a Bayesian ideal observer model, the perceptual estimate of the colour (grey curve) shifts towards the typical colour of the object. When an observer is asked to make the object to appear grey, the colour setting needs to shift towards the opposite direction, thus producing the memory colour effect.

Whether memory colour effects are an example of top-down effects in the sense of cognitive penetrability of perception depends on the definition of perception and cognition (Witzel & Hansen Reference Witzel, Hansen, Elliot, Fairchild and Franklin2015). We believe the notion that colour appearance is “low-level” whereas object recognition and memory are “high-level” (Eacott & Heywood Reference Eacott and Heywood1995) is too simplified. In any case, evidence for the memory colour effects has also been observed in neuroimaging experiments (Bannert & Bartels Reference Bannert and Bartels2013; Vandenbroucke et al. Reference Vandenbroucke, Fahrenfort, Meuwese, Scholte and Lamme2014) in early visual cortex, indicating that no matter at what stage they arise, they get propagated back to the early visual system.