A fundamental tenet of the integrative memory model proposed by Bastin et al. is the distinction between entity representation and relational representations. This distinction is based on the premise that there is a meaningful sense of an “entity” that is holistic and unitized and can be stored and retrieved without recollection or binding; for example, Bastin et al.’s claim that at the “level of the perirhinal cortex and anterolateral entorhinal all visual features are integrated in a single complex representation of the object that can be discriminated from other objects with overlapping features.”
It is natural to think that meaningful visual objects might be unitized entities for visual memory, and indeed many memory researchers take this claim for granted. However, we believe this view of unitized, fully bound representation of objects is inconsistent with the cognitive data on object memory. In particular, supposed “entities” such as visual objects are not unitized representations, but themselves are stored as separate features requiring binding in the same fundamental way that objects need to be bound to contexts. This calls into question the core distinction between entities and relations proposed by the integrative memory model and other similar models: If even single objects are stored in a way consistent with the “relational system” – where the “representation keeps components separate and flexibly bound” (target article, sect. 4.1, para. 7) – then it is not clear what an entity would be or whether the fundamental nature of the brain regions subserving object memory are really distinct from those subserving item-context integration.
Work from our labs shows that visual object features are stored and accessed independently in long-term memory: Different features of single individual objects are forgotten at different rates (Brady et al. Reference Brady, Konkle, Alvarez and Oliva2013); and people remember particular features but don't remember which objects these features belonged to (Utochkin & Brady Reference Utochkin and Brady2019). For example, if people are shown a blue open backpack and then asked to choose among sets of four backpacks that are blue/open, blue/closed, or red/open, red/closed, people can forget the color but still can remember its “open-ness” (Brady et al. Reference Brady, Konkle, Alvarez and Oliva2013). Or, if people are shown one mug (mug A) full of coffee and another mug (mug B) empty, they are well above chance reporting that they saw one full mug and one empty mug, and that they saw mug A and mug B, but they are at chance at ascribing the “fullness” and “emptiness” to the particular mugs A and B (Utochkin & Brady Reference Utochkin and Brady2019).
Indeed, the idea that some items might be stored in a fully unitized representation (in an “entity” system) seems incompatible with the cognitive problem that object representations are designed to solve. One of the central requirements of a visual memory system is robustness to variation (Schurgin & Flombaum Reference Schurgin and Flombaum2018). If object representations were totally integrated entities in memory, their recognition in the real world would be extremely problematic, given the infinite number of poses and states these objects can take, as well as variations in orientation, lighting, and more. One might argue that this invariant recognition is possible if a unitized “concept” of a particular object is formed during multiple episodes – connecting across multiple experiences when an object is presented in different states and viewpoints. But this claim immediately implies an independence of the features forming the core of this concept and those representing the way it changes across contexts.
In contrast to any view based on unitized object memories, we have demonstrated that people’s memories are extremely robust to variation even at the level of individual objects, suggesting that representations even at the level of objects are based on separate features that are flexibly bound together rather than unitized. For example, imagine you saw an open-doored cabinet, and then later we asked which cabinet you had seen – but now the “old” item was shown in a new state (the same cabinet now has its doors closed, changing a huge number of visual features). We have shown that people are nearly perfect at generalizing in this way, and can do so even if the “foil” presented at test is a new open-doored cabinet, designed to maximally mislead participants (Utochkin & Brady Reference Utochkin and Brady2019). Therefore, we believe the flexible nature of binding attributed by the integrative memory model only to item-context distinctions and recollection situations, need to be extended to nearly every level of representation of objects as well as contexts.
Similar evidence for independence and structured representation rather than unitized objects is present in the visual working memory literature, where it is frequently found that both objects and separate features can be stored and objects are not stored as single integrated units (see Brady et al. Reference Brady, Konkle and Alvarez2011 for review). Since working memory is critical for consolidation into long-term memory, this may be the beginning of the non-unitized, non-integral storage of items in memory.
Overall, we believe that entities are stored in a way that is not holistic or unitized – and thus, at nearly every level of representation, there is a need for flexible, relational encoding. If this is a common property of memories for individual objects (which is associated with “entities”) and complex episodes (requiring the involvement of “relational representations”), then is there a fundamental difference in the representation of these two kinds of information, as proposed by the authors? If there is, then how does one know where an entity ends and a relational representation begins? We believe our work and that of the rest of the visual memory community is more consistent with the idea that there is a hierarchy of representations, each requiring the storage of relational information and each allowing for the possibility of misbinding and other retrieval failures. Thus, rather than a strong dichotomy between entities and relational storage, the benefits of flexible, independent storage, and the resulting problem of binding features together, occur at every level of the hierarchy – from the simplest visual feature conjunctions to the binding of objects into contexts and into events.
You have
Access
A fundamental tenet of the integrative memory model proposed by Bastin et al. is the distinction between entity representation and relational representations. This distinction is based on the premise that there is a meaningful sense of an “entity” that is holistic and unitized and can be stored and retrieved without recollection or binding; for example, Bastin et al.’s claim that at the “level of the perirhinal cortex and anterolateral entorhinal all visual features are integrated in a single complex representation of the object that can be discriminated from other objects with overlapping features.”
It is natural to think that meaningful visual objects might be unitized entities for visual memory, and indeed many memory researchers take this claim for granted. However, we believe this view of unitized, fully bound representation of objects is inconsistent with the cognitive data on object memory. In particular, supposed “entities” such as visual objects are not unitized representations, but themselves are stored as separate features requiring binding in the same fundamental way that objects need to be bound to contexts. This calls into question the core distinction between entities and relations proposed by the integrative memory model and other similar models: If even single objects are stored in a way consistent with the “relational system” – where the “representation keeps components separate and flexibly bound” (target article, sect. 4.1, para. 7) – then it is not clear what an entity would be or whether the fundamental nature of the brain regions subserving object memory are really distinct from those subserving item-context integration.
Work from our labs shows that visual object features are stored and accessed independently in long-term memory: Different features of single individual objects are forgotten at different rates (Brady et al. Reference Brady, Konkle, Alvarez and Oliva2013); and people remember particular features but don't remember which objects these features belonged to (Utochkin & Brady Reference Utochkin and Brady2019). For example, if people are shown a blue open backpack and then asked to choose among sets of four backpacks that are blue/open, blue/closed, or red/open, red/closed, people can forget the color but still can remember its “open-ness” (Brady et al. Reference Brady, Konkle, Alvarez and Oliva2013). Or, if people are shown one mug (mug A) full of coffee and another mug (mug B) empty, they are well above chance reporting that they saw one full mug and one empty mug, and that they saw mug A and mug B, but they are at chance at ascribing the “fullness” and “emptiness” to the particular mugs A and B (Utochkin & Brady Reference Utochkin and Brady2019).
Indeed, the idea that some items might be stored in a fully unitized representation (in an “entity” system) seems incompatible with the cognitive problem that object representations are designed to solve. One of the central requirements of a visual memory system is robustness to variation (Schurgin & Flombaum Reference Schurgin and Flombaum2018). If object representations were totally integrated entities in memory, their recognition in the real world would be extremely problematic, given the infinite number of poses and states these objects can take, as well as variations in orientation, lighting, and more. One might argue that this invariant recognition is possible if a unitized “concept” of a particular object is formed during multiple episodes – connecting across multiple experiences when an object is presented in different states and viewpoints. But this claim immediately implies an independence of the features forming the core of this concept and those representing the way it changes across contexts.
In contrast to any view based on unitized object memories, we have demonstrated that people’s memories are extremely robust to variation even at the level of individual objects, suggesting that representations even at the level of objects are based on separate features that are flexibly bound together rather than unitized. For example, imagine you saw an open-doored cabinet, and then later we asked which cabinet you had seen – but now the “old” item was shown in a new state (the same cabinet now has its doors closed, changing a huge number of visual features). We have shown that people are nearly perfect at generalizing in this way, and can do so even if the “foil” presented at test is a new open-doored cabinet, designed to maximally mislead participants (Utochkin & Brady Reference Utochkin and Brady2019). Therefore, we believe the flexible nature of binding attributed by the integrative memory model only to item-context distinctions and recollection situations, need to be extended to nearly every level of representation of objects as well as contexts.
Similar evidence for independence and structured representation rather than unitized objects is present in the visual working memory literature, where it is frequently found that both objects and separate features can be stored and objects are not stored as single integrated units (see Brady et al. Reference Brady, Konkle and Alvarez2011 for review). Since working memory is critical for consolidation into long-term memory, this may be the beginning of the non-unitized, non-integral storage of items in memory.
Overall, we believe that entities are stored in a way that is not holistic or unitized – and thus, at nearly every level of representation, there is a need for flexible, relational encoding. If this is a common property of memories for individual objects (which is associated with “entities”) and complex episodes (requiring the involvement of “relational representations”), then is there a fundamental difference in the representation of these two kinds of information, as proposed by the authors? If there is, then how does one know where an entity ends and a relational representation begins? We believe our work and that of the rest of the visual memory community is more consistent with the idea that there is a hierarchy of representations, each requiring the storage of relational information and each allowing for the possibility of misbinding and other retrieval failures. Thus, rather than a strong dichotomy between entities and relational storage, the benefits of flexible, independent storage, and the resulting problem of binding features together, occur at every level of the hierarchy – from the simplest visual feature conjunctions to the binding of objects into contexts and into events.