We agree with the claim of Lake et al. that to obtain human-level learning speed and cognitive flexibility, future artificial-intelligence (AI) systems will have to incorporate key elements of human cognition: from causal models of the world, to intuitive psychological theories, compositionality, and knowledge transfer. However, the authors largely overlook the importance of a major challenge to implementation of the functions they advocate: the need to develop sophisticated architectures to learn, represent, and process the knowledge related to those functions. Here we call this the architecture challenge. In this commentary, we make two claims: (1) tackling the architecture challenge is fundamental to success in developing human-level AI systems; (2) looking at the brain can furnish important insights on how to face the architecture challenge.

The difficulty of the architecture challenge stems from the fact that the space of the architectures needed to implement the several functions advocated by Lake et al. is huge. The authors get close to this problem when they recognize that one thing that the enormous genetic algorithm of evolution has done in millions of years of the stochastic hill-climbing search is to develop suitable brain architectures. One possible way to attack the architecture challenge, also mentioned by Lake et al., would be to use evolutionary techniques mimicking evolution. We think that today this strategy is out of reach, given the “ocean-like” size of the search space. At most, we can use such techniques to explore small, interesting “islands lost within the ocean.” But how do we find those islands in the first place? We propose looking at the architecture of real brains, the product of the evolution genetic algorithm, and try to “steal insights” from nature. Indeed, we think that much of the intelligence of the brain resides in its architecture. Obviously, identifying the proper insights is not easy to do, as the brain is very difficult to understand. However, it might be useful to try, as the effort might give us at least some general indications, a compass, to find the islands in the ocean. Here we present some examples to support our intuition.

When building architectures of AI systems, even when following cognitive science indications (e.g., Franklin Reference Franklin, Want and Goertzel2007), the tendency is to “divide and conquer,” that is, to list the needed high-level functions, implement a module for each of them, and suitably interface the modules. However, the organisation of the brain can be understood on the basis of not only high-level functions (see below), but also “low-level” functions (usually called “mechanisms”). An example of a mechanism is brain organisation based on macro-structures, each having fine repeated micro-architectures implementing specific computations and learning processes (Caligiore et al. Reference Caligiore, Pezzulo, Baldassarre, Bostan, Strick, Doya, Helmich, Dirkx, Houk, Jörntell, Lago-Rodriguez, Galea, Miall, Popa, Kishore, Verschure, Zucca and Herreros2016; Doya Reference Doya1999): the cortex to statically and dynamically store knowledge acquired by associative learning processes (Penhune & Steele Reference Penhune and Steele2012; Shadmehr & Krakauer Reference Shadmehr and Krakauer2008), the basal ganglia to learn to select information by reinforcement learning (Graybiel Reference Graybiel2005; Houk et al. Reference Houk, Adams, Barto, Houk, Davids and Beiser1995), the cerebellum to implement fast time-scale computations possibly acquired with supervised learning (Kawato et al. Reference Kawato, Kuroda and Schweighofer2011; Wolpert et al. Reference Wolpert, Miall and Kawato1998), and the limbic brain structures interfacing the brain to the body and generating motivations, emotions, and the value of things (Mirolli et al. Reference Mirolli, Mannella and Baldassarre2010; Mogenson et al. Reference Mogenson, Jones and Yim1980). Each of these mechanisms supports multiple, high-level functions (see below).

Brain architecture is also forged by the fact that natural intelligence is strongly embodied and situated (an aspect not much stressed by Lake et al.); that is, it is shaped to adaptively interact with the physical world (Anderson Reference Anderson2003; Pfeifer & Gómez Reference Pfeifer, Gómez, Sendhoff, Körner, Ritter and Doya2009) to satisfy the organism's needs and goals (Mannella et al. Reference Mannella, Gurney and Baldassarre2013). Thus, the cortex is organised along multiple cortical pathways running from sensors to actuators (Baldassarre et al. Reference Baldassarre, Caligiore, Mannella, Baldassarre and Mirolli2013a) and “intercepted” by the basal ganglia selective processes in their last part closer to action (Mannella & Baldassarre Reference Mannella and Baldassarre2015). These pathways are organised in a hierarchical fashion, with the higher ones that process needs and motivational information controlling the lower ones closer to sensation/action. The lowest pathways dynamically connect musculoskeletal body proprioception with primary motor areas (Churchland et al. Reference Churchland, Cunningham, Kaufman, Foster, Nuyujukian, Ryu and Shenoy2012). Higher-level “dorsal” pathways control the lowest pathways by processing visual/auditory information used to interact with the environment (Scott Reference Scott2004). Even higher-level “ventral” pathways inform the brain on the identity and nature of resources in the environment to support decisions (Caligiore et al. Reference Caligiore, Borghi, Parisi and Baldassarre2010; Milner & Goodale Reference Milner and Goodale2006). At the hierarchy apex, the limbic brain supports goal selection based on visceral, social, and other types of needs/goals. Embedded within the higher pathways, an important structure involving basal ganglia–cortical loops learns and implements stimulus–response habitual behaviours (used to act in familiar situations) and goal-directed behaviours (important for problem solving and planning when new challenges are encountered) (Baldassarre et al. Reference Baldassarre, Mannella, Fiore, Redgrave, Gurney and Mirolli2013b; Mannella et al. Reference Mannella, Gurney and Baldassarre2013). These brain structures form a sophisticated network, knowledge of which might help in designing the architectures of human-like embodied AI systems able to act in the real world.

A last example of the need for sophisticated architectures starts with the recognition by Lake et al. that we need to endow AI systems with a “developmental start-up software.” In this respect, together with other authors (e.g., Weng et al. Reference Weng, McClelland, Pentland, Sporns, Stockman, Sur and Thelen2001; see Baldassarre et al. Reference Baldassarre, Mannella, Fiore, Redgrave, Gurney and Mirolli2013b; Reference Baldassarre, Stafford, Mirolli, Redgrave, Ryan and Barto2014, for collections of works) we believe that human-level intelligence can be achieved only through open-ended learning, that is, the cumulative learning of progressively more complex skills and knowledge, driven by intrinsic motivations, which are motivations related to the acquisition of knowledge and skills rather than material resources (Baldassarre Reference Baldassarre, Cangelosi, Triesch, Fasel, Rohlfing, Nori, Oudeyer, Schlesinger and Nagai2011). The brain (e.g., Lisman & Grace Reference Lisman and Grace2005; Redgrave & Gurney Reference Redgrave and Gurney2006) and computational theories and models (e.g., Baldassarre & Mirolli Reference Baldassarre and Mirolli2013; Baldassarre et al. Reference Baldassarre, Stafford, Mirolli, Redgrave, Ryan and Barto2014; Santucci et al. Reference Santucci, Baldassarre and Mirolli2016) indicate how the implementation of these processes indeed requires very sophisticated architectures able to store multiple skills, to transfer knowledge while avoiding catastrophic interference, to explore the environment based on the acquired skills, to self-generate goals/tasks, and to focus on goals that ensure a maximum knowledge gain.