Lake et al. cogently argue that artificial intelligence (AI) machines would benefit from more “reverse engineering” of the human brain and its cognitive systems. However, it may be useful to invert this logic and, in particular, to use basic principles of machine communication to provide a menu of analogies and, perhaps, mechanisms that could be investigated in human brains and cognition.
We should consider that one of the missing components in deep learning models of cognition – and of most large-scale models of brain and cognitive function – is an understanding of how signals are selectively routed to different destinations in brains (Graham Reference Graham2014; Graham and Rockmore Reference Graham and Rockmore2011).
Given that brain cells themselves are not motile enough to selectively deliver messages to their destination (unlike cells in the immune system, for example), there must be a routing protocol of some kind in neural systems to accomplish this. This protocol should be relatively fixed in a given species and lineage, and have the ability to be scaled up over development and evolution.
Turning to machine communication as a model, each general technological strategy has its advantages and ideal operating conditions (grossly summarized here for brevity):
Circuit switched (traditional landline telephony): high throughput of dense real-time signals
Message switched (postal mail): multiplexed, verifiable, compact addresses
Packet switched (Internet): dynamic routing, sparse connectivity, fault tolerance, scalability
We should expect that brains adopt analogous – if not homologous – solutions when conditions require. For example, we would expect something like circuit switching in somatosensory and motor output systems, which tend to require dense, real-time communication. However, we would expect a dynamic, possibly packet-switched system in the visual system, given limited windows of attention and acuity and the need for spatial remapping, selectivity, and invariance (Olshausen et al. Reference Olshausen, Anderson and Van Essen1993; Poggio Reference Poggio1984; Wiskott Reference Wiskott, Van Hemmen and Sejnowski2006; Wiskott and von der Malsburg Reference Wiskott, von der Malsburg, Sirosh, Miikkulainen and Choe1996).
There could be hybrid routing architectures at work in brains and several that act concurrently (consider by way of analogy that it was possible until recently for a single human communicator to use the three switching protocols described above simultaneously). Individual components of a given routing system could also be selectively employed in brains. For example, Fornito et al. (Reference Fornito, Zalesky and Bullmore2016) proposed a mechanism of deflection routing (which is used to reroute signals around damaged or congested nodes), to explain changes in functional connectivity following focal lesions.
Nevertheless, functional demands in human cognitive systems appear to require a dynamic mechanism that could resemble a packet-switched system (Schlegel et al. Reference Schlegel, Alexander and Peter2015). As Lake et al. note, the abilities of brains to (1) grow and develop over time and (2) flexibly, creatively, and quickly adapt to new events are essential to their function. Packet switching as a general strategy may be more compatible with these requirements than alternative architectures.
In terms of growth, the number of Internet hosts – each of which can potentially communicate with any other within milliseconds – has increased without major disruption over a few decades, to surpass the number of neurons in the cortex of many primates including the macaque (Fasolo Reference Fasolo2011). This growth has also been much faster than the growth of the message-switched U.S. Postal Service (Giambene Reference Giambene2005; U.S. Postal Service 2016). Cortical neurons, like Internet hosts, are separated by relatively short network distances, and have the potential for communication along many possible routes within milliseconds. Communication principles that allowed for the rapid rise and sustained development of the packet-switched Internet may provide insights relevant to understanding how evolution and development conspire to generate intelligent brains.
In terms of adapting quickly to new situations, Lake et al. point out that a fully trained artificial neural network generally cannot take on new or different tasks without substantial retraining and reconfiguration. Perhaps this is not so much a problem of computation, but rather one of routing: in neural networks, one commonly employs a fixed routing system, all-to-all connectivity between layers, and feedback only between adjacent layers. These features may make such systems well suited to learning a particular input space, but ill suited to flexible processing and efficient handling of new circumstances. Although a packet-switched routing protocol would not necessarily improve current deep learning systems, it may be better suited to modeling approaches that more closely approximate cortical networks' structure and function. Unlike most deep learning networks, the brain appears to largely show dynamic routing, sparse connectivity, and feedback among many hierarchical levels. Including such features in computational models may better approximate and explain biological function, which could in turn spawn better AI.
Progress in understanding routing in the brain is already being made through simulations of dynamic signal flow on brain-like networks and in studies of brains themselves. Mišić et al. (Reference Mišić, Sporns and McIntosh2014) have investigated how Markovian queuing networks (a form of message-switched architecture) with primate brain-like connectivity could take advantage of small-world and rich-club topologies. Complementing this work, Sizemore et al. (Reference Sizemore, Giusti, Betzel and Bassett2016) have shown that the abundance of weakly interconnected brain regions suggests a prominent role for parallel processing, which would be well suited to dynamic routing. Using algebraic topology, Sizemore et al. (Reference Sizemore, Giusti, Betzel and Bassett2016) provide evidence that human brains show loops of converging or diverging signal flow (see also Granger Reference Granger2006). In terms of neurophysiology, Briggs and Usrey (Reference Briggs and Usrey2007) have shown that corticothalamic networks can pass signals in a loop in just 37 milliseconds. Such rapid feedback is consistent with the notion that corticothalamic signals could function like the “ack” (acknowledgment) system used on the Internet to ensure packet delivery (Graham Reference Graham2014; Graham and Rockmore Reference Graham and Rockmore2011).
In conclusion, it is suggested that an additional “core ingredient of human intelligence” is dynamic information routing of a kind that may mirror the packet-switched Internet, and cognitive scientists and computer engineers alike should be encouraged to investigate this possibility.
You have
Access
Lake et al. cogently argue that artificial intelligence (AI) machines would benefit from more “reverse engineering” of the human brain and its cognitive systems. However, it may be useful to invert this logic and, in particular, to use basic principles of machine communication to provide a menu of analogies and, perhaps, mechanisms that could be investigated in human brains and cognition.
We should consider that one of the missing components in deep learning models of cognition – and of most large-scale models of brain and cognitive function – is an understanding of how signals are selectively routed to different destinations in brains (Graham Reference Graham2014; Graham and Rockmore Reference Graham and Rockmore2011).
Given that brain cells themselves are not motile enough to selectively deliver messages to their destination (unlike cells in the immune system, for example), there must be a routing protocol of some kind in neural systems to accomplish this. This protocol should be relatively fixed in a given species and lineage, and have the ability to be scaled up over development and evolution.
Turning to machine communication as a model, each general technological strategy has its advantages and ideal operating conditions (grossly summarized here for brevity):
Circuit switched (traditional landline telephony): high throughput of dense real-time signals
Message switched (postal mail): multiplexed, verifiable, compact addresses
Packet switched (Internet): dynamic routing, sparse connectivity, fault tolerance, scalability
We should expect that brains adopt analogous – if not homologous – solutions when conditions require. For example, we would expect something like circuit switching in somatosensory and motor output systems, which tend to require dense, real-time communication. However, we would expect a dynamic, possibly packet-switched system in the visual system, given limited windows of attention and acuity and the need for spatial remapping, selectivity, and invariance (Olshausen et al. Reference Olshausen, Anderson and Van Essen1993; Poggio Reference Poggio1984; Wiskott Reference Wiskott, Van Hemmen and Sejnowski2006; Wiskott and von der Malsburg Reference Wiskott, von der Malsburg, Sirosh, Miikkulainen and Choe1996).
There could be hybrid routing architectures at work in brains and several that act concurrently (consider by way of analogy that it was possible until recently for a single human communicator to use the three switching protocols described above simultaneously). Individual components of a given routing system could also be selectively employed in brains. For example, Fornito et al. (Reference Fornito, Zalesky and Bullmore2016) proposed a mechanism of deflection routing (which is used to reroute signals around damaged or congested nodes), to explain changes in functional connectivity following focal lesions.
Nevertheless, functional demands in human cognitive systems appear to require a dynamic mechanism that could resemble a packet-switched system (Schlegel et al. Reference Schlegel, Alexander and Peter2015). As Lake et al. note, the abilities of brains to (1) grow and develop over time and (2) flexibly, creatively, and quickly adapt to new events are essential to their function. Packet switching as a general strategy may be more compatible with these requirements than alternative architectures.
In terms of growth, the number of Internet hosts – each of which can potentially communicate with any other within milliseconds – has increased without major disruption over a few decades, to surpass the number of neurons in the cortex of many primates including the macaque (Fasolo Reference Fasolo2011). This growth has also been much faster than the growth of the message-switched U.S. Postal Service (Giambene Reference Giambene2005; U.S. Postal Service 2016). Cortical neurons, like Internet hosts, are separated by relatively short network distances, and have the potential for communication along many possible routes within milliseconds. Communication principles that allowed for the rapid rise and sustained development of the packet-switched Internet may provide insights relevant to understanding how evolution and development conspire to generate intelligent brains.
In terms of adapting quickly to new situations, Lake et al. point out that a fully trained artificial neural network generally cannot take on new or different tasks without substantial retraining and reconfiguration. Perhaps this is not so much a problem of computation, but rather one of routing: in neural networks, one commonly employs a fixed routing system, all-to-all connectivity between layers, and feedback only between adjacent layers. These features may make such systems well suited to learning a particular input space, but ill suited to flexible processing and efficient handling of new circumstances. Although a packet-switched routing protocol would not necessarily improve current deep learning systems, it may be better suited to modeling approaches that more closely approximate cortical networks' structure and function. Unlike most deep learning networks, the brain appears to largely show dynamic routing, sparse connectivity, and feedback among many hierarchical levels. Including such features in computational models may better approximate and explain biological function, which could in turn spawn better AI.
Progress in understanding routing in the brain is already being made through simulations of dynamic signal flow on brain-like networks and in studies of brains themselves. Mišić et al. (Reference Mišić, Sporns and McIntosh2014) have investigated how Markovian queuing networks (a form of message-switched architecture) with primate brain-like connectivity could take advantage of small-world and rich-club topologies. Complementing this work, Sizemore et al. (Reference Sizemore, Giusti, Betzel and Bassett2016) have shown that the abundance of weakly interconnected brain regions suggests a prominent role for parallel processing, which would be well suited to dynamic routing. Using algebraic topology, Sizemore et al. (Reference Sizemore, Giusti, Betzel and Bassett2016) provide evidence that human brains show loops of converging or diverging signal flow (see also Granger Reference Granger2006). In terms of neurophysiology, Briggs and Usrey (Reference Briggs and Usrey2007) have shown that corticothalamic networks can pass signals in a loop in just 37 milliseconds. Such rapid feedback is consistent with the notion that corticothalamic signals could function like the “ack” (acknowledgment) system used on the Internet to ensure packet delivery (Graham Reference Graham2014; Graham and Rockmore Reference Graham and Rockmore2011).
In conclusion, it is suggested that an additional “core ingredient of human intelligence” is dynamic information routing of a kind that may mirror the packet-switched Internet, and cognitive scientists and computer engineers alike should be encouraged to investigate this possibility.