The neural network revolution occurred more than 30 years ago, stirring intense debate over what neural networks (NNs) can and cannot learn and represent. Much of the target article resurrects these earlier concerns, but in the context of the latest NN revolution, spearheaded by an algorithm that was known, but failed because of scale and computational power, namely, deep learning (DL).
Claims that learning systems must build causal models and provide explanations of their inferences are not new (DeJong Reference DeJong and Mooney1986; Lenat Reference Lenat, Miller and Yokoi1995; Mitchell 1986), nor have they been proven successful. Advocating the idea that artificial intelligence (AI) systems need commonsense knowledge, ambitious projects such as “Cyc” (Lenat Reference Lenat and Guha1990) created hand-crafted and labor-intensive knowledge bases, combined with an inference engine to derive answers in the form of explicit knowledge. Despite feeding a large but finite number of factual assertions and explicit rules into such systems, the desired human-like performance was never accomplished. Other explanation-based and expert systems (e.g., WordNet [Miller Reference Miller, Beckwith, Fellbaum, Gross and Miller1990]) proved useful in some applied domains, but were equally unable to solve the problem of AI. At the essence of such projects lies the idea of “cognitive functionalism.” Proposing that mental states are functional states determined and individuated by their causal relations to other mental states and behaviors, it suggests that mental states are programmable with explicitly determined representational structures (Fodor, Reference Fodor1981; Hayes Reference Hayes1974; McCarthy & Hayes Reference McCarthy, Hayes, Meltzer and Michie1969; Putnam Reference Putnam, Capitan and Merrill1967). Such a view stresses the importance of “formalizing concepts of causality, ability, and knowledge” to create “a computer program that decides what to do by inferring in a formal language that a certain strategy will achieve its assigned goal” (McCarthy & Hayes, Reference McCarthy, Hayes, Meltzer and Michie1969, p. 1). Lake et al.'s appeal to causal mechanisms and their need for explicit model representations is closely related to this cognitive functionalism, which had been put forth as a set of principles by many founders of the AI field (Hayes Reference Hayes1974; McCarthy Reference McCarthy1959; McCarthy & Hayes Reference McCarthy, Hayes, Meltzer and Michie1969; Newell & Simon, Reference Newell and Simon1956).
One important shortcoming of cognitive functionalism is its failure to acknowledge that the same behavior/function may be caused by different representations and mechanisms (Block Reference Block1978; Hanson Reference Hanson, Chauvin and Rummelhart1995). Consequently, the problem with this proposition that knowledge within a learning system must be explicit is that it conflates the relationship between implicit knowledge and explicit knowledge and their representations. The ability to throw a low hanging fast ball would be difficult, if not impossible, to encode as a series of rules. However, this type of implicit knowledge can indeed be captured in a neural network, simply by having it learn from an analog perception–action system and a series of ball throws – all while also having the ability to represent rule-based knowledge (Horgan & Tienson Reference Horgan and Tienson1996). This associative versus rule learning debate, referred to in this article as “pattern recognition” versus “model building,” was shown a number of times to be a meaningless dichotomy (Hanson & Burr Reference Hanson and Burr1990; Hanson et al. Reference Hanson2002; Prasada & Pinker Reference Prasada and Pinker1993).
Although we agree with Lake et al. that “model building” is indeed an important component of any AI system, we do not agree that NNs merely recognize patterns and lack the ability to build models. Our disagreement arises from the presumption that “a model must include explicit representations of objects, identity and relations” (Lake et al. 2016, pp. 38–39). Rather than being explicit or absent altogether, model representation is implicit in NNs. Investigating implicitly learned models is somewhat more challenging, but work on learning dynamics and learning functions with respect to their relationship to representations provides insights into these implicit models (Caglar & Hanson Reference Caglar and Hanson2016; Cleeremans Reference Cleeremans1993; Hanson & Burr Reference Hanson and Burr1990; Metcalf et al. Reference Metcalfe, Cottrell and Mencl1992; Saxe et al. Reference Saxe, McClelland and Ganguli2014).
Recent work has shown that in DL, the internal structure, or “model,” accumulates at later layers, and is effectively constructing “scaffolds” over the learning process that are then used to train subsequent layers (Caglar & Hanson Reference Caglar and Hanson2016; Saxe Reference Saxe, McClelland and Ganguli2013). These learning dynamics can be investigated through analysis of the learning curves and the internal representations resultant in the hidden units. Analysis of the learning curves of NNs with different architectures reveals that merely adding depth to a NN results in different learning dynamics and representational structures, which do not require explicit preprogramming or pre-training (Caglar & Hanson Reference Caglar and Hanson2016). In fact, the shape of the learning curves for single-layer NNs and for multilayered DLs are qualitatively different, with the former fitting a negative exponential function (“associative”) and the latter fitting a hyperbolic function (“accumulative”). This type of structured learning, consistent with the shape of the learning curves, can be shown to be equivalent to the “learning-to-learn” component suggested by the authors. Appearing across different layers of the NNs, it also satisfies the need for “learning-to-learn to occur at multiple levels of the hierarchical generative process” (Lake et al., sect. 4.2.3, para. 5).
Furthermore, in category learning tasks with DLs, the internal representation of the hidden units shows that it creates prototype-like representations at each layer of the network (Caglar & Hanson Reference Caglar and Hanson2016). These higher-level representations are the result of concept learning from exemplars, and go far beyond simple pattern recognition. Additionally, the plateau characteristic of the hyperbolic learning curves provides evidence for rapid learning, as well as one-shot learning once this kind of implicit conceptual representation has been formed over some subset of exemplars (similar to a “prior”) (Saxe 2014). Longstanding investigation in the learning theory literature proposes that the hyperbolic learning curve of DLs is also the shape that best describes human learning (Mazur & Hastie Reference Mazur and Hastie1978; Thurstone Reference Thurstone1919), thereby suggesting that the learning mechanisms of DLs and humans might be more similar than thought (Hanso et al., in preparation).
Taken together, the analysis of learning curves and internal representations of hidden units indicates that NNs do in fact build models and create representational structures. However, these models are implicitly built into the learning process and cannot be explicitly dissociated from it. Exploiting the rich information of the stimulus and its context, the learning process creates models and shapes representational structures without the need for explicit preprogramming.
The neural network revolution occurred more than 30 years ago, stirring intense debate over what neural networks (NNs) can and cannot learn and represent. Much of the target article resurrects these earlier concerns, but in the context of the latest NN revolution, spearheaded by an algorithm that was known, but failed because of scale and computational power, namely, deep learning (DL).
Claims that learning systems must build causal models and provide explanations of their inferences are not new (DeJong Reference DeJong and Mooney1986; Lenat Reference Lenat, Miller and Yokoi1995; Mitchell 1986), nor have they been proven successful. Advocating the idea that artificial intelligence (AI) systems need commonsense knowledge, ambitious projects such as “Cyc” (Lenat Reference Lenat and Guha1990) created hand-crafted and labor-intensive knowledge bases, combined with an inference engine to derive answers in the form of explicit knowledge. Despite feeding a large but finite number of factual assertions and explicit rules into such systems, the desired human-like performance was never accomplished. Other explanation-based and expert systems (e.g., WordNet [Miller Reference Miller, Beckwith, Fellbaum, Gross and Miller1990]) proved useful in some applied domains, but were equally unable to solve the problem of AI. At the essence of such projects lies the idea of “cognitive functionalism.” Proposing that mental states are functional states determined and individuated by their causal relations to other mental states and behaviors, it suggests that mental states are programmable with explicitly determined representational structures (Fodor, Reference Fodor1981; Hayes Reference Hayes1974; McCarthy & Hayes Reference McCarthy, Hayes, Meltzer and Michie1969; Putnam Reference Putnam, Capitan and Merrill1967). Such a view stresses the importance of “formalizing concepts of causality, ability, and knowledge” to create “a computer program that decides what to do by inferring in a formal language that a certain strategy will achieve its assigned goal” (McCarthy & Hayes, Reference McCarthy, Hayes, Meltzer and Michie1969, p. 1). Lake et al.'s appeal to causal mechanisms and their need for explicit model representations is closely related to this cognitive functionalism, which had been put forth as a set of principles by many founders of the AI field (Hayes Reference Hayes1974; McCarthy Reference McCarthy1959; McCarthy & Hayes Reference McCarthy, Hayes, Meltzer and Michie1969; Newell & Simon, Reference Newell and Simon1956).
One important shortcoming of cognitive functionalism is its failure to acknowledge that the same behavior/function may be caused by different representations and mechanisms (Block Reference Block1978; Hanson Reference Hanson, Chauvin and Rummelhart1995). Consequently, the problem with this proposition that knowledge within a learning system must be explicit is that it conflates the relationship between implicit knowledge and explicit knowledge and their representations. The ability to throw a low hanging fast ball would be difficult, if not impossible, to encode as a series of rules. However, this type of implicit knowledge can indeed be captured in a neural network, simply by having it learn from an analog perception–action system and a series of ball throws – all while also having the ability to represent rule-based knowledge (Horgan & Tienson Reference Horgan and Tienson1996). This associative versus rule learning debate, referred to in this article as “pattern recognition” versus “model building,” was shown a number of times to be a meaningless dichotomy (Hanson & Burr Reference Hanson and Burr1990; Hanson et al. Reference Hanson2002; Prasada & Pinker Reference Prasada and Pinker1993).
Although we agree with Lake et al. that “model building” is indeed an important component of any AI system, we do not agree that NNs merely recognize patterns and lack the ability to build models. Our disagreement arises from the presumption that “a model must include explicit representations of objects, identity and relations” (Lake et al. 2016, pp. 38–39). Rather than being explicit or absent altogether, model representation is implicit in NNs. Investigating implicitly learned models is somewhat more challenging, but work on learning dynamics and learning functions with respect to their relationship to representations provides insights into these implicit models (Caglar & Hanson Reference Caglar and Hanson2016; Cleeremans Reference Cleeremans1993; Hanson & Burr Reference Hanson and Burr1990; Metcalf et al. Reference Metcalfe, Cottrell and Mencl1992; Saxe et al. Reference Saxe, McClelland and Ganguli2014).
Recent work has shown that in DL, the internal structure, or “model,” accumulates at later layers, and is effectively constructing “scaffolds” over the learning process that are then used to train subsequent layers (Caglar & Hanson Reference Caglar and Hanson2016; Saxe Reference Saxe, McClelland and Ganguli2013). These learning dynamics can be investigated through analysis of the learning curves and the internal representations resultant in the hidden units. Analysis of the learning curves of NNs with different architectures reveals that merely adding depth to a NN results in different learning dynamics and representational structures, which do not require explicit preprogramming or pre-training (Caglar & Hanson Reference Caglar and Hanson2016). In fact, the shape of the learning curves for single-layer NNs and for multilayered DLs are qualitatively different, with the former fitting a negative exponential function (“associative”) and the latter fitting a hyperbolic function (“accumulative”). This type of structured learning, consistent with the shape of the learning curves, can be shown to be equivalent to the “learning-to-learn” component suggested by the authors. Appearing across different layers of the NNs, it also satisfies the need for “learning-to-learn to occur at multiple levels of the hierarchical generative process” (Lake et al., sect. 4.2.3, para. 5).
Furthermore, in category learning tasks with DLs, the internal representation of the hidden units shows that it creates prototype-like representations at each layer of the network (Caglar & Hanson Reference Caglar and Hanson2016). These higher-level representations are the result of concept learning from exemplars, and go far beyond simple pattern recognition. Additionally, the plateau characteristic of the hyperbolic learning curves provides evidence for rapid learning, as well as one-shot learning once this kind of implicit conceptual representation has been formed over some subset of exemplars (similar to a “prior”) (Saxe 2014). Longstanding investigation in the learning theory literature proposes that the hyperbolic learning curve of DLs is also the shape that best describes human learning (Mazur & Hastie Reference Mazur and Hastie1978; Thurstone Reference Thurstone1919), thereby suggesting that the learning mechanisms of DLs and humans might be more similar than thought (Hanso et al., in preparation).
Taken together, the analysis of learning curves and internal representations of hidden units indicates that NNs do in fact build models and create representational structures. However, these models are implicitly built into the learning process and cannot be explicitly dissociated from it. Exploiting the rich information of the stimulus and its context, the learning process creates models and shapes representational structures without the need for explicit preprogramming.