R1. A new taxonomy for psychology?

One of the central contentions of After Phrenology (Anderson Reference Anderson2014) is that the taxonomy of psychology is in need of revision, and both Shine, Eisenberg, & Poldrack (Shine et al.) and McCaffery & Machery appear to agree. Where we disagree is what the outcome of such revision will be. In essence, Shine et al. argue that my position in Anderson (Reference Anderson2010) – where I hypothesized that there might be a set of fundamental psychological operations (“workings”) that could be strictly localized, but would be used in a variety of cognitive processes – is likely closer to the truth, and they think this model of reuse better accounts for the established data. Perhaps. Their arguments are clear and reasonable, and they usefully illuminate a number of outstanding issues regarding how best to respond to the ever-increasing evidence for functional complexity in the brain. And yet, I find the preponderance of evidence to point not back to my 2010 position, but forward to an understanding that can accommodate a range of functional arrangements, from local workings to true, irreducible polyfunctionality. Hence, are McCaffery & Machery led to ask whether I now actually deny the possibility of intrinsic functions, or if am I arguing instead for the less radical position that the best research strategy is to capture brain function in a multidimensional manner, which will sometimes lead to the discovery of fundamental operations (“workings”) and sometimes not?

I certainly endorse the latter statement. My position on intrinsic functions is accordingly a bit more nuanced than denial. I think that some partsFootnote ¹ of the brain, but probably not most, may come to have (relatively) fixed, intrinsic functions (as D'Souza & Karmiloff-Smith also insist – but note it is a different open question whether a computational vocabulary will be the best way to describe that function; McCaffery & Machery appear to conflate these two issues). I think that many parts of the brain will have multiple, repeatable configurations, and hence different functions in each.Footnote ² And I think that some parts of the brain will not have intrinsic functions at all, because the role they play in the coalitions of which they are a member will be irreducibly determined by the mutual constraints imposed by the many interacting parts (Anderson Reference Anderson, Metzinger and Windt2015), such that the overall function of the larger system may be unanalyzable into subfunctions. Hence, I do endorse the sort of functional gradation for the brain advanced by Shine et al.; I am just expecting the truth to lie a bit further to the left (as it were). That is, local multifunctionality and unanalyzable distributed functions will be much more common than they suspect – and note Silberstein argues that neural reality will prove to be further leftward still.

Because I am an advocate for a new and different taxonomy, I find some of Kaplan & Craver's concerns puzzling. For example, they argue that I “implicitly reif[y] the task domains of BrainMap” (para. 12), thereby trapping psychology inside the very taxonomy I am trying to escape. And yet, I explicitly reject any such reification and describe in some detail the many means for avoiding it. These include not just applying various machine learning, pattern analysis, dimensional reduction, and component analysis techniques to get at the hidden underlying structure of neuroimaging (and other neurofunctional) data, but also building fingerprints using multiple different initial taxonomies as starting points. And let me be as clear as I can be: Our current taxonomies are just that: starting points for the eventual articulation of a new vocabulary of psychological function.

Of course, Kaplan & Craver are right that solving the problem of functional registration is going to be wickedly difficult, for all of the reasons they cite. Any attempt to integrate scientific evidence across experiments risks including and thus mistakenly interpreting noise, and they worry that my results may reflect more noise than signal. I worry, too. That is why I take into account evidence gathered from many, many different methods, including not just fMRI but also TMS, single cell recording, neural attenuation experiments, and more. I believe that the most parsimonious hypothesis that accounts for all of these data is that most parts of the brain are functionally diverse. That said, I do not think that any of the fingerprints I have built accurately reflect the true functional profiles of any specific brain region. I think they do capture something important about the underlying functional dispositions of those regions (and the fact that fingerprints can be used to predict functional cooperation in the brain suggests this is not an unreasonable supposition), but I do not think we yet have the right way of characterizing underlying function.

Hence, here is where we need to separate the quality and accuracy of our specific current results from the value of the method going forward. As I argue in After Phrenology, representing brain activity in a multidimensional way and integrating experimental results across disparate domains, paradigms, and populations is the best way to reach an eventual understanding of the brain. If there really is a 1:1 mapping between function and structure to be found, this method will help us find it (and not integrating across different experiments will actually delay this discovery). And if there is not a 1:1 mapping to be found, then this method is the best we have for capturing the complex underlying truth. If it is right, as Pessoa, Pasqualotto, and I all argue, that the mapping between structure and function will remain many:many at multiple spatial scales, then however our taxonomy of function evolves, we need to be using tools that respect this fact.

Although I do believe that the analytic tools I advocate will prove essential to our science going forward, McCaffery & Machery raise an important concern regarding the use of unsupervised machine learning and dimensional reduction techniques to revise the taxonomy of psychology. As they note, the techniques are underdetermined, require arguably arbitrary decisions (e.g., in setting the number of dimensions or clusters), do not necessarily converge on a unique solution, and are far from guaranteed to produce meaningful results. Hence, I agree that “none of the possible spaces should be interpreted realistically” (para. 8). But this does not mean that none of the dimensions should be interpreted realistically. I think that these techniques for exploring large collections of data (neuroimaging data, yes, but also anything else we can get our hands on) will prove to be crucial guides to discovery, precisely because they will array things along nonintuitive dimensions and group things in surprising ways. And in this context, the variety of the techniques and the range of solutions is an asset. Yes, we should avoid the reification pitfall – none of these techniques will speak infallibly. But we should also avoid the cynicism pitfall of never trusting what sometimes fails. As Perlovsky notes, much is hidden from conscious awareness, and the development of language further problematizes the clear search for models of brain function, because the apparent structure of language itself can be (and has been) misconstrued to be a reflection of the underlying structure of the mind (a different instance of reification, perhaps?). By potentially defying our intuitions regarding what is neurally and psychologically similar across contexts, these techniques may help us catch a glimpse of what lies beneath. We should take what they show us seriously.

R2. Brains and bodies

Silberstein's opening line that After Phrenology is “the best thing written about the brain this century” (para. 1) already offers the core of my response to his concern that I am being too brain-centric: this appearance is simply the result of the fact that this is a book about the brain and how to study it in light of what the best neuroscience, ecological psychology, and evolutionary biology has to teach us. Hence, I completely agree with Silberstein that good neuroscience must also be what he calls “big picture” biology, and I suspect that part of what it will take to make substantial progress understanding the brain is a reform of graduate training in psychology and neuroscience to include more evolutionary and developmental biology, mathematical physics, and, yes, even philosophy (some of which is happening already). I do not expect or advocate for every scientist to master every relevant field, in the manner of TV-show physicians, but if we are to get the best advantage out of the greater interdisciplinary collaboration that everyone realizes is necessary, then we need to achieve sufficient familiarity with related disciplines to support effective communication.

Pezzulo usefully highlights one area where such opportunities have been insufficiently explored. He writes: “Progress in the field will benefit from a ‘new alliance’ between proponents of embodied and action-oriented views [of] cognition and computational modelers including roboticists” (para. 4). I completely agree – and in fact called for exactly this at the most recent conference on Advances in Cognitive Systems! I certainly hope that people heed Pezzulo's (and my) call, and I appreciate the many pointers he offers to underinvestigated theories and phenomena. I will certainly be urging my students to follow out some of these ideas.

Interestingly, I think the research path that Pezzulo advocates can also be an important part of addressing Silberstein's concerns. I agree with Silberstein and Pezzulo that the notion of constraint is going to provide a crucial organizing frame for the study of organism-environment systems (Anderson Reference Anderson, Metzinger and Windt2015), and cognitive robotics looks to be one important technique for exploring the essential questions of how the environmental and other constraints that Silberstein rightly highlights help determine brain function, and how brains are organized to take advantage of said constraints for cognitive and behavioral ends. As I note above, I am open to the notion that some parts of the brain lack intrinsic functions that can be localized, because they will be irreducibly determined by properties of whole systems, and I am also open to the possibility that some intrinsic functions can in fact be identified. Similarly, as Pezzulo advocates, I remain open to the discovery that some elements of some brain mechanisms will be best understood as internal representations, but I do not expect them to be central to our overall account of cognition. Silberstein, Pezzulo, and I are in complete accord that the neurosciences (and the cognitive sciences more generally) need to continue to move toward more embodied, embedded, dynamic, control-theoretical models of psychological phenomena.

Whereas authors including Pasqualotto, Pessoa, and Guida, Campitelli, & Gobet (Guida et al.) focus primarily on the neural evidence for the ubiquity of reuse, Wang & Bargh offer instead an excellent discussion of some of the cognitive evidence for and effects of reuse, including perceptual grounding – what they call “conceptual scaffolding” – and goal substitution. For example, they detail the fascinating ways in which physical sensations – of warmth, texture, and the like – both influence and are influenced by interpersonal social judgments of closeness and cohesion. My only concern with their account is that, although the associative connections they suggest are responsible for this influence are surely part of the story, in fact explicit associations may not always be necessary to generate the effects they discuss. As I have argued along with Penner-Wilger (e.g., Penner-Wilger & Anderson Reference Penner-Wilger and Anderson2013), all that is required to explain a given instance of reuse is that the reused neural element have the right functional properties to support each use. In the case of the finger-number relationship that we discuss, there need not have been direct experience relating fingers to numbers (for example, counting on one's fingers) for the relationship to obtain – it is enough that a particular part of the brain has the right functional structure to aid both sensory and numerical tasks.

Similarly, the connections between the literal physical experiences of closeness, warmth, and sweetness, and their corresponding social metaphors, might involve psychological associations, but it might also be simply that the relevant homeostatic mechanism has the right structure to help manage both physical and social relationships. If this is the case, then activating the mechanism for either purpose will often cause multiple effects and experiences, regardless of whether an association has been formed. Naturally, once this functional relationship exists, it could well lead to the formation of a psychological association, although not necessarily a conscious one. And I agree that, once these relationships are captured in language, then the causal story becomes significantly more complex and is likely to involve not just low-level neural mechanisms, but also conceptual and cultural ones.

R3. Which way forward?

D'Souza & Karmiloff-Smith suggest that an even greater focus on development could greatly enhance and strengthen the neural reuse framework. The questions they raise are clearly important, and I share their desire to see them addressed. As I tried to showcase in the book, I think that enhancing neuroconstructivism with some ideas and concepts arising within the reuse framework leads to a theory with greater explanatory power than either framework on its own. D'Souza & Karmiloff-Smith's own example of the complex variation observed in an individual's ability to recover from brain injury over developmental time offers a case in point: As they admit, neither the hypothesis of increasing neural commitment (decreasing neural plasticity) nor greater early vulnerability account for all of the data. But by combining these ideas with two from the neural reuse framework, one can perhaps do a bit better. Why do some early brain injuries result in worse outcomes than those that come a few years later? Perhaps because there is a crucial time for establishing the base-set of local cortical biases that will be woven into functional neural coalitions; if the “normal” base-set is missing crucial members or otherwise altered, so too will be the developmental trajectory that the individual will follow. So, from the perspective of the reuse framework, the observed “early vulnerability” is not intrinsic to the early-developing neurons themselves, but rather it is an aspect of the delicate early interactions between Hebbian plasticity and neural reuse. Similarly, reuse can potentially shed light on why it is sometimes possible to reverse the perceptual narrowing that is otherwise a crucial part of the language learning process. For reuse, much of the apparent decrease in plasticity over development comes not because Hebbian tuning is intrinsically difficult to reverse, but because any given local network becomes incorporated into multiple functional coalitions, and interactions with the other partnerships tend to reinforce existing configurations – a situation I describe using the evolutionary notion of “burden.” Neural commitment – functional tuning – does of course happen. But neurons also become burdened by the multiple uses they support, and this is an important contributor to the observed loss of behavioral and perceptual plasticity. In situations where that burden can be lessened or released, then the underlying neuroplasticity can be released and retuned. This kind of developmental thinking is in fact central to After Phrenology, although it is true that I did not discuss as much developmental phenomena and data as might have been desirable. I would like to thank the authors for highlighting some of these important findings, and I look forward to the time when we can answer the many crucial questions they raise here.

Parkinson & Wheatley and Stanley & De Brigard argue that the greater use of pattern analysis and graph theory (respectively) will smooth the path ahead. I agree, discuss both approaches at length in the book, and welcome the renewed emphasis these authors provide here. Parkinson & Wheatley may be right that neuroscience has finally turned decisively away from modular architectures (Pessoa and Badcock, Ploeger, & Allen [Badcock et al.] suggest the same), at least rhetorically, but a stroll through the poster session at any Society for Neuroscience meeting will quickly reveal that many bad habits remain – of assigning domain-restricted specialized functions to regions studied only under a single narrow range of conditions, of strictly separating perception, cognition, and action, both psychologically and neurally, and of making unwarranted reverse inferences, just to name a few (although McCaffery & Machery are right about the conditions under which reverse inference can be informative, in practice these conditions are rarely met or even explicitly considered). So perhaps the arguments against modular thinking have some work to do yet. In any case, I agree that getting in to the habit of analyzing data for distributed patterns will be an important part of the behavioral therapy that is (still) needed and promises to open up new horizons in our understanding of brain function – a promise that Parkinson & Wheatley beautifully illustrate in their own work on distance perception.

Similarly, Stanley & De Brigard are surely right that graph theory will remain a crucial tool for studying the networks of the brain (Sporns Reference Sporns2011), and I agree that finding “modules”Footnote ³ can sometimes lead to useful insights about brain structure and function. My only concern, one that Pessoa shares, is that the techniques for identifying network communities assume that each node is a member of exactly one community (which may also belong to a hierarchy). Stanley & De Brigard are right to emphasize that graph theory can help us detect dynamic and changing affiliations, but if nodes can also be members of more than one community in a given moment, then community detection algorithms will generally miss this. This does not lessen the importance of network analysis, but it does mean the results should be interpreted with due caution.

Perlovsky echoes D'Souza & Karmiloff-Smith in noting the challenges involved in specifying the underlying mechanisms of neural search; indeed, as I note in the book, this is one area ready for greater research attention. But Perlovsky's way of putting it implies that neural reuse requires the directed capacity to target specific subnetworks with which to establish new connections – as if the brain contained representations of what its various parts were capable of. Were neural reuse a process of network design, this might be an apt description, and I would agree with Perlovsky that the prospects for understanding reuse would indeed be dim. But this is a misunderstanding of the mechanism. Instead, as I explicitly argue in the book, what we seem to have is a parallel search process and a test/consolidation mechanism that results in new functional configurations. We see such a mechanism at work in brain-machine interfaces when, during the course of learning to control a new device, the existing coherence pattern in the relevant network is disrupted by both an overall increase in activity and an increase in the variability of the activity of each neural element. As I describe in After Phrenology, the effect of these two simple changes is to implement a systematic walk through coherence space – that is, it generates a search of possible functional configurations, with no anticipatory design goal or other telos required. As this search of possibilities is occurring, reward signals generated by successful trials reinforce the configurations responsible for the success, leading to the eventual consolidation of the effective configurations. I will be the first to admit that we do not yet fully understand how this works; but as Guida et al.'s comprehensive review of the expertise literature demonstrates, that it works is increasingly established, and we may be closer to understanding how than Perlovsky fears.

Perlovsky also asks how it is that we can simultaneously treat external symbols as objects to be manipulated, and as meaningful. It is an important question. I tried to point toward an answer with my notion of a cultural affordance, which Perlovsky flatteringly calls “beautiful” but then dismisses as “inexorably logical.” Here, Perlovsky simply makes a mistake: Cultural affordances may or may not turn out to provide part of the answer to this question, but to assimilate them to logic, as if all perceivable structure had to be objective, symbolic, and representational, is to miss the nature of the alternative being offered. Objects in the world are inherently significant to animals because of the relationships they have to abilities and needs. These relationships are directly perceivable and guide an organism's behavior. Cultural practices co-opt this basic behavior-guiding mechanism in a way that results in shared meaning in the context of dynamic social interactions. Here again, we do not yet understand this very well (but see Atmaca et al. Reference Atmaca, Sebanz, Prinz and Knoblich2008; Marsh et al. Reference Marsh, Richardson and Schmidt2009; Richardson & Dale Reference Richardson and Dale2005; Richardson et al. Reference Richardson, Marsh and Baron2007; Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006, to name just a few important contributions), and it is therefore no surprise that some scientists – including Perlovsky; Badcock et al., and others – remain skeptical or believe an affordance-centered cognitive science to be a mere variation on or extension of contemporary representation-centered approaches. This is why, in addition to chiding (with good justification, I might add!) ecological psychologists for largely ignoring the brain, experimental psychologists for largely ignoring evolution (not evolutionary psychologists, but we will come to that), and neuroscientists for largely ignoring organisms, I labored to provide a unified framework that could help illuminate for all how this work fits together and is mutually informative. Clearly, it will take more than this one effort for that promise to be fulfilled, but I think it is fair to say that After Phrenology offers the most comprehensive framework to date, and I do think it can be used to generate greater interdisciplinary understanding.

R4. Is this the revolution we were promised?

Of course, interdisciplinary accord requires that we adopt and share some unifying framework, and Badcock et al. are reluctant to adopt this one. Although they agree that massive modularity “can no longer be reasonably sustained” (para. 1) (a concession that, while perhaps trivial for these particular scientists, is in fact going to have profound and far-reaching effects across the behavioral and life sciences as its ramifications come gradually to be more fully understood), they are skeptical of the claim that perception is not reconstructive – a skepticism that stands in the way of recognizing that a paradigm shift may be in the offing. I guess I agree that if we do not give up reconstructive perception (and the perception-first, stimulus-response, sense-think-act model of psychology more generally), then the coming revolution will be less radical than it might otherwise be. Fully accommodating neural reuse will mean only that we must reimagine the functional architecture of the brain (as Pasqualatto, Parkinson & Wheatley, Pessoa, and Silberstein all appear to recognize), reconsider the relationships between psychological states and processes once thought distinct (some of the implications of which are reviewed by Wang & Bargh, Shine et al., and McCaffery & Machery) and rethink the relationship between evolution, plasticity, and development (as D'Souza & Karmiloff-Smith urge us to do in a much more comprehensive way). Giving up on reconstructive perception means that in addition we must fundamentally change our idea of what the brain is for, and therefore how it does what it does and how we do what we do.

Badcock et al. are skeptical not just that we are going to get a paradigm shift, but also that we need one, because they think we may already know more-or-less what the brain is for, how it does what it does, and why we do what we do. In support of this conservatism, they point to the success of the predictive coding framework, on the one hand, and the fertility of evolutionary psychology, on the other. These authors are right that the book engages with the predictive coding framework only very indirectly, via a few scattered references to Bayesian networks and a brief discussion of causal learning (to help excuse this lacuna, I plead the necessity to restrain what was already an overly long book). For the record, then, I think the predictive coding framework represents an impressive and highly successful empirical research project. I think it is undeniable that brain function involves a great deal of prediction – in a sense of “prediction” closely allied with the notion of correlation, as when we commonly say that the value of one variable “predicts” another (height predicts weight; education predicts income, etc.; see Anderson & Chemero Reference Anderson and Chemero2013 for further discussion). But I do not think that the work merits the cognitivist, representationalist, reconstuctive gloss that it is commonly given by its main proponents (e.g., Clark Reference Clark2013b) – and it is worth noting that Pezzulo, who associates predictive coding with the cybernetic framework (Seth Reference Seth, Metzinger and Windt2015), apparently sees things similarly. This is a debate that is only in its infancy (Anderson & Chemero Reference Anderson and Chemero2013; Barrett & Bar Reference Barrett and Bar2009; Gallagher & Bower Reference Gallagher and Bower2014; Gładziejewski Reference Gładziejewski2016; Hohwy Reference Hohwy2013), but my personal starting position is that none of the empirical results emerging from the predictive coding literature requires cognitivist, reconstructive explanations. They can be read as compatible with reconstructive perception or with an action-oriented, affordance-based framework (and indeed, I doubt the adjudication between these interpretations is going to rest on results from this literature, but rather on the relative fertility of the competing frameworks going forward – but that is yet another debate). The conclusion of my discussion of causal learning in After Phrenology seems equally apropos here:

Possibly Badcock et al. could counter that the representationalist gloss serves as a research-guiding heuristic, an aid to hypothesis generation to be understood instrumentally rather than as a substantive hypothesis in its own right. To see what is worrisome in such proposals, consider their discussion of massive modularity, where they suggest exactly this: “Regardless of the veracity of massive modularity, evolutionary computational theories continue to guide research in a systematic and highly productive way” (para. 7). In support of this contention, Badcock et al. cite the example of inherited perceptual biases toward threatening stimuli – for example, snake detection – and argue that this can be explained only by reference to an evolutionary adaptation. As a critique of After Phrenology, this misses the mark, for of course I fully support integrating psychology and evolutionary biology and encourage thinking about the mind as one of the many products of natural selection. But I also want to better understand the mechanisms of selection and inheritance, in a way that is sensitive to what neuroscience tells us about the architecture of the brain. It is here that evolutionary psychology, as they conceive it, is in a bit of a bind. For scientists like Badcock et al. are faced with a choice: either posit a snake-detection module, despite overwhelming evidence that the brain isn't built that way, or shrug your shoulders over the “how” question and move on to the next hypothesis. Badcock et al. seem content with the latter, but I am not, and I do not think anyone else should be, either. Letting each segregated subfield pursue its fancy in isolation is not a recipe for scientific understanding.

Still, Badcock et al. would be right to worry if the framework I am advocating were in fact unable to provide substantive hypotheses about cognition and behavior. Although time will provide the true test of their contention, as we discover whether my proposals resonate enough to spur the scientific imagination and continue to generate fruitful research activity, it is perhaps worth noting that Wang & Bargh call the hypothesis “remarkably generative and supportive of research activity on embodied cognition, motivation, and behavior” (para. 1), and they, along with Pasqualotto and D'Souza & Karmiloff-Smith further suggest that it can help account for phenomena as diverse as perceptual grounding, goal substitution, synesthesia, and cross-modal plasticity. I can also point to some things that are happening in my lab as we continue to test, refine, and further substantiate the reuse framework. Because we argue that affordance processing underlies much of higher-order cognition, including language (Glenberg & Kaschak Reference Glenberg and Kaschak2002; Kaschak & Glenberg Reference Kaschak and Glenberg2000) and decision making (Cisek Reference Cisek2007; Cisek & Kalaska Reference Cisek and Kalaska2010), we believe that indirectly manipulating affordances – by using real objects rather than pictures in psychological experiments, for example (Snow et al. Reference Snow, Pettypiece, McAdam, McLean, Stroman, Goodale and Culham2011), or changing the context within which an experiment takes place – will allow us to modulate higher-order cognitive outcomes such as similarity judgments and categorization behavior. Moreover, we have specific hypotheses about how affordances – in a nonmodular brain – integrate with emotions, and hence how emotion induction can change the affordance landscape in ways leading to detectible cognitive and behavioral consequences. The first set of experiments is already designed (and by the time this appears in print, the first data will have been gathered). So please stay tuned; there is plenty more to come!