from Part IV - Graph Representations and Efficient Computation Models
Published online by Cambridge University Press: 05 June 2013
Introduction
There exists a large gap between the empirical evidence of the computational capabilities of neural networks and our ability to systematically analyze and design those networks. Although it is well known that classical Fourier analysis is a very effective mathematical tool for the design and analysis of linear systems, such a tool was not available for artificial neural networks, which are inherently nonlinear. In the late 1980s, the spectral analysis tool was introduced in the domain of discrete neural networks. The application of the spectral technique led to a number of new insights and results, including lower and upper bounds on the complexity of computing with neural networks as well as methods for constructing optimal (in terms of performance) feedforward networks for computing various arithmetic functions.
The focus of the presentation in this chapter is on an elementary description of the basic techniques of Fourier analysis and its applications in threshold circuit complexity. Our hope is that this chapter will serve as background material for those who are interested in learning more about the progress and results in this area. We also provide extensive bibliographic notes that can serve as pointers to a number of research results related to spectral techniques and threshold circuit complexity.
To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Find out more about the Kindle Personal Document Service.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.
To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.