Approximation by ridge functions and neural networks with a bounded number of neurons

Abstract

In this paper, we consider the problem of approximation of continuous multivariate functions by neural networks with a bounded number of neurons in hidden layers. We prove the existence of single-hidden-layer networks with bounded number of neurons, which have approximation capabilities not worse than those of networks with arbitrarily many neurons. Our analysis is based on the properties of ridge functions.

Keywords:

• neural network
• MLP model
• activation function
• weight
• ridge function
• extremal element
• path
• orbit

AMS Subject Classifications:

• 41A30
• 41A63
• 68T05
• 92B20

Acknowledgements

The author thanks the referees for many useful suggestions and comments on improving the original manuscript.

Notes

This research has been supported by SOCAR Science Foundation of Azerbaijan [grant number SOCAR EF 2013].