Architecture analysis of MLP by geometrical interpretation

Abstract

Traditionally, the main focus regarding the architecture selection of MLP has been centered upon the growing and pruning and the evolutionary algorithms, in which a priori information regarding the geometrical shape of the target function is usually not exploited. In contrast to this, it will be demonstrated in this paper that it is the geometrical information that will simplify the task of architecture selection significantly. We wish to suggest some general guidelines for selecting the architecture of the MLP, provided that the basic geometrical shape of the target function is known in advance, or can be perceived from the training data. These guidelines will be based upon the geometrical interpretation of the weights, the biases, and the number of hidden neurons and layers. The controversial issue of whether four-layered MLP is superior to the three-layered MLP is also carefully examined with this geometrical interpretation.