Hidden node activation differential - a new neural network relevancy criteria

Abstract

Neural networks have been used in many problems such as character recognition, time series forecasting and image coding. The generalization of the network depends on its internal structure. Network parameters should be set correctly so that data outside the class will not be overfitted. One mechanism to achieve an optimal neural network structure is to identify the essential components (hidden nodes) and to prune off the irrelevant ones. Most of the proposed criteria used for pruning are expensive to compute and impractical to use for large networks and large training samples. In this paper, a new relevancy criteria is proposed and three existing criteria are investigated. The properties of the proposed criteria are covered in detail and their similarities to existing criteria are illustrated.