Incremental neural network training with an increasing input dimension

Abstract

Conventional Neural Network (NN) training is done by introducing training patterns in the full input dimension under batch mode. In this paper, an incremental training method with an increasing input dimension (ITID) is presented. ITID works by dividing the whole input dimension into several sub dimensions each of which corresponds to an input attribute. Instead of learning input attributes altogether as an input vector in a training instance, NN learns input attributes one after another through their corresponding sub-networks and the NN structure is grown incrementally with an increasing input dimension. During training, information obtained from a new sub-network is merged together with the information obtained from the old ones to refine the current NN structure. With less internal interference among input attributes, ITID achieves higher generalization accuracy than the conventional method. The experiment results of several benchmark problems show that ITID is efficient and effective for both classification and regression problems.