AN IMPROVED UNSUPERVISED TRAINING ALGORITHM FOR NEOCOGNITRON

Abstract

Based on computer simulations, we have investigated the unsupervised training algorithm of neocognitron. We observed that extended and strong features such as line segments are extracted in the intermediate stages, due to a positive feedback effect in the training algorithm. We then propose a refined training algorithm that discourages the extraction of these features. Models trained with this algorithm are able to extract a higher number of distinct complex features in the deeper layers, and are able to recognize handwritten characters more accurately. Analysis in multidimensional vector space suggests that when the influence of the extended and strong features is reduced, the integrated features vectors are projected into a "wider" space, and the overlapping between the vectors is reduced. This explains the improved differentiation between distinct classes of vectors and the higher accuracy of recognition. We also investigated the use of information theory as an objective measure of the competence of neocognitron as a pattern classifier and recognizer.