Task decomposition using pattern distributor

Abstract

In this paper, we propose a new task decomposition method for multilayered feedforward neural networks, namely Task Decomposition with Pattern Distributor to shorten the training time and improve the generalization accuracy of a network under training. This new method uses the combination of modules (small-size feedforward networks) in parallel and series, to produce the overall solution for a complex problem. Based on a 'divide-and-conquer' technique, the original problem is decomposed into several simpler sub-problems by a pattern distributor module in the network, where each sub-problem is composed of the whole input vector and a fraction of the output vector of the original problem. These sub-problems are then solved by the corresponding groups of modules, where each group of modules is connected with the pattern distributor module and the modules in each group can work in parallel. The design details and implementation of this new method are introduced in this paper. Several benchmark classification problems are used to test this new method. The analysis and experimental results show that this new method could reduce training time and improve generalization accuracy.