Please use this identifier to cite or link to this item:
Title: Systemical convergence rate analysis of convex incremental feedforward neural networks
Authors: Chen, L. 
Huang, G.-B.
Pung, H.K. 
Keywords: Convergence rate
Feedforward neural networks
Random hidden neurons
Universal approximation
Issue Date: 2009
Citation: Chen, L., Huang, G.-B., Pung, H.K. (2009). Systemical convergence rate analysis of convex incremental feedforward neural networks. Neurocomputing 72 (10-12) : 2627-2635. ScholarBank@NUS Repository.
Abstract: In this paper, we systemically investigate several convex incremental feedforward neural networks. Firstly, we prove the universal approximation and the convergence rate of a generalized convex incremental (GCI) structure, which provides us a wider parameter selection. Second, according to the convergence rate proof of GCI, we further prove the convergence rate of a best convex incremental (BCI) structure, moreover its proof also illustrates that BCI can achieve a better generalization performance than GCI. But we should note that the hidden neurons of BCI and GCI both are constructed on the maximum principle (not random). Next, we introduce the random neuron conception based on CI-ELM (convex incremental extreme learning machines), and further propose an alternative algorithm (improved CI-ELM, ICI-ELM) between CI-ELM and BCI, which removes the "useless" neurons in CI-ELM and improves the efficiency of neural networks. ICI-ELM randomly generates a group of parameters, among which we determine the best parameters leading to the smallest residual error. Therefore ICI-ELM can achieve a faster convergence rate than CI-ELM, meanwhile it still retains the same convergence rate as BCI. On the other hand, ICI-ELM also provides an alternative scheme to replace conventional gradient methods, which are only suitable for differential functions and often achieves local minima. The experimental results based on several benchmark regression problems support our claims. Crown Copyright © 2008.
Source Title: Neurocomputing
ISSN: 09252312
DOI: 10.1016/j.neucom.2008.10.016
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.