Please use this identifier to cite or link to this item: https://doi.org/10.1016/j.neucom.2008.10.016
Title: Systemical convergence rate analysis of convex incremental feedforward neural networks
Authors: Chen, L. 
Huang, G.-B.
Pung, H.K. 
Keywords: Convergence rate
Feedforward neural networks
Random hidden neurons
Universal approximation
Issue Date: 2009
Source: Chen, L., Huang, G.-B., Pung, H.K. (2009). Systemical convergence rate analysis of convex incremental feedforward neural networks. Neurocomputing 72 (10-12) : 2627-2635. ScholarBank@NUS Repository. https://doi.org/10.1016/j.neucom.2008.10.016
Abstract: In this paper, we systemically investigate several convex incremental feedforward neural networks. Firstly, we prove the universal approximation and the convergence rate of a generalized convex incremental (GCI) structure, which provides us a wider parameter selection. Second, according to the convergence rate proof of GCI, we further prove the convergence rate of a best convex incremental (BCI) structure, moreover its proof also illustrates that BCI can achieve a better generalization performance than GCI. But we should note that the hidden neurons of BCI and GCI both are constructed on the maximum principle (not random). Next, we introduce the random neuron conception based on CI-ELM (convex incremental extreme learning machines), and further propose an alternative algorithm (improved CI-ELM, ICI-ELM) between CI-ELM and BCI, which removes the "useless" neurons in CI-ELM and improves the efficiency of neural networks. ICI-ELM randomly generates a group of parameters, among which we determine the best parameters leading to the smallest residual error. Therefore ICI-ELM can achieve a faster convergence rate than CI-ELM, meanwhile it still retains the same convergence rate as BCI. On the other hand, ICI-ELM also provides an alternative scheme to replace conventional gradient methods, which are only suitable for differential functions and often achieves local minima. The experimental results based on several benchmark regression problems support our claims. Crown Copyright © 2008.
Source Title: Neurocomputing
URI: http://scholarbank.nus.edu.sg/handle/10635/43143
ISSN: 09252312
DOI: 10.1016/j.neucom.2008.10.016
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

8
checked on Dec 6, 2017

WEB OF SCIENCETM
Citations

9
checked on Nov 18, 2017

Page view(s)

67
checked on Dec 10, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.