Backpropagation using generalized least squares

Abstract

The backpropagation algorithm is essentially a steepest gradient descent type of optimization routine minimizing a quadratic performance index at each step. In this paper, the backpropagation algorithm is re-cast in the framework of Generalized Least Squares. The main advantage is that it eliminates the need to predict an optimal value for the step size required in the standard backpropagation algorithm. A simulation result on the approximation of a non-linear dynamical system is presented to show its rapid rate of convergence compared to the backpropagation algorithm.