A hybrid training procedure for artificial neural networks leading to parametric stability and cost minimization

Abstract

This paper presents a novel training algorithm for artificial neural networks. The algorithm combines the gradient descent technique with variable structure systems approach. The combination is performed by expressing the conventional weight update rule in continuous time and application of sliding mode control method to the gradient based training procedure. The proposed combination therefore exhibits a degree of robustness with respect to the unmodeled multivariable internal dynamics of gradient descent. With conventional training procedures, the excitation of this dynamics during a training cycle can lead to instability, which may be difficult to alleviate due to the multidimensionality of the solution space and the ambiguities on the free design parameters, such as learning rate or momentum coefficient. This paper demonstrates that a computationally intelligent system can be trained such that the adjustable parameter values are forced to settle down (parameter stabilization) while minimizing an appropriate cost function (cost optimization). The proposed approach is applied to the control of a robotic arm using feedforward neural networks.