An algorithm for training multilayer perceptrons for data classification and function interpolation

Abstract

This paper addresses the issue of employing a parametric class of nonlinear models to describe nonlinear systems. This model class consists of a subclass of artificial neural networks, multilayer perceptrons. Specifically, we discuss the application of a "globally" convergent optimization scheme to the training of the multilayer perceptron. The algorithm discussed is termed the conjugate gradients-trust regions algorithm (CGTR) and combines the merits of two well known "global" algorithms-the conjugate gradients and the trust region algorithms. In this paper we investigate the potential of the multilayer perceptron, trained using the CGTR algorithm, towards function approximation in two diverse scenarios: i) signal classification in a multiuser communication system, and ii) approximating the inverse kinematics of a robotic manipulator. Until recently, the most widely used training algorithm has been the backpropagation algorithm, which is based on the linearly convergent steepest descent algorithm. It is seen that the multilayer perceptron trained with the CGTR algorithm is able to approximate the desired functions to a greater accuracy than when trained using backpropagation. Specifically, in the case of the multiuser communication problem, we obtain lower probabilities of error in demodulating a given user's signal; and in the robotics problem, we observe lower root mean square errors in approximating the inverse kinematics function.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>