An Analytic End-to-End Collaborative Deep Learning Algorithm

Abstract

In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This letter presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning is used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset is presented to demonstrate the performance of the proposed approach. In addition, an online kinematics control task of a UR5e robot arm is formulated to illustrate the regression approximation and online updating ability of the proposed algorithm.