Deep Learning of Robotic Manipulator Structures by Convolutional Neural Network

Abstract

This paper benefits from recent developments in learning of visual features by deep nets and highlights the possibility of learning kinematic features to achieve structure information without vision inputs and only by physical variables measured by sensors such as inertial measurement units (IMUs). It proposes to extract structural kinematic information through long-term monitoring of mechanically connected bodies and variations in the acceleration and angular velocity. This paper shows that training a deep network of linear and nonlinear layers over a variety of serial manipulators provides the ability to realize the kinematic chain for a randomly placed set of sensors. The results present the efficacy of this method for a serial manipulator in the detection of its graph with success rate of 83% in detection of links and joints. An out-of-the-domain test is performed on a heavy duty manipulation setup, which shows acceptable performance change from simulated environment to the real autonomous system demonstrated on a video.