A Learning-Based Two-Stage Method for Submillimeter Insertion Tasks With Only Visual Inputs

Abstract

Insertion tasks are commonly encountered in robot assembly automation. However, traditional vision positioning methods are difficult to identify the target positions exactly due to uncertainties in the sensor, robot, and environment. Reinforcement learning shows great promise in solving insertion tasks under such uncertainties. Nevertheless, reinforcement learning struggles with sample efficiency in the real world. In this article, a two-stage method is proposed to solve the submillimeter insertion tasks with only visual inputs. First, an eye-to-hand vision sensor is used to get an approximate pose around the target by a traditional vision positioning method. Second, a local policy learned with an eye-in-hand vision sensor guides the robot to accomplish the task. Furthermore, a new state representation is introduced, instead of the raw image captured by the eye-in-hand camera, to improve the training efficiency. Extensive simulation experiments are presented to determine the optimal training scheme. The proposed method is validated on a real robot, succeeding in performing the submillimeter insertion tasks within 3 h of interaction. Finally, the performances under the tabletop condition and the challenging 6-D pose uncertainty condition are evaluated.