Knowledge-Driven Deep Deterministic Policy Gradient for Robotic Multiple Peg-in-Hole Assembly Tasks

Abstract

It remains a formidable challenge for traditional control strategies to perform automatic multiple peg-in-hole assembly tasks due to the complicated and dynamic contact states. Inspired by that human could generalize the learned skills to perform the different assembly tasks well, a general learning-based algorithm based on deep deterministic policy gradient (DDPG) is proposed. To make robots learn the multiple peg-in-hole assembly skills from experience efficiently and stably, the learning process is driven by the basic knowledge like PD force control strategy. To achieve a fast learning process in the real-world assembly tasks, a hybrid exploration strategy is applied to drive a efficient exploration during policy search phase. A dual peg-in-hole assembly simulation and real-world experiments are implemented to verify the effectiveness of the proposed algorithm. The performance measured by the assembly time and the maximum contact forces demonstrates that the multiple peg-in-hole assembly skills could be improved only after 150 training episodes in dual peg-in-hole assembly task.