Efficient Policy Learning for General Robotic Tasks with Adaptive Dual-memory Hindsight Experience Replay Based on Deep Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) features powerful ability of perception and decision-making, which is reward-driven and learns strategies through the interaction between the agent and the environment. However, the discrete reward mechanism makes it difficult for DRL to obtain positive feedback in the early stage of interaction, resulting in low learning efficiency. The hindsight experience replay (HER) mechanism can improve the deficiency of the discrete reward, but it also causes a lot of data redundancy. This paper proposes an adaptive dual-memory hindsight experience replay structure. The success rate of the algorithm can be improved while the training efficiency can be ensured by using the dual-memory bank structure to split the empirical data and adjusting the proportion of HER mechanism. The proposed method is applied to the DRL algorithm and verified on a 7-DoF robot, and experimental results show that the algorithm has good performance.