A deep reinforcement learning algorithm based on modified Twin delay DDPG method for robotic applications

摘要

This paper proposes a deep reinforcement learning algorithm for autonomous robotics, in which we modify twin delay deep deterministic policy gradient (TD3) to adapt for autonomous robots with higher degree freedom in movement. To provide a robot with free movement in the 2D space without collisions against some obstacles, such as wall, a robot is equipped with three cameras. The images captured by camera are used to train Convolutional Neural Networks (CNN) to understand environment with collisions or not-collisions. We added two additional parameters, observation’ O’, which are images obtained from cameras, and degrees of turns' deg’ into the original TD3’ s parameters composed of four values: [state's', reward ‘r’, action ‘a’ and next-state's' ‘]. To determine a next action with higher reward from the observation, two additional Neural Networks are constructed, being the first one determines an action from observation and the second one determines degree of turn from the observation and the action. The simulation results under three environments constructed by CoppeliaSim show a good performance of the proposed algorithm, reaching the target with higher rewards, even though the environments are unknown by robots.