Kinematic Control of Manipulators using Multi Deep Q-Learning

Abstract

In this paper, we investigate the problem of controlling the position and orientation of a manipulator end-effector, in tasks such as pick-and-place or trajectory tracking, using deep reinforcement learning. Unlike other approaches in the literature that control the robot joints using just one learning agent, enabling it to perform the tasks, here we propose to separate each degree of freedom of the manipulator into a different agent, learning distinct policies for each joint. As a consequence, when using discrete action-based RL algorithms, such as DQN, instead of having a large set of possible actions as the output of a single neural network given by the combination of all individual actions in the joints, we have multiple parallel neural networks with a small number of actions for each joint. In other words, we reduce the problem’s action space by learning it as a multiagent system. As simulated and real-world experiments demonstrate, our multi-deep reinforcement learning strategy enables faster policy learning than conventional approaches.