Multi-Task Decomposition Architecture based Deep Reinforcement Learning for Obstacle Avoidance

摘要

Obstacle avoidance is a basic skill of mobile robots. Currently, various Deep Reinforcement Learning (DRL) based approaches have been proposed to enable the robot to navigate in complex environments. However, these existing approaches merely employ collision-related reward to guide the learning of deep models, and thus fail to capture good domain knowledge for obstacle avoidance policy. Actually, practical applications also have strict requirements on speed and energy consumption, except for safety. In addition, the learning efficiency of the above DRL-based approaches is low or even unstable. To handle the above challenges, in this paper, we propose a Multi-task Decomposition Architecture (MDA) based Deep Reinforcement Learning for robot moving policy. This method decomposes robot motion control into two related sub-tasks, including speed control as well as orientation control, with obstacle avoidance inserted into each sub-task. Each sub-task is associated with one single reward and is solved using Dueling Double Q-learning (D3QN) algorithm. Q-values from two different sub-tasks are fused through aggregator to derive final Q-values which are used for selecting actions. Experiments indicate this low dimensional representation makes learning more effective, including better security and control over speed and direction. Moreover, robots can be widely used in new environments, even dynamic ones.