A Multi-Target Trajectory Planning of a 6-DoF Free-Floating Space Robot via Reinforcement Learning

Abstract

Space robots have played an essential role in space junk removal. Compared with traditional model-based methods, model-free reinforcement learning methods are promising in tackling space capture missions, which is challenging due to the dynamic singular problem and measuring errors of dynamics parameters. Nevertheless, current research mostly focus on the single-target environment. In this paper, we propose a multi-target trajectory planning strategy of a 6-DoF free-floating space robot optimized by the Proximal Policy Optimization (PPO) algorithm. Furthermore, we adopt some augmentation techniques to improve the PPO algorithm on precision and stability of reaching multiple targets. In particular, we introduce an Action Ensembles Based on Poisson Distribution (AEP) method, which facilitates the policy to efficiently approximate the optimal policy. Our method can be easily extended to realize the task that the end-effector tracks a specific trajectory. We evaluate our approach on four tasks: circle trajectory tracking, external disturbances at joints, different masses of the base, and even single joint failure, without any further fine-tuning. The results suggest that the planning strategy has comparably high adaptability and anti-inference capacity. Qualitative results (videos) are available at [36].