Control of Free-Floating Space Robots to Capture Targets Using Soft Q-Learning

Abstract

In this work, a new control method for free-floating space robots to capture targets is proposed. This method can efficiently train a variety of target capturing policies without knowing the dynamic models of the robot system. We use soft Q-learning (SQL) algorithm to train stochastic energy based policies for space manipulator motion planning. Firstly, the soft Q-network and the state-conditioned stochastic policy network are constructed based on the maximum entropy objective. Then, the experience replay pool is built to store the training samples with which the both networks are trained under SQL procedures. Finally, soft Q-learning trains policies that maximize both accumulated reward and policy entropy such that the robot can successfully perform the task while act as randomly as possible. We have validated this method on the free-floating space robot with single-arm and duel-arm manipulator in simulation respectively and the results show that the free-floating space robots with stochastic energy based policies can manage to capture the target in different ways.