Adaptive reinforcement learning in box-pushing robots

摘要

In this paper, an adaptive state aggregation Q-Learning method, with the capability of multi-agent cooperation, was proposed to enhance the efficiency of reinforcement learning (RL) and applied to box-pushing tasks for humanoid robots. First, a decision tree was applied to partition the state space according to temporary differences in reinforcement learning, so that a real valued action domain could be represented by a discrete space. Furthermore, adaptive state Q-Learning, which is the modification of estimating Q-value by tabular or function approximation, was proposed to demonstrate the efficiency of reinforcement learning when a humanoid robot pushing a box was simulated. During the process of a robot pushing a box, because the box moves along with the direction the robot asserts force and the pushing point on the box, the robot needs to learn how to adjust angles, avoid obstacles, keep gravity, and push the box to the target point. From the simulation results, the proposed method shows its learning efficiency outperforms the Q-Learning without using adaptive states.