RIRL: A Recurrent Imitation and Reinforcement Learning Method for Long-Horizon Robotic Tasks

Abstract

The developments in reinforcement learning provide a powerful and efficient learning framework for autonomous robotic systems. However, prior works rarely embed historical observations due to the exponentially increasing complexity, which may not perform well for large-scale long-horizon tasks that might require hundreds and thousands of steps to complete. In this paper, we propose Recurrent Imitation and Reinforcement Learning (RIRL) to address the challenges and enable robots for such tasks. The proposed RIRL incorporates a long short-term memory (LSTM) network to retain long-term memories, which could be an effective and efficient method to tackle the long dependency problem raised in long-horizon robotic tasks. To assess the performance of the RIRL, we test it with an optimized path planning problem for a robot to perform a Radiofrequency identification (RFID) inventory in dynamic and previously unknown environments. We experimentally validate RIRL’s feasibility and effectiveness in a visual game-based simulation platform, where the proposed RIRL model outperforms three baseline schemes with considerable gains.