Multi-State-Space Reasoning Reinforcement Learning for Long-Horizon RFID-Based Robotic Searching and Planning Tasks

Abstract

In recent years, reinforcement learning (RL) has shown high potential for robotic applications. However, RL heavily relies on the reward function, and the agent merely follows the policy to maximize rewards but lacks reasoning ability. As a result, RL may not be suitable for long-horizon robotic tasks. In this paper, we propose a novel learning framework, called multiple state spaces reasoning reinforcement learning (SRRL), to endow the agent with the primary reasoning capability. First, we abstract the implicit and latent links between multiple state spaces. Then, we embed historical observations through a long short-term memory (LSTM) network to preserve long-term memories and dependencies. The proposed SRRL's ability of abstraction and long-term memory enables agents to execute long-horizon robotic searching and planning tasks more quickly and reasonably by exploiting the correlation between radio frequency identification (RFID) sensing properties and the environment occupation map. We experimentally validate the efficacy of SRRL in a visual game-based simulation environment. Our methodology outperforms three state-of-the-art baseline schemes by significant margins.