Curiosity-based topological reinforcement learning

Abstract

Recent works involved in enhancing the learning convergence of reinforcement learning (RL) in mobile robot navigation have investigated methods to obtain knowledge from efficiently exploring the robot's environment. In RL, this knowledge is highly desirable to reduce the high number of interactions required for updating the value function and to eventually find an optimal or suboptimal policy for the agent. In this work, we propose a curiosity-based topological RL (CBT-RL) algorithm that makes use of the topological relationships among the observed states of the environment in which the agent acts. This algorithm builds an incremental topological map of the environment using Instantaneous Topological Map (ITM) model, which we use for facilitating value function updates as well as providing a guided exploration. We evaluate our algorithm against the original Q-Learning and Influence Zone algorithms in static and dynamic environments.