Learning strategy fusion for acquiring crawling behavior in multiple environments

Abstract

Though a reinforcement learning method is considered as a promising method for learning a robot's behavior from reward signals and adapting it for unknown environment, a standard reinforcement learning method is for a single environment. In this paper, to make a robot working in wider environments, we develop a reinforcement learning method for (1) estimating the current environment, (2) choosing a suitable policy for a known environment, and (3) making learning efficient when learning in a new environment by using transfer learning. To achieve them, we extend the learning strategy (LS) fusion method [1]. LS fusion is a method to learn multiple policies for a single task by applying multiple learning strategies (LSs) step by step. The key idea of environment estimation is using reward statistics of learned policies. For efficient learning, we design a learning strategy to transfer a policy learned in a different environment to one for the current environment. To verify the proposed method, we conducted some experiments where a small size humanoid robot learned a crawling task in several kinds of environments.