Learning a Generic Olfactory Search Strategy From Silk Moths by Deep Inverse Reinforcement Learning

摘要

Despite their simple nervous systems, insects efficiently search for and find sources of odorants. Hence, it is necessary to model and implement such behavior in artificial agents (robots), to enable them to detect dangerous substances such as drugs, gas leaks, and explosives. Previous studies have approached behavioral modeling with either statistical or machine-learning methods. In this study, we determined the behavior trajectories of male silk moths using a virtual reality (VR) system. We then modeled these trajectories as a Markov decision process (MDP) and employed inverse reinforcement learning (IRL) to learn their reward function. Furthermore, we estimated the optimal policy from the learned reward function. We then conducted olfactory search simulations and determined that the IRL-based policy could locate odor sources with a high success rate. This was also investigated under environmental conditions different from those faced by real moths on the VR system. The obtained results indicate that IRL can generically represent olfactory search strategies that are adaptable to various environments.