Population-coded Spiking Neural Network with Reinforcement Learning for Mapless Navigation

Abstract

Most of the navigation methods currently applied to mobile robots cost too much in building and maintaining maps. Therefore, it is crucial to implement mapless navigation for mobile robots. Although the recent deep reinforcement learning (DRL) methods have been able to make full use of the on-board resources to explore unknown space, their high energy cost limits their application. The low energy consumption of the spiking neural network (SNN) can help the DRL to overcome this difficulty. In this paper, we combine the SNN with the deep deterministic policy gradient (DDPG) method. To address the problem of long intervals between two adjacent pulses in the integrate-and-fire (LIF) neuron model, we change the way the membrane voltage resets and the dynamics of the neuron during the refractory period. On this basis, the environmental information was encoded using neuron population coding method and the two networks were trained jointly using an extended spatial-temporal backpropagation (STBP) method. The simulation results show that the proposed method achieves a higher success rate in navigation compared to traditional deep learning algorithms.