Self-Learning Neuromorphic Robot Based on Reward-Driven Spiking Neural Network

Abstract

While there are adequate tools available to simulate Spiking Neural Networks (e.g. Brian2, snnTorch), as well as the tools for simulating robots and their environments, there remains a need for integrated tools that enable researchers to jointly simulate realistic brain models, robots, and sensory-rich environments. This work introduces a comprehensive neuromorphic robotic system, which combines neuromorphic computing with neuromorphic (and conventional) sensory and motor devices. We emulate the neuromorphic computing on a conventional low-power CPU, specifically a Virtual Machine on a Raspberry Pi 5, integrating Python and specialised packages for real-time Spiking Neural Networks (SNN) simulations. We achieve: (i) a cost-effective alternative to dedicated neuromorphic hardware, (ii) built-in GPIO and USB ports for seamless sensor and motor interfacing. We have built a demonstrator system: a robotic goalkeeper, using a DVS camera, a digital servo motor, and a touch sensor for a reward signal. The SNN uses a combination of unsupervised and supervised (reinforcement) learning. The system off-line and on-line learning was demonstrated, and some performance metrics reported.