Neuromorphic self-driving robot with retinomorphic vision and spike-based processing/closed-loop control

Abstract

We present our work on a neuromorphic self-driving robot that employs retinomoprhic visual sensing and spike based processing. The robot senses the world through a spike-based visual system - the Asynchronous Time-based Image Sensor (ATIS) - and processes the sensory data stream using IBM's TrueNorth Neurosynaptic System. A convolutional neural network (CNN) running on the TrueNorth determines the steering direction based on what the ATIS “sees.” The network was trained on data from three different environments (indoor hallways, large campus sidewalks, and narrow neighborhood sidewalks) and achieved steering decision accuracies from 68% to 82% on development data from each dataset.