Martian Flight: Enabling Motion Estimation of NASA's Next-Generation Mars Flying Drone by Implementing a Neuromorphic Event-Camera and Explainable Fuzzy Spiking Neural Network Model

Abstract

The event camera is researched, developed, and designed to imitate the human eye; it is a groundbreaking vision sensor with the following advantages over a standard camera: a net rate that is much faster, a latency that is far less, a high dynamic range, and it uses far less power. These fundamental properties assist in enabling the design of third-generation algorithms in Spiking Neural Networks intended to mimic the human brain and vision processing. Moreover, these fundamental properties enable swift robotics despite the challenges of motion blur and high latency that standard cameras face. Also, these properties should enable motion estimation from the surface features on Mars, which is difficult to achieve with standard cameras. For this research, the team is using a NASA Space use-case application. For this NASA Space application, the team has chosen a challenging motion-estimation task involving a Mars-based above-ground Helicopter beyond “Ingenuity” and the planet and surface of Mars. Event-based cameras have been gaining interest within the computer vision community. They are particularly suitable for applications with challenging temporal constraints and safety requirements. Thus, Event-based sensors are an excellent match for Spiking Neural Networks (SNNs), as coupling an asynchronous sensor with neuromorphic hardware can result in real-time systems with minimal power requirements. Moreover, methods to verify and validate event-based sensing platforms for space applications are lacking. In this paper, we investigate the addition of fuzzy logic models to arrive at an explainable SNN algorithm for the NASA space use-case. Ultimately, the team aims to develop a unified model yielding reasonably accurate optical flow estimates.