NeuTac: Zero-Shot Sim2Real Measurement for Neuromorphic Vision-Based Tactile Sensors

Abstract

Neuromorphic vision-based tactile sensors (NVBTSs) have recently attracted significant attention in robotic perception. However, developing neuromorphic vision-based tactile perception algorithms remains challenging due to the unconventional, asynchronous output of the neuromorphic vision sensor (NVS). To address this gap, this article introduces NeuTac, a novel zero-shot simulation to reality (Sim2Real) transfer method for NVBTSs. NeuTac proposes a lightweight neural network with a novel loss function to denoise neuromorphic events to robustly extract neuromorphic vision-based measurements (NVBMs) and mitigate measurement uncertainty for tactile perception. During real-time deployment, the extracted NVBMs subsequently resemble the simulated measurements in finite-element analysis (FEA), bridging the Sim2Real gap for NVBTSs. We demonstrate NeuTac in a framework for neuromorphic vision-based contact pose measurement and contact force estimation. Various experimental scenarios were conducted to validate the method. The results show that NeuTac facilitates Sim2Real transfer for neuromorphic vision-based tactile perception, and its performance is on par with Sim2Real methods for conventional vision-based tactile sensors.