20.8 Space-Mate: A 303.5mW Real-Time Sparse Mixture-of-Experts-Based NeRF-SLAM Processor for Mobile Spatial Computing

摘要

Recently, spatial computing has become popular in mobile devices, such as autonomous robots and augmented reality (AR) glasses [1], and it enables cyber-physical interaction through accurate user position and 3D geometric information of the surrounding environment obtained with the simultaneous localization and mapping (SLAM) algorithm. Previous SLAM processors [2–5] accelerated mapping and tracking, but they supported few (<5K) point features, and required additional post processing (volumetric fusion [6]) for dense 3D map acquisition. Real-time SLAM processing is impossible on memory-constrained mobile devices due to the large (>60MB) dense 3D map representation which stores color/distance values in high resolution (<4 cm) voxel. A neural radiance fields (NeRF)-based dense SLAM system [7] realizes a small memory footprint (<1MB) 3D map representation with a combination of low-resolution (20cm) 3D embedding and a color/distance decoding multi-layer perceptron (MLP). However, intensive MLP computations (~13TFLOPs for 30fps) require a high-end GPU which is unsuitable for mobile devices. In this paper, a Sparse-Mixture-of-Experts (SMoE) [8] - based NeRF-SLAM system is proposed and it dynamically utilizes a small fragment of MLP parameters (called experts) on every layer for compact computation (6.9× lower) and memory size (67.2× lower) with lower error (2.3× lower in the Replica dataset [9]) than a traditional dense SLAM.