Visual Odometry with Deep Learning for Joint Semantic Segmentation

Abstract

Visual odometry (VO) is a motion estimation task using information acquired by visual sensors, which is widely used in autonomous vehicles, robots, and augmented reality. Existing learning-based VO methods often suffer from scale ambiguity and large relative pose estimation errors, leading to significant error accumulation. Considering that feature points extracted from dynamic targets in traditional geometry-based methods can have a significant impact, we designed a VO network that incorporates semantic segmentation. To improve the accuracy of each relative pose estimation and reduce the computational cost, we utilized an attention mechanism and a variant of depthwise separable convolution (DSC). Experiments with the KITTI Odometry dataset show that our method still exhibits excellent scale consistency in monocular approach, and outperforms learning-based VO methods in normal scenarios. Additionally, it still achieves optimality in some metrics compared to geometry-based methods.