Attentional Separation-and-Aggregation Network for Self-supervised\n Depth-Pose Learning in Dynamic Scenes

Abstract

Learning depth and ego-motion from unlabeled videos via self-supervision from\nepipolar projection can improve the robustness and accuracy of the 3D\nperception and localization of vision-based robots. However, the rigid\nprojection computed by ego-motion cannot represent all scene points, such as\npoints on moving objects, leading to false guidance in these regions. To\naddress this problem, we propose an Attentional Separation-and-Aggregation\nNetwork (ASANet), which can learn to distinguish and extract the scene's static\nand dynamic characteristics via the attention mechanism. We further propose a\nnovel MotionNet with an ASANet as the encoder, followed by two separate\ndecoders, to estimate the camera's ego-motion and the scene's dynamic motion\nfield. Then, we introduce an auto-selecting approach to detect the moving\nobjects for dynamic-aware learning automatically. Empirical experiments\ndemonstrate that our method can achieve the state-of-the-art performance on the\nKITTI benchmark.\n