Edge-Assisted Epipolar Transformer for Industrial Scene Reconstruction

摘要

Given a set of calibrated images, Multiple View Stereo (MVS) applies end-to-end depth inference network to recover scene structure. However, previous methods designed pixel-visibility modules to aggregate cross-view cost, ignoring the consistency assumption of 2D contextual features in the 3D depth direction. The current multi-stage depth inference model also relies on intensive depth samples, which requires high memory consumption. To alleviate these problems, this work exploits edge-assisted epipolar Transformer for multi-view depth inference. The improvements of this work are summarized as follows: 1) The epipolar Transformer block is developed for reliable cross-view cost aggregation, and the edge detection branch is designed to constrain the consistency of epipolar geometry and edge features. 2) The dynamic depth range sampling mechanism based on probability volume is adopted to improve the accuracy of uncertain areas. Comprehensive comparisons with the state-of-the-art works indicate that our work can reconstruct dense scene representations with limited memory bottleblock <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Learning-based MVS can obtain dense point clouds with accurate depth map estimation, which are widely applied in the fields of unmanned driving, battlefield environment perception and robot navigation. MVS-based scene reconstruction technology is the premise of the subsequent planning, decision-making and control of the human-machine system. To obtain dense scene representation with limited memory and runtime, this work proposes a multi-view stereo network with edge-assisted epipolar Transformer. Experiments on public benchmarks verify the feasibility and effectiveness of our model, which has good potential in battlefield environment reconstruction and human-computer interaction fields, and can provide intuitive and dense scene representation for decision-making assistance.