Monocular Depth Estimation for Equirectangular Videos

摘要

Depth estimation from panoramic imagery has received minimal attention in contrast to standard perspective imagery, which constitutes the majority of the literature on the key research topic. The vast - and frequently complete - field of view provided by such panoramic photographs makes them appealing for a variety of applications, including robots, autonomous vehicles, and virtual reality. Consumer-level camera systems capable of capturing such images are likewise growing more affordable, and may be desirable complements to autonomous systems' sensor packages. They do, however, introduce significant distortions and violate some assumptions regarding perspective view images. Additionally, many state-of-the-art algorithms are not designed for its projection model, and their depth estimation performance tends to degrade when being applied to panoramic imagery. This paper presents a novel technique for adapting view synthesis-based depth estimation models to omnidirectional vision. Specifically, we: 1) integrate a “virtual” spherical camera model into the training pipeline, facilitating the model training, 2) exploit spherical convolutional layers to perform convolution operations on equirectangular images, handling the severe distortion, and 3) propose an optical flow-based masking scheme to mitigate the effect of unwanted pixels during training. Our qualitative and quantitative results demonstrate that these simple yet efficient designs result in significantly improved depth estimations when compared to previous approaches.