Stereo Matching With Multiscale Hybrid Cost Volume

Abstract

Stereo matching estimates the disparity between a pair of rectified left and right images, which plays an important role in robot navigation, autonomous driving, and other related tasks. Despite the remarkable progress of deep learning in stereo matching, there are still many challenges. One of the challenges is that current stereo models mostly generate a single scale cost volume based on costly 3D convolutions method or quick 2D convolutions method, but neither of these two methods can achieve a fair trade-off between quality and time. In this paper, we propose to construct multi-scale hybrid cost volume which aims at achieving fast speed while maintaining comparable accuracy. Concretely, we generate the correlation cost volume and the concatenation cost volume respectively, and then integrate together to form a hybrid cost volume which can significantly improve the accuracy and reduce the computational complexity. At the multi-scale level, we generate three hybrid cost volumes at different scales and then aggregate them by 2D convolutions which are faster than 3D convolutions. In addition, we adopt 2D CNN stacked hourglass with fused cost volume for cost aggregation. Specifically, the proposed method provides competitive performance with state-of-the-art methods, while being faster than most top-performing methods (e.g., 3.7× than PSMNet, 8.1× than GCNet, 16.3× than GANet). Extensive experiments of our model on the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate the competitiveness of our results.