Structure-based auto-calibration of RGB-D sensors

Abstract

The readily available image and depth data from commodity RGB-D sensors has had tremendous impact in the robotics and computer vision community recently. To jointly leverage both modalities, the depth and image measurements need to be registered. Typical calibration approaches make use of artificial landmarks and special calibration targets. However, this is not feasible if on-line (re-)calibration is necessary or the sensor setup is inaccessible, e.g., for already captured datasets. Instead of using specific calibration patterns, we propose to leverage a sparse environment model as geometric prior for the calibration. Structure-from-motion or SLAM can provide such a sparse 3D scene model, and hence our approach allows for self-calibration without the need for any manual interaction. We validate our hypothesis by introducing an optimization that jointly minimizes the alignment error between the sparse map and all recorded depth maps. Since the accuracy of depth measurements is known to degrade considerably with scene depth, we account for this distortion via a spatially varying correction term. The evaluation of our approach demonstrates that we are able to compute an accurate extrinsic and intrinsic calibration, which for example allows dense 3D modeling at improved precision.