Etalonnage de caméra plénoptique et estimation de profondeur à partir des données brutes

摘要

Unlike a standard camera that records two dimensions of a light field, a plenoptic camera is designed to locally capture four of its dimensions. The richness of data from this sensor can be use for many applications : it is possible to post-process new points of view, to refocus on different scene’s planes, or to compute depth maps of a scene from a single acquisition and thus obtain 3D reconstructions of the environment. This passive sensor allows the capture of depth using a compact optical system, which makes it attractive for robotics applications. However, depth estimations from such a sensor requires its precise calibration. This camera is composed of a substantial number of elements, including a micro-lens array placed in front of the sensor, and its raw data is complex. Most of the state-of-the-art calibration approaches then consist in formulating simplified projection models and exploiting interpreted data such as synthesized images and associated depth maps. Hence, in our first contribution, carried out in collaboration with the TUM laboratory, we proposed a calibration method from a 3D test pattern using interpreted data. Then we proposed a new calibration approach based on raw data. We formalized a physical-based model of the camera and proposed a minimization expressed directly in the sensor data space to estimate its parameters. Finally, we proposed a new metric scaled depth estimation method using the camera projection model. This direct approach uses an error minimization between each micro-image content and the texture reprojection of the micro-images that surround it. Our algorithms performance was evaluated both on a simulator developed during this thesis and on real scenes. We have shown that the calibration is robust to bad model initialization and the depth estimation accuracy competes with the state-of-the-art.