Omnidirectional multi-view systems: Calibration, features and 3D information

摘要

In this thesis we focus on omnidirectional multi-view systems, in particular central catadioptric systems. We go from the very early step of calibration to the high level task of 3D information recovery. We also consider a very important intermediate step, which is the development of properly adapted features for central catadioptric systems. In the following paragraphs we describe in moredetail the goals reached on this thesis. We selected the sphere camera model to deal with the central catadioptric systems since it gives information about the elements of the system than other models. In particular it gives important information about the mirror shape used in the catadioptric system. We introduce this model in detail in Chapter 2 along with an analysis of the relation between this model and actual central catadioptric systems. We also introduce the so-called lifted coordinates. In Sturm and Barreto (2008) the complete general theoretic projection matrix that considers all central catadioptric systems has been presented. We use this projection matrix to construct a new approach to calibrate any single- viewpoint catadioptric camera. Chapter 3 presents this approach, which requires twenty 3D-2D correspondences to compute the projection matrix. The 3D points must lie on at least three different planes. The projection of 3D points on a catadioptric image is performed linearly using a 6 × 10 projection matrix, which uses lifted coordinates for image and 3D points. From this matrix, an initial estimation of the intrinsic and extrinsic parameters of the catadioptric system is obtained. We use these parameters to initialize a non-linear optimization process. This approach is also able to calibrate slightly non-central cameras, in particular, fish-eye cameras. Since reprojection error is not sufficient to determine the accuracy of the approach, we decide to perform a 3D reconstruction from two omnidirectional images. During the development of our proposed calibration algorithm we realize that there was a lack of deep analysis and comparison of the existing calibration methods for central catadioptric systems. Moreover, for the robotics community where most tasks require to recover information from the environment, calibration of cameras is a basic step. At the same time this step should be easy to perform and reliable. In this order we perform a classification of the existing approaches. On the other hand we select those approaches which are already available as OpenSource and which do not require a complex pattern or scene to perform a comparison using synthetic and real images, so the user could select the more convenient for its particular case. Chapter 4 presents these methods and an analysis of their advantages and drawbacks. In Chapter 5 we present a deep analysis of the two-view relations of uncalibrated central catadioptric systems. We particularly pay attention to the mixture of central catadioptric systems and perspective cameras, which we call hybrid. The two-view geometric relations we consider are the hybrid fundamental matrix and the hybrid planar homography. These matrices contain useful geometric information. We study three different types of matrices, varying in complexity depending on their capacity to deal with a single or multiple types of central catadioptric systems. The first and simplest one is designed to deal with paracatadioptric systems, the second one and more complex, considers the combination of a perspective camera and any central catadioptric system. The last one is the complete and generic model which is able to deal with any combination of central catadioptric systems. We show that the generic and most complex model sometimes is not the best option when we deal with real images. Simpler models are not as accurate as the complete model in the ideal case, but they provide a better and more accurate behavior in presence of noise, being simpler and requiring less correspondences to be computed. Finally