Sensor Deployment for Visual 3D Perception: A Perspective of Information Gains

Abstract

Visual 3D perception is considered as an essential technique in autonomous systems and has been widely used in robot perception, autonomous driving, VR/AR, etc. This article focuses on the influence of visual sensor deployment on perception quality, and studies both temporal and spatial cooperations in visual 3D perception from a perspective of information gains. In particular, we first propose a metric to evaluate the perception quality under a specific sensor deployment based on statistical analysis. Then the essential properties of the Fisher information matrix in visual 3D perception are derived to reveal the information gains of observations from visual sensors of an arbitrary pose, with the geometric interpretations given in the eigenspace. Besides, we also investigate both temporal and spatial cooperations for optimal sensor deployment with respect to visual 3D perception quality, and propose a next-best-view selection scheme for the temporal cooperation and a camera configuration scheme for the spatial cooperation. Numerical results validate our theoretical analysis and the performance of our schemes.