Self-calibrating active depth perception via motion parallax

Abstract

A hallmark of biological systems is their ability to self-calibrate sensory-motor loops during their development. Understanding the principles of such self-calibration will enable the design of robots with similar autonomous learning abilities. Here we consider the problem of active depth perception based on motion parallax. When an observer moves sideways while looking at an object with a single eye, the eye rotation necessary to keep the object at the center of gaze provides information about the object's distance. Based on the recently proposed active efficient coding (AEC) approach, we present a self-calibrating system which autonomously learns to represent image motion and perform compensatory eye rotations to keep the object fixated during side-to-side movements — thereby learning to actively estimate the object's distance. A neural network is used to provide a calibrated depth estimate. We evaluate the system's performance in simulation and in a hardware implementation.