Object exploration using vision and active touch

Abstract

Achieving object exploration with passive vision and active touch has been under investigation for thirty years. We build upon recent progress in biomimetic active touch that combines perception via Bayesian evidence accumulation with controlling the tactile sensor using perceived stimulus location. Here, passive vision is combined with active touch by providing a visual prior for each perceptual decision, with the precision of this prior setting the relative contribution of each modality. The performance is examined on an edge following task using a tactile fingertip (the TacTip) mounted on a robot arm. We find that the quality of exploration is a U-shaped function of the relative contribution of vision and touch; moreover, multi-modal performance is more robust, completing the contour when touch alone fails. The overall system has several parallels with biological theories of perception, and thus plausibly represents a robot model of visuo-tactile exploration in humans.