First contact: an active vision approach to segmentation

Abstract

How a robot should grasp an object depends on its size and shape. Such parameters can be estimated visually, but this is fallible, particularly for unrecognized, unfamiliar objects. Failure will result in a clumsy grasp or glancing blow against the object. If the robot does not learn something from the encounter, then it will be apt to repeat the same mistake again and again. This paper shows how to recover information about an object's extent by poking it, either accidentally or deliberately. Poking an object makes it move, and motion is a powerful cue for visual segmentation. The periods immediately before and after the moment of impact turn out to be particularly informative, and give visual evidence for the boundary of the object that is well suited to segmentation using graph cuts. The segmentation algorithm is shown to produce results consistent enough to support autonomous collection of datasets for object recognition, which enables often-encountered objects to be segmented without the need for further poking.