The Whole World in Your Hand: Active and Interactive Segmentation

Abstract

Object segmentation is a fundamental problem\nin computer vision and a powerful resource for\ndevelopment. This paper presents three embodied approaches to the visual segmentation of objects. Each approach to segmentation is aided\nby the presence of a hand or arm in the proximity of the object to be segmented. The first\napproach is suitable for a robotic system, where\nthe robot can use its arm to evoke object motion. The second method operates on a wearable system, viewing the world from a human's\nperspective, with instrumentation to help detect\nand segment objects that are held in the wearer's\nhand. The third method operates when observing\na human teacher, locating periodic motion (finger/arm/object waving or tapping) and using it\nas a seed for segmentation. We show that object segmentation can serve as a key resource for\ndevelopment by demonstrating methods that exploit high-quality object segmentations to develop\nboth low-level vision capabilities (specialized feature detectors) and high-level vision capabilities\n(object recognition and localization).