Retrieving unknown objects using robot in-the-loop based interactive segmentation

Abstract

For a robot to operate efficiently in a human centered environment, it should be able to interact and learn unknown objects autonomously. Such capabilities will enable a robot to enrich its internal knowledge of the environment without human assistance. However, a crucial limitation of robots is their inability to comprehend representations of novel objects without priors. Human efforts are required to provide pertinent prerequisites for learning novel objects. In this paper, we exploit the visual and manipulative capabilities of a mobile robot to interact with an unknown cluttered scene on a support plane and retrieve objects requested by humans. The object to retrieve may support other unknown objects, which the robot has to identify and carefully remove. The boundaries of objects in a cluttered scene are estimated using 3D geometrical relationships between the surface normals. Using this estimate, the robot interacts with the objects through graspless action (push), and visual changes to the scene are used to revamp the initial hypothesis for final object region estimation. The estimated region is then used for grasping and removing the supported objects in order to retrieve the target object. The presented approach is model free and requires no prior object knowledge.