Progressive object modeling with a continuum manipulator in unknown environments

Abstract

It is important to enable a robot to manipulate a target object that has no 3-D model information and is situated in an environment with other unknown objects nearby. This poses an open problem of how to combine perception and manipulation to enable the robot to build an appearance-based model of the target object on the spot to facilitate further manipulation of the object while avoiding the other unknown obstacles in the way. In this paper, we introduce an approach to enable a continuum manipulator, which is apt to maneuver through a crowded environment, to gradually build a 3-D surface model of the target object by moving an RGB-D sensor around the object while also detecting and avoiding surrounding unknown obstacles. Our approach interleaves perception and manipulation such that perception guides the manipulator movement, which in turn allows more perception of the target object for object model building and further manipulator motion. Our approach is characterized by a progressive strategy to register RGB-D images of the target object to build and extend a partial model of the object and the corresponding motion planning strategy for the continuum robot to carry out model building and avoid obstacles at the same time. To demonstrate the effectiveness of our approach, experiments on progressive model building of real objects from real RGB-D images are conducted, where a simulated continuum robot plans and executes its motion to carry the RGB-D camera around a target object for taking those images in an augmented reality setting.