Learning-based object abstraction method from simple instructions for human support robot HSR

Abstract

This study proposes the development of a simple remote-controlled daily assistive robot to assist physically challenged individuals. Specifically, we present a method for target object selection using a single click on a graphical user interface. Using this information, the robot can automatically estimate the unknown target object region to plan to grasp and fetch the object. The challenging task is to correctly estimate the region of the object of interest. The proposed system is implemented using the following framework for estimating the region of the object. First, the robot automatically estimates the object region based on user input. Second, the user can intervene by interactively drawing and erasing the estimated region while the system sequentially updates the estimation method based only on the user's correction. The advantage of this system is that only limited inputs are required from the user, a feature that is useful for handicapped users. Moreover, we introduce (1) graph cuts, comprising “HyperPixels” and three-dimensional information, to enable the system to recognize the rich features around the user-specified region for robust segmentation, (2) interactive correction of the automatically estimated object region while the system calculates good graph parameters for the correct estimation, and (3) recall and use of the learned parameters for the estimation based on the database of features around the clicked point.