Grasping an Indicated Object in a Complex Environment

Abstract

This paper presents a task instruction system for picking up an indicated object in a complex environment. The basic setup of the system consists of a manipulator, a stereo camera, and a monitor. The task instruction scheme comprises the following four main steps: 1) The user indicates the target object on the camera image by a mouse click, and selects the object category and grasping form from a pop-up list. In this way, the system obtains information about the 3D position coordinates of the clicked point on the target object, the primitive shape of the object, and the task model for object picking with the selected grasping form. 2) The system acquires the information about the target object and constructs an environment model around the target by stereo vision using the information obtained from the task instruction. 3) The system finds a grasp point based on a grasp evaluation using the acquired information. 4) The robot performs picking up the indicated object. The features of the scheme are that the user can easily instruct the robot how to perform the object picking task by simple clicking operation, and the robot can pick up the indicated object placed in a complex environment without any previous knowledge of the target object.