Pick-place of dynamic objects by robot manipulator based on deep learning and easy user interface teaching systems

Abstract

Purpose Development of autonomous robot manipulator for human-robot assembly tasks is a key component to reach high effectiveness. In such tasks, the robot real-time object recognition is crucial. In addition, the need for simple and safe teaching techniques need to be considered, because: small size robot manipulators’ presence in everyday life environments is increasing requiring non-expert operators to teach the robot; and in small size applications, the operator has to teach several different motions in a short time. Design/methodology/approach For object recognition, the authors propose a deep belief neural network (DBNN)-based approach. The captured camera image is used as the input of the DBNN. The DBNN extracts the object features in the intermediate layers. In addition, the authors developed three teaching systems which utilize iPhone; haptic; and Kinect devices. Findings The object recognition by DBNN is robust for real-time applications. The robot picks up the object required by the user and places it in the target location. Three developed teaching systems are easy to use by non-experienced subjects, and they show different performance in terms of time to complete the task and accuracy. Practical implications The proposed method can ease the use of robot manipulators helping non-experienced users completing different assembly tasks. Originality/value This work applies DBNN for object recognition and three intuitive systems for teaching robot manipulators.