Towards a Vision-Based Concept for Gesture Control of a Robot Providing Visual Feedback

Abstract

Human-Robot Interaction (HRI) plays an important and growing role, both in industrial applications and in game development. Over recent years, robots can be controlled by gestures via special devices, but these methods are not intuitive and require usually a learning phase. This paper proposes an intuitive method for controlling a robot end-effector using human gestures. Vision based techniques were used to track the position of the user's hand, which is directly translated in control signals. The use of a 3D camera sensor allows to easily control the robot tool position in all dimensions. Our approach includes a Graphical User Interface (GUI), to ease the control through interactive, visual feedback. This interface, including 3D markers, text messages and the visualization of the user's point cloud and the robot model, enables a control mechanism which does not require a teaching phase. Our approach was tested and evaluated using realistic experiments to prove that our approach works reliably and is extremely intuitive.