Tracking and recognition of a human hand in dynamic motion for Janken (rock-paper-scissors) robot

Abstract

In this paper, we focus on a human hand recognition system for human-robot interaction. The motion of the human hand is so dynamic that the hand needs to be tracked at high speed, and non-blurred images are required to achieve stable and correct recognition. Moreover, a wide recognition area is necessary for natural interaction. To achieve these aims, we constructed a new active sensing system that can actively track and recognize the human hand by using a high-speed vision system. An application that clearly requires high-speed recognition and actuation is the rock-paper-scissors game between a human and a robot, and we focused on this task to demonstrate the proposed system. In this situation, we track a human hand at high speed and detect its sign (rock, paper, or scissors) correctly in every frame. Our experiments showed that the robot hand formed its sign to beat the human opponent in natural interaction thanks to its high speed and wide recognition area.