Finger direction recognition toward human-and-robot cooperative tasks

Abstract

In human-and-robot cooperative tasks, a robot must estimate human intentions. For example, if we want a gadget from another person, we naturally and casually point at it with our index finger. In this paper, we develop an object detection system through finger pointing using a stereo vision sensor to apply the above simple gesture to human-and-robot cooperative tasks. In this system, the robot calculates the pointing direction to detect the object indicated by the fingertip based on image processing of the pointing finger. We improved our algorithm to enhance its accuracy using two methods. One uses the convex hull to solve the problem that determination of the fingertip candidates depends on the contour condition; the other calculates the movement amount of the pointed object's center. Our evaluation test shows that the former method enables our system to detect the fingertips in all the frames, and the latter method can identify whether the human is intentionally pointing at the object.