Human pointing as a robot directive

Abstract

People are accustomed to directing other people's attention using pointing gestures. People enact and interpret pointing commands often and effortlessly. If robots understand human intentions (e.g. as encoded in pointing-gestures), they can reach higher levels of engagement with people. This paper explores methods that robots can use to allow people to direct them to move to a specific location, using an inexpensive Kinect sensor. The joint positions of the pointing human's right arm and hand in 3D-space are extracted and used by the robot to identify the direction of user's pointing gesture. We evaluated the proposed approach on a PR2 robot whose task was to move to a location that a human pointed to on the ground. This method enables the robot to follow human pointing gestures on the fly and in real-time. It will be deployed on a PR2 in the wild in a new building environment where the robot will be expected to interact with people and interpret their human pointing behaviors.