Natural Gesture Extraction Based on Hand Trajectory

Abstract

Automatic natural gesture recognition can be useful both for the development of human-robot applications and as an aid in the study of human gesture. The goal of this study is to recognize natural gestures using only an RGB video without machine learning methods. To develop and test the proposed method we recorded videos in which a speaker gestured naturally but in a controlled way. The advantage of using this method over lab-recorded data is that the data contain variations in gestures that are typically encountered when analyzing gestures of TV news or speech videos on the Internet. The hand positions are computed by a pose estimation method, and we recognize the gestures based on the hand trajectories, assuming that the gesturing hand(s) do(es) not change its direction abruptly during each phase of a gesture. Based on ground-truth annotations provided by linguistic experts, the accuracies were 92.15%, 91.76% and 75.81% for three natural gestures selected.