Recognition of Multi-Pose Head Gestures in Human Conversations

Abstract

We address the problem of recognizing multi-pose head nodding and shaking gestures in human conversations. Existing methods mainly recognize head gestures in restricted environments like human robot interaction, where face poses are near frontal and head motions are not natural. However, in human conversations, faces of subjects might be in arbitrary poses while head gestures are often subtle. Since the face pose change and head gesture movement are of different scale, we propose to track the face of varied poses with a mixed-state particle filter and detect the subtle head movement by a Kanade-Lucas-Tomasi tracker. The motion patterns in both horizontal and vertical directions are detected and then head gestures are analyzed by a Finite State Machine. Experiments on natural human conversations demonstrated the effectiveness of our method.