Using a human action database to recognize actions in monocular image sequences: recovering human whole body configurations

Abstract

Understanding human behaviors and generating human-like motions are key technologies for human-robot interaction, motion synthesis in computer animation, sports training, and rehabilitation. Motion capture systems have been developed to accomplish this, and marker-based motion capture systems, in particular, have been used in measuring human actions and performing action recognition. However, marker-based motion capture systems have several drawbacks; in particular, the capture system is expensive, intrusive, and complex to use. Markerless motion capture systems have the potential to overcome these drawbacks. Recently, databases containing a large number of configurations of human whole body actions are available, and they are expected to be reused as new approaches to recognizing actions and recovering action configurations from motion depicted in videos comprising monocular images. This paper describes a design of an action database that consists of action configurations, pose descriptors from silhouette images, a stochastic model encoding each sequence of the pose descriptors, and relations between the data and the stochastic models. The proposed action database is applied to recognizing a video containing a performer doing a specific action and to recovering all the joint angles from the video. We tested the action database on recognition and configuration recovery, and the results show that the database is suitable for this purpose.