Automatic action recognition for assistive robots to support MCI patients at home

Abstract

This paper presents a novel approach for automatic human action recognition, focusing on user behaviour monitoring needs of assistive robots that aim to support Mild Cognitive Impairment (MCI) patients at home. Our action recognition method utilizes the human's skeleton joints information, extracted from a low-cost depth sensor mounted on a service robot. Herein, we extend the state of art EigenJoints descriptor to improve recognition robustness for a series of actions involved in common daily activities. Specifically, we introduce novel features, so as to take into account action specificities such as the joints' travelled distance and their evolution trend in subsequent frames to the reference one. In addition, we use information related to the user's manipulated objects, taking into account that several actions may be similar, yet performed with different objects, as well as the fact that real, practical applications involve continuous input video streams rather than pre-segmented action sequences. Through experimental evaluation on the MSR Action3D dataset, our approach has been found to outperform the state of art in action recognition performance. Evaluation has also been performed on a custom dataset, providing further promising results for future practical applications of our overall action recognition framework.