Human action recognition from inertial sensors with Transformer

Abstract

Human action recognition is an attractive research topic because it opens many practical applications such as healthcare, entertainment or robot interaction. Hand gestures in particular are becoming one of the most convenient means of communication between humans and machines. In this study, transformer model - a deep learning neural network developed primarily for the natural language processing and vision tasks, is investigated for analysis of time-series signals. The self-attention mechanism inherent in the transformer expresses individual dependencies between signal values within time series. As a result, it can boost the performance of state-of-the-art convolutional neural networks in terms of memory requirement and computational times. We evaluate the proposed method on three published sensor datasets (CMDFALL, C-MHAD and DaLiAc) and showed that the proposed method achieves better performance than conventional ones, specifically on the S <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</inf> group in the CMDFall data set, the F1 Score is 19.04 % higher than that of the conventional method. On C-MHAD dataset, the accuracy is up to 99.56 %. The results confirms the role of transformer models for human activity recognition.