Vision based Human Activity Recognition using Hybrid Deep Learning

Abstract

Human activity recognition is a wide research area of computer vision that finds applications in smart surveillance system, healthcare, and human robotic interactions. Nowadays, deep learning methods have achieved more interest due to its ability of executing feature extraction and classification steps simultaneously. In this paper, we have focused on the vision based human activity recognition using deep learning algorithms. Long short term memory (LSTM) is a special form of recurrent neural networks (RNN), specifically designed for long term data dependencies. Also it is a known fact that among deep learning algorithms, convolutional neural networks (CNN) have earned high performance in image classification. To overcome the limitation of LSTM in case of classification of static images, a hybrid CNN-LSTM model is proposed in which features are firstly extracted through CNN and then feed to LSTM as a sequence by the means of time distributed layer. This model is utilized for classifying six activities from two datasets which have shown the accuracy of 96.24% and 93.39% on KTH and Weizmann datasets, respectively. We have also implemented the CNN and LSTM models separately on these datasets with same parameters as used in hybrid model to study their impact on accuracy and loss.