Human Activity Recognition Using Ensemble of CNN-Based Transfer Learning Models

Abstract

Human activity recognition (HAR) models offer knowledge about human actions that can be used to analyze a person's behavior in a real-world setting. It can be applied in medical assistance, virtual reality, robotics, scene classification, sports analysis, and expert systems. Among various models currently transfer learning based HAR approaches are getting popular among researchers due to the benefits of taking less time and computational resources. In this paper, we propose a neural architecture that exploits the benefits of various convolutional neural network (CNN) based transfer learning models for recognizing human activities. We utilize three pre-trained transfer learning models including ResNet50V2, MobileNetV2, and EfficientNet BO, and fine-tuned them for our human activity recognition task to extract the effective task-specific contextual features. Later, a linear classifier is coupled with each model to get the prediction labels. Finally, we apply a fusion scheme through majority voting to identify the activity label. We conducted our experiments using a human activity detection dataset from Kaggle <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://tinyurl.com/HARDataset. It contains a total of 18000 images with 15 different classes. Experimental results demonstrated the efficacy of our approach over standard baselines. We also provide various insightful findings that might be beneficial for others working in this problem domain.