Human Activity Recognition for Indoor Robotics: A Deep Learning Based Approach Using a Human Detection Stage

Abstract

In this paper, it is proposed a Deep-Learning (DL) based Human Activity Recognition (HAR) framework which uses as input RGB video data. It consists on a two-stage cascade approach: 1) a human detection stage where a bounding box, containing the person performing the activity, is extracted from each frame; 2) a DL-based feature extraction and classification stage. The main focus is on daily activities that are useful for social, service and assistive robots operating in indoor environments. Three different frameworks, including the one proposed, are evaluated and compared. Seven different DL architectures for the feature extraction and classification step are trained and tested. Training and testing is performed on the large scale NTU RGB+D 120 [1] action recognition dataset. The runtime performance of each framework is also evaluated by calculating the number of samples that each processes per second. Obtained results show that the proposed framework improves the classification accuracy up to 22.6 percentage points (depending on the DL architecture used for the feature extraction and classification module). The reported real-time performances indicate that a real-world scenario application for the evaluated frameworks is possible, given the current technology available in laptop units.