A Deep CNN-Based Hand Gestures Recognition Using High-Resolution Thermal Imaging

Abstract

Hand gesture recognition has numerous applications that include automotive user interfaces, Human-computer interaction, and industrial robotics. These applications often make use of vision-based images, such as depth camera hand gesture photographs and red-blue-green (RGB) images. However, the performance of RGB and depth images is limited in dimly lit environments and with complicated backdrops. Consequently, it might be difficult to recognize hand motions in dimly lit environments with complicated backdrops, particularly when employing RGB/depth images. Therefore, the design and implementation of an end-to-end edge computing system that can precisely classify hand movements taken under complicated background conditions using a high-resolution infrared camera is described in this study. Under difficult backgrounds, a 4800 picture thermal dataset representing 10 classes is constructed, with 480 thermal photos for each sign. The proposed method classifies hand gestures using a novel lightweight deep convolutional neural network model using live images from the thermal camera stream. Our findings demonstrate that the proposed CNN model achieves an accuracy of 99.79% on the test dataset, along with the added advantage of being insensitive to ambient lighting. Finally, we assess the performance of the suggested lightweight CNN model in terms of inference time and classification accuracy against the pre-trained models.