A Lightweight Vision-Based Measurement for Hand Gesture Information Acquisition

Abstract

The utilization of hand gesture recognition (HGR) can achieve a simpler and more humanized human–computer interaction (HCI) system, which can be used to simplify some aspects of production life and can also be used for sign language interpretation robots. However, HGR has a highly customized design. It is a very time-consuming process to design and train a separate model for each set of datasets. Therefore, designing a hand gesture information (HGI) acquisition instrument to analyze the rigid body motion and internal posture of the hand at the same time is a necessary process. This article presents a lightweight vision-based measurement (VBM) system to achieve this function. The system is implemented based on the convolutional neural network (CNN) structure, the improved Kalman filter (KF) is used to compensate error of fast moving, and the testing effect is good. After testing, the improved hand position detection system has the highest recall rate in the single-stage network. Compared with the original you look only once (YOLO)-v3, the number of parameters is reduced by 46.7M, but the recall rate is increased by 0.28%, which solves the loss of hand detection under complex background. With the help of the improved KF algorithm, the average precision (AP) metric of the prediction box is improved by 2.0 compared with the previous one. Compared with the simple baseline, the area under curve (AUC) metric of the keypoints detector is improved by 0.5%.