Automated hand detection method for tasks of gesture recognition in human-machine interfaces

摘要

Subject of Research. The paper presents a solution for automatic analysis and recognition of human hand gestures. Recognition of the elements of sign languages is a topical task in the modern information world. The problem of gesture recognition efficiency has not been resolved due to the presence of cultural diversities in the world sign languages, the differences in the conditions for showing gestures. The problem becomes more complicated by the small size of fingers. Method. The presented method is based on the analysis of frame sequences of a video stream obtained using an optical camera. For processing of the obtained video sequences, it is proposed to use a depth map and a combination of modern classifiers based on Single Shot MultiBox Detector deep neural network architectures with a reduced network model of ResNet-10, NASNetMobile and LSTM type. Main Results. Experiments on automatic video analysis of hand movements and gesture recognition in real time show great potential of the proposed method for human-machine interaction tasks. The recognition accuracy of 48 one-handed gestures based on TheRuSLan database is 79 %. This result is better as compared to the other approaches to solving this problem. Practical Relevance. The results can be used in automatic systems for recognition of sign languages, as well as in the situations where contactless interaction of various user groups is necessary, for example, people with hearing and vision impairments, mobile information robots through automatic recognition of sign information.