Diver Gestures Recognition in Underwater Human-Robot Interaction Using Recurrent Neural Networks

Abstract

The topic of underwater human-robot collaboration is becoming more popular. Collaborative robots can significantly improve the efficiency and safety of working underwater. However, the problem of interpretation signals from human to robot remains open. Existing solutions mainly face limitations due to the need for additional devices and training to transmit commands from humans to robots. This article describes a real-time method for recognizing standard diver hand signals in a video stream from an optical camera on an underwater robot. An image from an underwater video camera is transmitted to the input of a convolutional neural network for recognizing a person's posture. Next, a sequence of poses is fed to the input of a recurrent neural network to classify a diver's gesture. The article discusses the classification of gestures produced only by large parts of the hands (shoulder, forearm). The article also compares the accuracy of various architectures of recurrent neural networks. The proposed solution greatly facilitates human-robot interaction.