A System for Automatic Recognition of Robotic Hand Gesture Based on Raspberry Pi and Convolutional Neural Network by Using Specklegram

Abstract

A robotic hand gesture recognition system is developed on a Raspberry Pi by effectively analyzing the specklegrams from five single-mode–multimode fiber (S–M) structures fixed on the robotic hand. The five S–M structures act as the bionic neuron to percept the bending of each robotic finger, and the specklegrams from them are detected by a microcamera connected to the Raspberry Pi and analyzed simultaneously. A large amount of specklegrams obtained from different robotic hand gestures are then used to train a convolutional neural network (CNN) running on the Raspberry Pi to recognize the hand gesture. The proposed system exhibits a high recognition accuracy (99.6%) for 30 hand gestures and demonstrates the capability to recognize a slight change in hand gesture with a resolution of 0.36°. Finally, the reliability of the system was evaluated through the stability experiments. By recognizing the specklegram of the robotic hand after multiple movements and a long period of time, the repeatability and stability of the system are also proved. The proposed system provides a compact and low-cost alternative to conventional systems, which may find a potential application in the robotic hand gesture recognition, human motion recognition, and intelligent glove.