One-Handed Wonders: A Remote Control Method Based on Hand Gesture for Mobile Manipulator

Abstract

This paper proposes a remote control method based on one-handed gestures for mobile manipulator, so that the operator can control the entire robotic system with only one hand. In this study, we combine the real-time hand key points detection technology provided by MediaPipe with the RealSense D435i depth camera to address the inaccuracy in depth recognition problem of the original method. Then, we analyze the position, pitch, and rotation of the hand to generate corresponding control commands. And in order to use specific gestures for switching between controlled objects, we propose a lightweight gesture recognition model based on Gated Recurrent Unit (GRU), achieving $\mathbf{1 0 0 \%}$ recognition rate for three types of gestures. Finally, we conduct physical experiments on a mobile manipulator operating platform to preliminarily verify the effectiveness of the proposed method.