A Real-time Approach for Recognizing German Sign Language

Abstract

In this paper, an innovative approach utilizing artificial intelligence (AI) for the recognition of German Sign Language (GSL) gestures is presented, aimed at controlling an assistance robot. Sign language is an important method of communication for the hearing impaired and poses special challenges for automated recognition due to its complex and nuanced gestures. Leveraging advancements in deep learning techniques, particularly Long Short-Term Memory (LSTM) and The MediaPipe Holistic Landmarker for extracting hand, face, and pose landmarks, a robust GSL recognition system is proposed. The proposed model is trained to interpret a specified set of GSL gestures, focusing on common tasks or objects that an assistance robot can do or grab, respectively. Our findings demonstrate promising results, with the LSTM network achieving a validation accuracy of 96.55 % with minimal false positive classifications. This research contributes to the advancement of assistive technologies by harnessing the power of AI to pave the way for seamless integration of GSL into robotic control systems, empowering individuals with hearing impairments to interact intuitively with robotic platforms.