Mid-Air Gesture Based Multi-Finger Control System For Paralyzed Patients Using Leap Motion

Abstract

Social security, mobile technology, system security, art and culture, sign language recognition, defense technologies, and soon have all recently paid a great deal of attention to image processing, and specifically Air Gesture Recognition (HGR). One of the most important uses of Air movements is in sign language, and this study has implications for resolving issues with this vital communication modality. However, its lack of universal comprehension undermines the entire sign language environment. The computer’s ability to recognise Air gestures is being put to good use in this area. Therefore, it is necessary to develop an adequate Air gesture detection system for use in robotics, gaming and virtual reality, human computer interaction, sharing information contained in historical artefacts, etc. Current methods for recognising Air gestures typically rely on either a sensor based or a Vision-based approach. Sensor-based approaches, such as gloves, Inertial Measurement Unit (IMU), Electromyography (EMG), Wi-Fi, etc., record the location and motion of the Air, while vision-based approaches acquire the Air gesture using a camera, webcam, etc. The sensor based method is more efficient because it eliminates the need for pre-processing and segmentation; nevertheless, it is prohibitively expensive to set up a dedicated laboratory for this purpose, and it can be unpleasant for users to wear bulky sensors for extended periods of time. Because they encode texture and colour aspects of Air gesture for recognition, vision-based techniques are cost-effective. However, there are a number of obstacles that must be overcome in order to create a vision-based Air gesture detection system. The study’s overarching objective is to enhance existing approaches to problem-solving and system efficiency in order to design a system that can successfully navigate these obstacles. Pre-processing, feature Extraction, and classification are the three main building blocks of any Air gesture detection system. In this study, we provide LMS (Leap Motion sensor), a feature-based descriptor that was built from the ground up. The static Air gesture identification problem is addressed by suggesting an Extended Radial mean response pattern. When compared to state-of-the-art Aircrafted methods, the suggested descriptor improves recognition system robustness to noise and light fluctuations.