Methodology for finger gesture control of mechatronic systems

Abstract

This paper proposes a methodology for the real-time finger gesture following and control of mechatronic systems based on computer vision and machine learning techniques. The goal of this research is to develop a human-machine interface that could be able to control a mechatronic system by performing finger gestures in space or on a surface without the use of any kind of keyboard neither a joystick. The finger gestures will be continuously followed and directly mapped with commands of mechatronic systems such as start moving, stop moving, forward moving, backward moving etc. The proposed methodology relies on the finger gesture data acquisition, hand segmentation, fingertips localization/ identification, high-level feature extraction, early recognition and prediction using machine learning techniques and its integration into a mechatronic system. The LEGO MINDSTORMS NXT robotics platform controlled by matlab software could be used in the proposed methodology.