A rule based fuzzy gesture recognition system to interact with Sphero 2.0 using a smart phone

Abstract

In this study, we will present a rule based fuzzy gesture recognition system where a user will interact with a spherical robot with hand gestures performed with a smart phone and the droid will respond by imitating this movements. In this context, we will take up the Gesture Recognition, Fuzzy Logic and Internet of Things (IoT) frameworks to construct such a Human-Machine Interface (HMI). In the proposed structure, the IoT collect the necessary IMU data from the smart phone for classification purposes while also providing the necessary data to the Sphero 2.0 droid. To recognize/classify the hand gestures performed with a smart phone, we will use and train fuzzy classifier. For proof of concept purposes, we have defined two hand gesture movements which are circular and linear gesture movement. The presented results clearly show that the performance of the fuzzy classifier is satisfactory even though each user has unique gesture characteristic (different magnitudes and velocities). Finally, we have also tested the proposed system in real-time with a user from whom we have not collected any IMU data. The presented results of the paper will show that the performance fuzzy logic based gesture recognition and interaction system is satisfactory.