Deep Learning-Based Unmanned Aerial Vehicle Control with Hand Gesture and Computer Vision

Abstract

Human drone Interaction is the subfield of Human-robot interaction (HRI), which deals with human interaction with drones; Drones are conventionally controlled by joysticks, onboard computers, mobile applications, and remote controllers. Drones controlled by conventional methods are affected by electromagnetic wave interference due to an unreliable connection between the drone and the controller. Special care is needed when working with drones in close vicinity to humans. In this proposed work, we have developed an agentless and non-wearable communication system between the drone and the user. The proposed system consists of two modules, i.e., the hand recognition module and drone controlling module. The hand recognition model is further subdivided into two modules, i.e., hand detection and gesture recognition. After the landmarks (palm and hand detection) are detected, they are given to the TensorFlow-based Deep learning model for classification among different hand gestures. The classified hand movement is further converted into actionable drone movements such as take-off, hovering, and landing. Experimentation was done on Parrot Mambo Drone, and it was concluded that there should be no more than 121.92cm distance between user and webcam for working system effectively.