Real-time gesture control UAV with a low resource framework

Abstract

This study showcases a low-resource framework that enables people with no technical know-how to interact with drones, it also explores the capabilities of 2D- computer vision and deep learning techniques for gesture based interface systems on a low-cost micro drone with an onboard RGB camera. This Human-Robot Interaction system processes the real-time human pose to allow a user to command the drone, i.e., by providing direction to move and execute actions. A linear PD controller and image processing techniques are implemented to track humans whilst maintaining a safe distance from the user by perceiving depth information through pose estimation. We incorporated the gesture recognition results into a drone using the Robot Operating System (ROS) and evaluated system performance indoor and outdoor. This low computation framework can be applied further to control robotic arms or mobile robots.