Development of an Optical Tracking Based Teleoperation System with Virtual Reality

Abstract

In this paper, we propose an optical tracking based teleoperation system with a virtual reality interface for mobile robots. The system allows the user to teleoperate a mobile robot using bare hands and the user can adjust the autonomy of the robot between two levels: direct control and autonomous navigation. A Leap Motion sensor based non-contact teleoperation method is developed to translate sensor messages into velocity commands to the robot in order to interact with a remote mobile robot in a natural manner. By incorporating HTC Vive virtual reality device, the user is fully immersed into the virtual space with visual feedback from the remote site. The system features cost-effective and extendable by leveraging commercial virtual reality devices and integrating it with open source robotic control middleware.