Mapping System with Virtual Reality for Mobile Robot Teleoperation

Abstract

Robots using teleoperation, especially linking robots to users in real-time have been studied. For a convenient and immersive operation to a robot, it should be able to use sensors which are mounted on the robot, and effectively visualize it to increase the immersive feeling of the user. We propose a system that creates a map using a camera and a sensor installed in the robot, recognizes the surrounding environment. It visualizes PointCloud data in real time on the user’s virtual reality device. To demonstrate our proposed system, we used TurtleBot and Gear VR as robot and user device, respectively. Also, a laptop connected with TurtleBot processes the PointCloud data obtained from Depth Camera to Octomap, sends it to the VR device, and the VR device visualizes received data in real time. We experimented in a real environment by creating a VR-based teleoperation system that remotely manipulates TurtleBot and visualizes the data. The transmitted PointCloud data were reflected in the VR device in real time so that the user can remotely recognize the situation of the site.