A real-time stereoscopic viewer for telerobotics using open source software

Abstract

Robots are deployed to perform repeatable as well as hazardous tasks because of their inherent safety, reliability, consistency and accuracy. Recently robots are manipulated remotely using the binocular stereo vision. In this paper, we propose and implement a real-time stereoscopic viewer for telerobotics using open source software. We manipulate the KUKA KR-6 robot remotely using the Phantom haptic device. We assemble our stereo camera using a pair of IP cameras. We make use of the GStreamer multimedia framework for capturing and processing image streams coming from these IP cameras. We utilize the OpenCV library for camera calibration, stereo calibration and real-time stereo rectification. We extend predefined widgets from the Qt framework with our own widgets for the OpenGL based Quad buffering. We apply the software engineering design patterns for development of our in-house stereoscopic viewer. We also discuss the stereo vision using the DLP Link, the NVIDIA 3D Vision and the NVIDIA 3D Vision Pro with their pros and cons.