Registration of planar virtual fixtures by using augmented reality with dynamic textures

Abstract

This paper presents a method to align a virtual plane with a real plane in an augmented reality environment. The method addresses the challenge of configuring a planar virtual fixture during the teleoperation of a robot in a visually challenging environment. The method assists the operator to manually align the virtual plane with the real plane by providing visual cues by dynamically texturing the virtual plane. Following the manual initialization a robust evaluation of the optical flow is used to refine the alignment of the virtual plane. The system combines a stereo camera with a stereo display and generates visual distortions in the displayed images to indicate a misalignment of the virtual plane with the real one. The distortion is computed by displaying warped images from the left camera in the right display and vice versa. Results demonstrate that the method improves the accuracy of the alignment with a real plane that is covered with reflective multi-layer insulation.