Augmented reality-enhanced structural inspection using aerial robots

Abstract

This paper investigates the arising potential when automated path planning for aerial robotic structural inspection is combined with an Augmented Reality interface that provides live feed of stereo views fused with real-time 3D reconstruction data of the environment, while allowing seamless on-the-fly adaptation of the next robot viewpoints using intuitive head motions. The proposed solution aims to address the problem of accurate inspection and mapping of structures and environments for which a prior model exists but is not accurate, potentially outdated, or does not encode important features and semantics such as human-readable indications and other texture information. To approach the problem, the robot computes an optimized inspection path given any prior knowledge of the environment, while the human operator utilizes the live camera views and the real-time derived 3D map data to locally adjust the reference trajectory of the robot, such that it visits an updated set of viewpoints which provides the desired coverage of the real environment and sufficient focus on certain features and details. An autonomous aerial robot capable of navigation and mapping in GPS-denied environments is employed and combined with the Augmented Reality interface to experimentally demonstrate the potential of the approach in structural inspection applications.