Unity-Based Mixed Reality HRI for Underwater Exploration and Object Recovery Using a I-AUV and Hololens

Abstract

Inspection, Maintenance, and Repair (IMR) of underwater structures is a growing research area. In recent years, more underwater robots capable of performing manipulation tasks have been developed by both research centers and industry, most of which are Autonomous Underwater Vehicles (AUVs). Most scientific efforts have focused on increasing the autonomy of these robots. However, another important research line focuses on teleoperation, exploring different ways for human operators to control the robot, including sensing what the robot is touching or representing a holographic view of the robot in the sensed environment, allowing it to be controlled through human hand or body movements. The complexity of Mixed Reality teleoperation increases significantly when operating over low-bandwidth communication links, which are typical in underwater environments. Merging these two worlds—the autonomous behavior of robots and Mixed Reality-based teleoperation—opens the possibility of sharing control between the robot and the human operator. This article presents a shared autonomy approach where the operator can visualize the intervention environment and teleoperate the robot using Mixed Reality, thanks to a Unity-based Human-Robot Interface (HRI). The application focuses on exploration and object recovery tasks, where the operator can trigger different autonomous behaviors, such as moving the robot to a point, exploring an area, or even recovering an object.