Prescient Teleoperation of Humanoid Robots

Abstract

Teleoperated humanoid robots are ideally suited to act as human avatars in remote environments. Unfortunately, their deployment is hindered by the communication delays between the human input and the video feedback from the robot. Here, we introduce a direct teleoperation system in which the operator receives a synchronized video feed of real images, even when the communication channel imposes a 1 to 2-second delay. Our key idea is to leverage machine learning to allow the robot to execute commands before the operator performs them, so that the operator receives a delayed video stream that is almost indistinguishable from realtime feedback. In our experiments, the iCub humanoid robot (32 degrees of freedom) was successfully controlled to perform several whole-body manipulation tasks, including reaching different targets, picking up an object, and moving a box. This new technique may enable real-life avatars on long-range radio networks, from remote maintenance to space missions.