HiLoTEL: Virtual Reality Robot Interface-Based Human-in-the-Loop Task Execution and Learning in the Physical World Through Its Digital Twin

Abstract

HiLoTEL is a flexible virtual-reality framework for executing and learning tasks in virtual and physical environments. It enables hu-man experts to collaborate with learning agents and intervene when necessary through human-in-the-loop imitation learning. HiLoTEL reduces the need to carry out repetitive tasks, providing the user with an intuitive supervision interface. The system is tested on Pick-and-Place task, considering both teleoperated and passthrough interaction modalities. The results show that HiLoTEL improves success rates while maintaining human-level completion time and providing users with 71% hands free supervision time, thus enabling effective human-robot collaboration.