Web-Based Remote Assistance to Overcome Robot Perceptual Limitations

Abstract

This paper addresses the problem of overcoming vi-sual perception limitations in service robots with re-mote assistance from human users. In particular, con-sider a scenario where a user requests the robot to per-form some task that requires a perceptual ability, e.g., check if a specific mug, “my mug, ” is in the lab or in an office, but the robot may not know how to recog-nize that object. We propose to equip the robots with the abilities to: (i) identify their own perceptual limi-tations, (ii) autonomously and remotely query human users for assistance, and (iii) learn new object descrip-tors from the interaction with humans. We successfully developed a complete initial version of our approach on our CoBot service mobile robot. The interaction with the user builds upon our previously developed semi-autonomous telepresence image sharing and control. The user can now further identify the object and the robot can save the descriptor and use it in future situ-ations. We illustrate our work with the task of learning to identify an object in the environment, and to report its presence to a user. Our ongoing work includes ad-dressing a dynamic interaction between the robot and the remote user for visual focus of attention and differ-ent object viewing, as well as the effective storage, la-beling, accessing, and sharing of multiple learned object descriptors, in particular among robots. Our goal is also to contribute the learned knowledge to crowd-robotics efforts.