Learning Language-Conditioned Robot Behavior from Offline Data and\n Crowd-Sourced Annotation

Abstract

We study the problem of learning a range of vision-based manipulation tasks\nfrom a large offline dataset of robot interaction. In order to accomplish this,\nhumans need easy and effective ways of specifying tasks to the robot. Goal\nimages are one popular form of task specification, as they are already grounded\nin the robot's observation space. However, goal images also have a number of\ndrawbacks: they are inconvenient for humans to provide, they can over-specify\nthe desired behavior leading to a sparse reward signal, or under-specify task\ninformation in the case of non-goal reaching tasks. Natural language provides a\nconvenient and flexible alternative for task specification, but comes with the\nchallenge of grounding language in the robot's observation space. To scalably\nlearn this grounding we propose to leverage offline robot datasets (including\nhighly sub-optimal, autonomously collected data) with crowd-sourced natural\nlanguage labels. With this data, we learn a simple classifier which predicts if\na change in state completes a language instruction. This provides a\nlanguage-conditioned reward function that can then be used for offline\nmulti-task RL. In our experiments, we find that on language-conditioned\nmanipulation tasks our approach outperforms both goal-image specifications and\nlanguage conditioned imitation techniques by more than 25%, and is able to\nperform visuomotor tasks from natural language, such as "open the right drawer"\nand "move the stapler", on a Franka Emika Panda robot.\n