Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue

Abstract

This paper describes an approach for a robotic arm to learn new actions through dialogue in a simplified blocks world. In particular, we have developed a three-tier action knowledge representation that on one hand, supports the connection be-tween symbolic representations of lan-guage and continuous sensorimotor repre-sentations of the robot; and on the other hand, supports the application of existing planning algorithms to address novel situ-ations. Our empirical studies have shown that, based on this representation the robot was able to learn and execute basic actions in the blocks world. When a human is engaged in a dialogue to teach the robot new actions, step-by-step instructions lead to better learning performance compared to one-shot instructions. 1