Experiments in socially guided exploration: lessons learned in building robots that learn with and without human teachers

Abstract

We present a learning system, Socially Guided Exploration, in which a social robot learns new tasks through a combination of self-exploration and interpersonal interaction. The system’s motivational drives (novelty, mastery), along with social scaffolding from a human partner, bias behavior to create learning opportunities for a Reinforcement Learning mechanism. The robot is able to learn on its own, but can flexibly use the guidance of a human teacher to improve performance. We report the results of a series of experiments where the robot learns on its own in addition to being taught by human subjects. We analyze these interactions to understand human teaching behavior and the social dynamics of the human-teacher/robot-learner system. With respect to learning performance, human guidance results in a task set that is significantly more focused and efficient, while self-exploration results in a broader set. Analysis of human teaching behavior reveals insights of social coupling between human teacher and robot learner, different teaching styles, strong consistency in the kinds and frequency of scaffolding acts across teachers, and nuance in the communicative intent behind positive and negative feedback.