Extending Policy from One-Shot Learning through Coaching

Abstract

Humans generally teach their fellow collaborators to perform tasks through a small number of demonstrations, often followed by episodes of coaching that tune and refine the execution during practice. Adopting a similar framework for teaching robots through demonstrations makes teaching tasks highly intuitive and imitating the refinement of complex tasks through coaching improves the efficacy. Unlike traditional Learning from Demonstration (LfD) approaches which rely on multiple demonstrations to train a task, we present a novel one-shot learning from demonstration approach, augmented by coaching, to transfer the task from task expert to robot. The demonstration is automatically segmented into a sequence of a priori skills (the task policy) parametrized to match task goals. During practice, the robotic skills self-evaluate their performances and refine the task policy to locally optimize cumulative performance. Then, human coaching further refines the task policy to explore and globally optimize the net performance. Both the self-evaluation and coaching are implemented using reinforcement learning (RL) methods. The proposed approach is evaluated using the task of scooping and unscooping granular media. The self-evaluator of the scooping skill uses the realtime force signature and resistive force theory to minimize scooping resistance similar to how humans scoop. Coaching feedback focuses modifications to sub-domains of the action space, using RL to converge to desired performance. Thus, the proposed method provides a framework for learning tasks from one demonstration and generalizing it using human feedback through coaching achieving a success rate of ≈90%.