Reinforcement learning for improving imitated in-contact skills

Abstract

Although motor primitives (MPs) for trajectory-based skills have been studied extensively, much less attention has been devoted to studying in-contact tasks. With robots becoming more commonplace, it is both economical and convenient to have a mechanism for learning an in-contact task from demonstration. However, transferring an in-contact skill such as wood planing from a human to a robot is significantly more challenging than transferring a trajectory-based skill; it requires a simultaneous control of both pose and force. Furthermore, some in-contact tasks have extremely complex contact environments. We present a framework for imitating an in-contact skill from a human demonstration and automatically enhancing the imitated force profile using a policy search method. The framework encodes both the the demonstrated trajectory and the normal contact force using Dynamic Movement Primitives (DMPs). In experiments, we utilize Policy Improvement with Path Integral (PI <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) algorithm for updating the imitated force policy. Our results demonstrate the effectiveness of this approach in improving the performance of a wood planing task. After only two update rounds, all the updated policies have outperformed the imitated policy at a significance level of P <; 0.001.