Human-in-the-Loop Training Leads to Faster Skill Acquisition and Adaptation in Reinforcement Learning-Based Robot Control

Abstract

This work presents a training procedure for reinforcement learning-based robot control to achieve skill acquisition with reduced training time and data requirements. It achieves this by actively incorporating the actions provided via human inputs in addition to actions generated by the learning algorithm. As the algorithm acquires the target skill, the contribution of the human actions is gradually decreased for autonomous task execution. To demonstrate the efficacy of the proposed approach, a ball-balancing task with manipulability index maximization was chosen. In this task, a 7-DoF robot arm achieved skill acquisition via reinforcement learning in which two distinct training approaches were employed: i) human-in-the-loop training approach, and ii) autonomous training. The task required the robot to bring the ball to the center of a tray attached to its end-effector, starting from arbitrary initial ball positions in the face of perturbations. Simulation experiments were carried out with 24 human participants where each participant guided robot learning in a human-in-the-loop shared control setting. Compared to autonomous training, the human-in-the-loop training approach showed superior performance in terms of reduced training time and data usage while exhibiting favorable ball-balancing skills.