Accelerating Virtual Fixture Estimation for Robot Manipulation using Reinforcement Learning and Human Demonstrations

Abstract

Teleoperation has the potential to greatly simplify the programming of industrial robots by allowing experts to teach tasks from remote. In order to facilitate the accomplishment of a particular task, Virtual Fixtures (VFs) are often added to the teleoperation system, but designing them is a tedious process that requires time and expertise. In this paper, human teleoperated demonstrations are combined with Proximal Policy Optimization (PPO) to decrease the convergence time of a Reinforcement Learning (RL) algorithm that automatically estimates the parameters of a VF for a Pegin-Hole insertion task. The acceleration of the convergence allows us to also estimate parameters of more intricate VFs, where the RL algorithm did not converge otherwise. The experiments show that this technique can successfully infer the expected parameters of predefined VFs in a third of the time compared to the state-of-the-art approach. Additionally, the learned behaviour of the robot manipulator in simulation is executed on a robot arm using an impedance controller to demonstrate that the virtual system closely replicates its real counterpart, and that the learned VFs have higher chances of being useful to a user controlling the real system.