Characterization of Modeling Errors Affecting Performances of a Robotics Deep Reinforcement Learning Controller in a Sim-to-Real Transfer

Abstract

Simulation is a powerful tool used to train Reinforcement Learning (RL) agents involved in robotic tasks. It allows to collect large amount of data in comparatively faster and safer way than on the real robot. However, a simulator is only an approximation of the physical system to be controlled. Due to modeling errors, a controller learned on the simulator dynamics may behave differently once applied to the real robot. In the worst case, the controller, although being successful when applied on the simulator, may fail when applied on the real platform. In this paper, we deal with the sim-to-real transfer of a RL controller for a Poppy Ergo-Jr robotic arm involved in a positioning task: i.e., moving the servo joints in order to reach a desired target position with the end-effector. In particular, we want to investigate the differences between the real robot and its simulator and how they affect the controller performance after its transfer from the simulator to the real platform.