Curriculum Design and Sim2Real Transfer for Reinforcement Learning in Robotic Dual-Arm Assembly

Abstract

Robotic systems are crucial in modern manufacturing. Complex assembly tasks require the collaboration of multiple robots. Their orchestration is challenging due to tight tolerances and precision requirements. In this work we set up two Franka Panda robots performing a peg-in-hole insertion task. We structure the control system hierarchically, planning the robots’s trajectories feedback-based with a central policy trained with reinforcement learning. These trajectories are executed by a low-level impedance controller on each robot. To enhance training convergence, we use reverse curriculum learning incorporating domain randomization, varying initial configurations of the task. After training, we test the system in a simulation, studying the impact of curriculum parameters on emerging process characteristics like process time and variance. Finally, we transfer the trained model to a real-world setup, comparing results with simulation as well as classical path planning and control approaches.