A Sim-to-Real Learning-Based Framework for Contact-Rich Assembly by Utilizing CycleGAN and Force Control

Abstract

Deep reinforcement learning (RL) has succeeded in robotic manipulation applications. However, training robots in the real world is challenging due to sample efficiency and safety concerns. Sim-to-real transfer has been proposed to address the aforementioned concerns but introduces the reality gap. In this work, we introduce a sim-to-real learning framework for vision-based assembly tasks and perform training in a simulation environment by employing raw image inputs from a single camera to address the aforementioned issues. We build a robotic Peg-in-Hole (PiH) training environment that requires low-level simulation knowledge. We also present a domain adaptation method based on a cycle-consistent generative adversarial network (CycleGAN) and a force control transfer approach to bridge the reality gap. The proposed framework, trained in a simulation environment with different environmental scenes, can be successfully transferred to a real PiH setup with a UR5e robot. We then reproduce these results with a Diana7 robot and different peg shapes to verify the generalization ability of the framework.