Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement\n Learning

Abstract

Most successes in robotic manipulation have been restricted to single-arm\nrobots, which limits the range of solvable tasks to pick-and-place, insertion,\nand objects rearrangement. In contrast, dual and multi arm robot platforms\nunlock a rich diversity of problems that can be tackled, such as laundry\nfolding and executing cooking skills. However, developing controllers for\nmulti-arm robots is complexified by a number of unique challenges, such as the\nneed for coordinated bimanual behaviors, and collision avoidance amongst\nrobots. Given these challenges, in this work we study how to solve bi-manual\ntasks using reinforcement learning (RL) trained in simulation, such that the\nresulting policies can be executed on real robotic platforms. Our RL approach\nresults in significant simplifications due to using real-time (4Hz) joint-space\ncontrol and directly passing unfiltered observations to neural networks\npolicies. We also extensively discuss modifications to our simulated\nenvironment which lead to effective training of RL policies. In addition to\ndesigning control algorithms, a key challenge is how to design fair evaluation\ntasks for bi-manual robots that stress bimanual coordination, while removing\northogonal complicating factors such as high-level perception. In this work, we\ndesign a Connect Task, where the aim is for two robot arms to pick up and\nattach two blocks with magnetic connection points. We validate our approach\nwith two xArm6 robots and 3D printed blocks with magnetic attachments, and find\nthat our system has 100% success rate at picking up blocks, and 65% success\nrate at the Connect Task.\n