PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable\n Physics

Abstract

Simulated virtual environments serve as one of the main driving forces behind\ndeveloping and evaluating skill learning algorithms. However, existing\nenvironments typically only simulate rigid body physics. Additionally, the\nsimulation process usually does not provide gradients that might be useful for\nplanning and control optimizations. We introduce a new differentiable physics\nbenchmark called PasticineLab, which includes a diverse collection of soft body\nmanipulation tasks. In each task, the agent uses manipulators to deform the\nplasticine into the desired configuration. The underlying physics engine\nsupports differentiable elastic and plastic deformation using the DiffTaichi\nsystem, posing many under-explored challenges to robotic agents. We evaluate\nseveral existing reinforcement learning (RL) methods and gradient-based methods\non this benchmark. Experimental results suggest that 1) RL-based approaches\nstruggle to solve most of the tasks efficiently; 2) gradient-based approaches,\nby optimizing open-loop control sequences with the built-in differentiable\nphysics engine, can rapidly find a solution within tens of iterations, but\nstill fall short on multi-stage tasks that require long-term planning. We\nexpect that PlasticineLab will encourage the development of novel algorithms\nthat combine differentiable physics and RL for more complex physics-based skill\nlearning tasks.\n