Learning to Manipulate Object Collections Using Grounded State\n Representations

Abstract

We propose a method for sim-to-real robot learning which exploits simulator\nstate information in a way that scales to many objects. We first train a pair\nof encoder networks to capture multi-object state information in a latent\nspace. One of these encoders is a CNN, which enables our system to operate on\nRGB images in the real world; the other is a graph neural network (GNN) state\nencoder, which directly consumes a set of raw object poses and enables more\naccurate reward calculation and value estimation. Once trained, we use these\nencoders in a reinforcement learning algorithm to train image-based policies\nthat can manipulate many objects. We evaluate our method on the task of pushing\na collection of objects to desired tabletop regions. Compared to methods which\nrely only on images or use fixed-length state encodings, our method achieves\nhigher success rates, performs well in the real world without fine tuning, and\ngeneralizes to different numbers and types of objects not seen during training.\n