GPU-Accelerated Robotic Simulation for Distributed Reinforcement\n Learning

Abstract

Most Deep Reinforcement Learning (Deep RL) algorithms require a prohibitively\nlarge number of training samples for learning complex tasks. Many recent works\non speeding up Deep RL have focused on distributed training and simulation.\nWhile distributed training is often done on the GPU, simulation is not. In this\nwork, we propose using GPU-accelerated RL simulations as an alternative to CPU\nones. Using NVIDIA Flex, a GPU-based physics engine, we show promising\nspeed-ups of learning various continuous-control, locomotion tasks. With one\nGPU and CPU core, we are able to train the Humanoid running task in less than\n20 minutes, using 10-1000x fewer CPU cores than previous works. We also\ndemonstrate the scalability of our simulator to multi-GPU settings to train\nmore challenging locomotion tasks.\n