Towards real-time robot simulation on uneven terrain using neural networks

Abstract

Simulation is a valuable tool for robotics research and development, and various simulation packages have been proposed. However, we are aware of no freely-available packages which implement the required fidelity to accurately model earth-moving robots that manipulate the terrain itself. The software which does exist for this is difficult if not impossible to run in real-time while achieving the desired accuracy. This paper proposes a simulation system in which a neural network is trained using data generated in a 3D high-fidelity, non-real-time simulator. The resulting neural network is used to accurately predict the motion of a robot in a 2D simulator, while also taking into consideration a height-field representing a 3D terrain. Using a trained neural network to drive the new simulation provides considerable speedup over the high-fidelity 3D simulation, allowing behaviour to be simulated in real-time while still capturing the physics of the agents and the environment.