Evaluating task-agnostic exploration for fixed-batch learning of\n arbitrary future tasks

Abstract

Deep reinforcement learning has been shown to solve challenging tasks where\nlarge amounts of training experience is available, usually obtained online\nwhile learning the task. Robotics is a significant potential application domain\nfor many of these algorithms, but generating robot experience in the real world\nis expensive, especially when each task requires a lengthy online training\nprocedure. Off-policy algorithms can in principle learn arbitrary tasks from a\ndiverse enough fixed dataset. In this work, we evaluate popular exploration\nmethods by generating robotics datasets for the purpose of learning to solve\ntasks completely offline without any further interaction in the real world. We\npresent results on three popular continuous control tasks in simulation, as\nwell as continuous control of a high-dimensional real robot arm. Code\ndocumenting all algorithms, experiments, and hyper-parameters is available at\nhttps://github.com/qutrobotlearning/batchlearning.\n