Addressing Sample Complexity in Visual Tasks Using HER and Hallucinatory\n GANs

Abstract

Reinforcement Learning (RL) algorithms typically require millions of\nenvironment interactions to learn successful policies in sparse reward\nsettings. Hindsight Experience Replay (HER) was introduced as a technique to\nincrease sample efficiency by reimagining unsuccessful trajectories as\nsuccessful ones by altering the originally intended goals. However, it cannot\nbe directly applied to visual environments where goal states are often\ncharacterized by the presence of distinct visual features. In this work, we\nshow how visual trajectories can be hallucinated to appear successful by\naltering agent observations using a generative model trained on relatively few\nsnapshots of the goal. We then use this model in combination with HER to train\nRL agents in visual settings. We validate our approach on 3D navigation tasks\nand a simulated robotics application and show marked improvement over baselines\nderived from previous work.\n