Cosine Similarity Based Representation Learning for Adversarial Imitation Learning

Abstract

Adversarial imitation learning (AIL) aims to recover the reward signal from expert demonstrations and learn expert policy by employing reward and reinforcement learning. However, the raw state-action features of the demonstrations usually have redundant information for a particular control task, and therefore the reward learned from the raw features is often biased, which eventually results in low sample efficiency and instability in AIL. To address these issues, we present CSAIL: Cosine Similarity based Adversarial Imitation Learning. CSAIL extracts expert policy representations from demonstrations via a novel cosine similarity based loss and recovers a robust and unbiased reward function by the learned representations. Based on the reward, CSAIL mimics the expert policy by the Wasserstein distance optimization method. Experimental results show that CSAIL outperforms existing state-of-the-art AIL methods on challenging Mujoco robot control and autonomous driving tasks.