Learning Time-Invariant Reward Functions through Model-Based Inverse\n Reinforcement Learning

Abstract

Inverse reinforcement learning is a paradigm motivated by the goal of\nlearning general reward functions from demonstrated behaviours. Yet the notion\nof generality for learnt costs is often evaluated in terms of robustness to\nvarious spatial perturbations only, assuming deployment at fixed speeds of\nexecution. However, this is impractical in the context of robotics and\nbuilding, time-invariant solutions is of crucial importance. In this work, we\npropose a formulation that allows us to 1) vary the length of execution by\nlearning time-invariant costs, and 2) relax the temporal alignment requirements\nfor learning from demonstration. We apply our method to two different types of\ncost formulations and evaluate their performance in the context of learning\nreward functions for simulated placement and peg in hole tasks executed on a\n7DoF Kuka IIWA arm. Our results show that our approach enables learning\ntemporally invariant rewards from misaligned demonstration that can also\ngeneralise spatially to out of distribution tasks.\n