Learning multiple behaviours using hierarchical clustering of rewards

Abstract

Learning models of behaviours has many applications in robotics spanning both control, e.g. learning from demonstration and perception, e.g. monitoring and surveillance. Inverse reinforcement learning encodes behaviours as a reward function learned from a set of demonstrations. This paper addresses the problem of learning from unlabelled datasets containing an unknown number of behaviours in continuous action-state spaces. The proposed method uses a hierarchical clustering approach to directly group trajectories that share a common reward function. The similarity metric is based on the distribution of maximum entropy of the feature counts computed using path integrals. We evaluated the method in three different tasks: navigation on a set of synthetic maps, human driving styles on a simulator and human reaching. Results show that clustering in the reward space is able to discover the latent reward structure resulting in compact models that can generate all the observed behaviours.