Multi-View Contrastive Learning from Demonstrations

Abstract

This paper presents a framework for learning visual representations from unlabeled video demonstrations captured from multiple viewpoints. We show that these representations are applicable for imitating robotic tasks. We use contrastive learning to enhance task-relevant information while suppressing irrelevant information in the feature embeddings. We validate the proposed method on the publicly available Multi-View Pouring and a custom Pick and Place data sets and compare it with the TCN and CMC baselines. We evaluate the learned representations using three metrics: viewpoint alignment, stage classification and reinforcement learning. In all cases, the results improve when compared to state-of-the-art approaches.