Semi-Supervised Haptic Material Recognition for Robots using Generative\n Adversarial Networks

Abstract

Material recognition enables robots to incorporate knowledge of material\nproperties into their interactions with everyday objects. For example, material\nrecognition opens up opportunities for clearer communication with a robot, such\nas "bring me the metal coffee mug", and recognizing plastic versus metal is\ncrucial when using a microwave or oven. However, collecting labeled training\ndata with a robot is often more difficult than unlabeled data. We present a\nsemi-supervised learning approach for material recognition that uses generative\nadversarial networks (GANs) with haptic features such as force, temperature,\nand vibration. Our approach achieves state-of-the-art results and enables a\nrobot to estimate the material class of household objects with ~90% accuracy\nwhen 92% of the training data are unlabeled. We explore how well this approach\ncan recognize the material of new objects and we discuss challenges facing\ngeneralization. To motivate learning from unlabeled training data, we also\ncompare results against several common supervised learning classifiers. In\naddition, we have released the dataset used for this work which consists of\ntime-series haptic measurements from a robot that conducted thousands of\ninteractions with 72 household objects.\n