Extracting Multimodal Dynamics of Objects Using RNNPB

Abstract

Dynamic features play an important role in recognizing objects that have similar static features in color or shape. This paper focuses on active sensing that exploits the dynamic feature of an object. An extended version of the robot, Robovie-IIs, uses its arms to move an object and determine its dynamic features. At issue is how to extract symbols from different temporal states of the object. We use a <I>recurrent neural network with parametric bias</I> (RNNPB) that generates self-organized nodes in parametric bias space. We trained an RNNPB with 42 neurons using data on sounds, trajectories, and tactile sensors generated while the robot was moving or hitting an object with its arm. Clusters of 20 types of objects were self-organized. Experiments with unknown (untrained) objects showed that our proposal configured them appropriately in PB space, demonstrating its <I>generalization</I>.