Learning Appearance Features to Support Robotic Manipulation

Abstract

A predominant purpose of vision is to facilitate interaction with the world. Motivated by biological vision, we argue that many common vision-guided activities can be carried out directly on the basis of features of appearance, and do not require elaborate world models. We describe a visual feature learning system that supports a hapticallyguided, dextrous robotic grasping system. It learns features, combinations of Gaussian-derivative filter responses, that correlate well with successful grasping parameters. Without explicit knowledge of object identities or categories, the system learns to propose object-specific grasp parameters, considerably improving the quality of hapticallyguided grasps with respect to the “blind” system. The combined system is loosely anthropomorphic in that it is guided by vision for hand pre-shaping, and by haptics during execution of a grasp, without explicit object recognition, scene reconstruction, or path planning.