Learning Visual Features to Predict Hand Orientations

Abstract

This paper is a preliminary account of current work on a visual system that learns to aid in robotic grasping and manipulation tasks. Localized features are learned of the visual scene that correlate reliably with the orientation of a dextrous robotic hand during haptically guided grasps. On the basis of these features, hand orientations are recommended for future gasping operations. The learning process is instancebased, on-line and incremental, and the interaction between visual and haptic systems is loosely anthropomorphic. It is conjectured that critical spatial information can be learned on the basis of features of visual appearance, without explicit geometric representations or planning. 1. Introduction When a human reaches for an object, the hand is oriented and shaped appropriately in anticipation of the grasp. This anticipatory preconfiguration takes place before contact with the object is made, and is informed by visual cues. For example, when reaching for a v...