Multi-modal stereognosis

摘要

Introduction In this work, vision and touch (artificial) senses are integrated in a cooperative active system. Multi-modal sensory information acquired on-line is used by a robotic agent to perform real-time tasks involving categorization of objects. The visual-touch system proposed is able to foveate (verge) the eyes onto an object, to move the arms to touch an object, and to choose another object by shifting its focus of attention. Also, the system can detect changes occurred on a previously visited region. We propose a hybrid computational system in which an associative memory remembers the visual and tactile signature of objects for recognition and reinforcement learning tasks (Q-learning) are formulated to learn active sensing policies. Several vision architectures have been proposed recently. Kosslyn's architecture [6], based on results in experimental neuro-psychology, seems to be technically possible with special hardware. Despite practical issues it is intuitively att