Acquiring the ability of object localization by vision and audition through motion

Abstract

Abstract This paper proposes a sound source localization learning model based on iterated perception by vision and audition through neck rotation. Learning is performed through object localization by vision and audition with motion. The model is composed of three multilayered perceptron‐type neural networks. The error signal to be used in the learning by error backpropagation is acquired by motion. The learning process is as follows. For each generation of sound, four actions are performed: auditory localization, visual localization, search, and step centering. The learning is based on the acquired information. In order to verify the validity of the proposed model, a computer simulation experiment and an experiment using a robot and a real sound source were performed. The experiments demonstrated that auditory and visual localization are performed simultaneously. Generalization of the unlearned direction and efficient operation are also demonstrated. The localization error is examined as a function of the distance between the source and the robot. The mapping by the neural network after learning is also investigated. © 2002 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 85(4): 58–67, 2002; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/ecjb.10046