Vision-Action Semantic Associative Learning Based on Spiking Neural Networks for Cognitive Robot

摘要

Establishing cognitive environments help cognitive robots understand human actions, languages and observed objects. In this paper, a cognitive robotic model based on a novel spiking bidirectional associative memory (BAM) method is presented to establish a cognitive environment. The spiking BAM network uses a supervised spike-based learning rule to learn the relationship between the semantic information of vision and action, where a new coding scheme is proposed to encode and decode vision and action semantic information. Vision semantic information is obtained by the deeply salient shape detection (DSSD) method. Action semantic information is obtained by a spiking neural network (SNN) with the proposed coding method. Experimental results demonstrate that the spiking BAM achieves a good convergence ability and relatively high recall accuracy. Specially, for the cognitive development of the robot, this system can be used to improve the robot’s ability to infer human intentions in its natural interactions with humans.