Interactive system for sharing objects information by gesture and voice recognition between human and robot with facial expression

Abstract

This paper proposes a system that the robot with facial expression of the system shares information of objects by using Multimodal Interaction between human and robot. Multi-modal Interaction works by combining indicating and one word utterance which are the results of gesture recognition and voice recognition. According to the situation of the interaction, the robot expresses appropriate facial feelings. Proposed system obtains an object's place as three dimensional coordinates by indicating motion, and also obtains the object's name, color, and shape by one word utterance. Proposed system manages the objects by storing, deleting, and searching information of the objects. In this study, examinations showed the proposed system is effective.