Multi-mode Natural Language Processing for human-robot interaction

Abstract

As more and more open knowledge resources become available, it is interesting to explore opportunities of enhancing autonomous agents’ capacities by utilizing the knowledge in these resources, instead of hand-coding knowledge for agents. A major challenge towards this goal lies in the translation o f the open knowledge organized in multiple modes, unstructured or semi-structured, into the internal representations of agents. In this paper we present a set of multi-mode NLP techniques to formalize the open knowledge for autonomous agents. Two case studies are reported in which our robot, equipped with the multi-mode NLP techniques, succeeded in acquiring knowledge from the microwave oven manual and from the open knowledge database, OMICS, and solving problems that could not be solved before the robot acquired the knowledge. Experiments for evaluating the performance of our approach show that our approach is promising.