Robotic Scene Understanding by Using a Dictionary

Abstract

Scene understanding is a fundamental task for intelligent robots, especially in human-robot interaction. It is challenging due to the complexity of the human environment. In this paper, the proposed method integrates semantic analysis with object detection that enables robots to perceive scenes and deeply understands working environments. The model can extract deterministic entities of objects by analyzing their dictionary definitions. Therefore, robots can understand a scene at the object-level. These deterministic entities include the category, function, property, and composition of objects, and they can be used to generate feedback on how much robots can understand a scene by describing it in natural language. The feedback on how well robots understand the working space is an essential aspect to eliminate confusion during human-robot interactions. The experiment part of this paper discussed the applicability of the proposed method on robots.