Semantic knowledge based reasoning framework for human robot collaboration

Abstract

Human robot collaborative assemblies, where humans and robots work together, are becoming the new frontier in industrial robotics. However, the aspect of humans to easily and intuitively interact/collaborate with a robot, especially for teaching new tasks, is still open. In this work, we present a semantic knowledge-based-reasoning framework that first learns to recognize human activities using perception data (action recognition and object tracking) and semantically links them to an assembly process. Thus deriving the intention of the human interaction in the teaching process. The framework then extracts relevant parameters for the robot action execution using an interactive skill-based programming approach. To resolve ambiguities during the teaching process, the reasoning framework initiates an interaction at an (semantically) abstract level with the user by exploiting previous knowledge and the current environmental setup. The reasoning framework is demonstrated in two different application scenarios involving human-robot collaborative teaching and a user guidance system respectively.