Demonstration of an Open-Source ROS 2 Framework and Simulator for Situated Interactive Social Robots

Abstract

We introduce an open-source ROS 2 architecture for situated social robots, along with a simulator that allows mixed-reality development and interactions. The architecture is a hybrid symbolic/subsymbolic system that integrates explicit ontology semantics for perception, reasoning, and execution, with LLMs. It features multimodal social perception by leveraging the open source ROS4HRI framework; LLMs (both edge- and cloud-based) to facilitate natural language interaction between the user and system; KnowledgeCore, an open-source knowledge base, to reason about facts in the world; and an intent-based controller to supervise the execution of parallel/sequential tasks and skills. We demonstrate our system architecture with a social robot running the mixed-reality system.