Multi-modal communication system for mobile robot

Abstract

This paper presents a cloud-based multi-modal human-robot interaction architecture. The architecture consists of parts running on the edge (mobile) platform and cluster services. Software on the mobile platform is represented primarily by a Robot Operating System, an MQTT broker and the client side of the SpeechCloud architecture, which provides the communication with the computing cluster. Its detailed description is a part of the paper. The main contribution of the paper is a design of an architecture based on standard frameworks, protocols, and validated JSON messages. The paper also focuses on the description of a tested mobile platform equipped with an NVIDIA Jetson family computer and information about tasks that the robot will be able to perform using the proposed architecture.