Multi‐Robot Collaborative Complex Indoor Scene Segmentation via Multiplex Interactive Learning

Abstract

ABSTRACT Indoor scene semantic segmentation is essential for enabling robots to understand and interact with their environments effectively. However, numerous challenges remain unresolved, particularly in single‐robot systems, which often struggle with the complexity and variability of indoor scenes. To address these limitations, we introduce a novel multi‐robot collaborative framework based on multiplex interactive learning (MPIL) in which each robot specialises in a distinct visual task within a unified multitask architecture. During training, the framework employs task‐specific decoders and cross‐task feature sharing to enhance collaborative optimisation. At inference time, robots operate independently with optimised models, enabling scalable, asynchronous and efficient deployment in real‐world scenarios. Specifically, MPIL employs specially designed modules that integrate RGB and depth data, refine feature representations and facilitate the simultaneous execution of multiple tasks, such as instance segmentation, scene classification and semantic segmentation. By leveraging these modules, distinct agents within multi‐robot systems can effectively handle specialised tasks, thereby enhancing the overall system's flexibility and adaptability. This collaborative effort maximises the strengths of each robot, resulting in a more comprehensive understanding of environments. Extensive experiments on two public benchmark datasets demonstrate MPIL's competitive performance compared to state‐of‐the‐art approaches, highlighting the effectiveness and robustness of our multi‐robot system in complex indoor environments.