Multimodal human-robot collaboration: advancements and future directions

Abstract

Human-robot collaboration (HRC) envisioned for future factories has been actively explored to facilitate higher overall productivity. The wide applications of HRC in multiple fields, such as manufacturing and production, have seen a series of milestones. In recent years, a shift towards intuitive and natural collaboration between humans and robots has been investigated and discussed for symbiotic scenarios and complex tasks. For this purpose, advancements of multimodality in HRC enable multimodal human-robot interactions and collaboration by utilising different communication channels such as auditory, vision, gestures, haptics, and even brain signals. In addition, understanding human behaviours in terms of intent and motion can be beneficial in achieving mutual human-robot assistance. Within an HRC setting, the digital twin of such a physical collaborative workcell offers a promising tool to implement sim-to-real transformation and on-demand support for real practice. Within the context, this study provides an overview of the past and current status of multimodal HRC and its applications and highlights future research directions. [Submitted 8 May 2024; Accepted 8 July 2025]