Leveraging multimodal data for intuitive robot control towards human-robot collaborative assembly

Abstract

In human-robot collaborative assembly, robots help human operators complete complex assembly tasks. However, controlling these robots is not intuitive given the constraints posed by the close proximity of the operator. In response to this need, a novel approach using multimodal data is developed for human-centred robot control in human-robot collaborative assembly. An interface design is developed to fuse multimodal communication channels for robust and adaptive robot control and then multimodal data are defined as control input for assembly task execution. This control scheme offers human operators symbiotic multimodal tools for proactive HRC with enhanced flexibility and adaptability.