A Mixed-Reality-Augmented Deep Reinforcement Learning Approach for Multi-Robot Safe Motion Generation in Human–Robot Collaborative Manufacturing Cells

摘要

Augmenting capabilities of human operators with multi-robot cells offers substantial advantages for increasing productivity in manufacturing applications. This synergy effectively combines the strengths of both robots and humans, maximizing operational efficiency and leveraging human capabilities. However, achieving these benefits requires real-time, reactive coordination of multi-robot motion generation in response to human motion. Current approaches face significant challenges, particularly in dealing with uncertainties in human motions. To address these issues, this paper introduces the Deep Reinforcement Learning (DRL) approach for end-to-end safe motion generation in human multi-robot collaborative workspaces. First, the DRL approach is augmented by adopting mixed-reality (MR) features to facilitate efficient state perception and representation of tasks, humans, robots, and scenes for enabling effective learning motion generation policy. Moreover, to better promote high-dimensional action generation of the multi-robot systems involving human, an advanced DRL approach is developed. The approach leverages memory-enhanced representation learning, intrinsic reward-guided exploration, and action space pruning to better address the motion generation challenges. Empirical testing demonstrates the effectiveness of the proposed system, with experiments showing high success rates across tasks with varying team sizes and difficulty levels, thereby demonstrating applicability in human-robot collaborative manufacturing tasks.