An offline reinforcement learning-based framework for proactive robot assistance in assembly task

Abstract

Proactive robot assistance plays a critical role in human–robot collaborative assembly (HRCA), enhancing operational efficiency, product quality and workers’ ergonomics. The shift toward mass personalisation in industries brings significant challenges to the collaborative robot that must quickly adapt to product changes for proactive assistance. State-of-the-art knowledge-based task planners in HRCA struggle to quickly update their knowledge to adapt to the change of new products. Different from conventional methods, this work studies learning proactive assistance by leveraging reinforcement learning (RL) to train a policy, ready to be used for robot proactive assistance planning in HRCA. To address the limitations therein, we propose an offline RL framework where a policy for proactive assistance is trained using the dataset visually extracted from human demonstrations. In particular, an RL algorithm with a conservative Q-value is utilised to train a planning policy in an actor-critic framework with carefully designed state space and reward function. The experimental results show that with only a few demonstrations performed by workers as input, the algorithm can train a policy for proactive assistance in HRCA. The assistance task provided by the robot can fully meet the task requirement and improve human assembly preference satisfaction by 47.06% compared to a static strategy. • An offline reinforcement learning-based framework for quickly enabling a collaborative robot for proactive assistance in assembly tasks. • A symbolic-level task planning method requires only human assembly demonstrations as input and offers high learning efficiency. • A real human-robot collaborative assembly experiment shows that the learned assistance can fully meet task requirements and improve the human operation preference.