A vision-based human-robot collaborative system for digital twin

Abstract

Flexible and safe human-robot collaboration depends on accurately capturing the three-dimensional motion of humans and robots in the field of smart manufacturing. In this paper, a novel approach to developing a human-robot collaborative assembly system is proposed and applied to the field of digital twins. Within the context, a deep learning-based model is explored to develop a depth camera-based human recognition system for accurate prediction of key points for human skeletons model and high-precision human localisation in a human-robot collaborative setting. After the functional mapping of robot calibration, a collision warning module leverages coordinates of key human-robot points to facilitate efficient and safe human-robot collaborative assembly.