Multi-Task Deep Learning-Based Human Intention Prediction for Human-Robot Collaborative Assembly

摘要

Construction robots have great potential to serve as assistants to relieve construction workers from repetitive and physically demanding tasks. It is essential for robots to understand and predict human intention in order to adapt their motion to ensure smooth human-robot collaboration. This study proposes a long short-term memory model-based multi-task learning framework to simultaneously predict multi-level human intention in assembly tasks, including high-level actions and objects, and low-level body movements, from observed body movements and associated assembly components extracted from videos. The proposed models were trained and tested using 54 videos collected with nine participants performing six assembly tasks, achieving an accuracy of 82% and 98% in action and object prediction, respectively, and an average displacement error of 8.71 pixels in pose prediction. The incorporation of work context significantly improves the accuracy of object prediction by 11.36%, with the performance of other two tasks increasing slightly.