Cross-View Human Intention Recognition for Human-Robot Collaboration

摘要

Benefiting from the promise of sixth generation (6G) wireless networks, multimodal machine learning based on exploiting complementarity among video, audio, and haptic signals, becomes a key enabler for human intention recognition, which is critical to realize effective human-robot collaboration in Industry 4.0 scenarios. However, as multimodal human intention recognition is limited by expensive equipment and a demanding environment, it is hard to strike an efficient trade-off between inference accuracy and system overhead. Naturally, how to induce more intention semantics from readily available videos emerges as a fundamental issue for human intention recognition. In this article, we use cross-view human intention recognition to solve the above issue and demonstrate the effectiveness of our method with well-designed evaluation metrics. Specifically, we first compensate for the scarcity of intention semantics in the body view by adding a face view. Second, we deploy the cross-view generative model to capture intention semantics induced by the mutual generation of two views. Finally, in the human-robot collaboration experiments, our method gets closer to human performance regarding response time and inference accuracy.