NeuroFusion-Trans: A Novel Transformer-Based EEG-EMG Fusion Model for Assistive Robotics

摘要

User intent recognition from multimodal neurophysiological signals, particularly electroencephalography (EEG) and electromyography (EMG), is critical for enhancing human-machine interaction in assistive robotics. Recent advances in neurophysiological signal processing have enabled enhanced user intent recognition for assistive robotics and human-machine interfaces. However, achieving high accuracy and real-time adaptability in electromyography (EMG) and electroencephalography (EEG)-based gesture recognition remains challenging due to temporal misalignment, weak cross-modality fusion, and lack of adaptive learning. This paper proposes NeuroFusion-Trans, a novel transformer-based framework that improves EEG-EMG gesture recognition by improving temporal resolution, using cross-modality attention, and integrating adaptive online learning. Temporal resolution enhancement ensures dynamic EEG-EMG synchronization for improved signal alignment. The cross-modality attention mechanism captures interdependencies between EEG and EMG signals, leading to more accurate intent classification. Adaptive online learning enables real-time personalization by dynamically adjusting to user-specific variations. The model is evaluated on two publicly available EEG-EMG upper-limb gesture datasets: Dataset 1 (5,296 for training, 1,324 for validation) and Dataset 2 (5,276 for training, 1,304 for validation). NeuroFusion-Trans achieves state-of-the-art performance, with an accuracy of 97% and 96% and Cohen’s Kappa of 0.97 and 0.95 after online adaptation, significantly outperforming baseline models such as CNN-LSTM, GRU, and LSTMNet. Ablation studies reveal that removing the cross-modality attention mechanism reduces accuracy by 6.1%, underscoring its importance in leveraging EEG-EMG dependencies. Turning off synchronization leads to a 6.7% performance drop, demonstrating the necessity of real-time learning for robust intent recognition. Furthermore, NeuroFusion-Trans enhances EEG-EMG synchronization, achieving synchronization scores of 0.7471 on Dataset 1 and 0.7687 on Dataset 2, confirming its effectiveness in temporal alignment. The proposed approach demonstrates robust generalization, improved responsiveness, and potential applications in real-time assistive robotics.