NeuTRL: Neural Trust-Guided Reinforcement Learning for Human-Robot Collaboration

摘要

Reinforcement Learning from Human Feedback (RLHF) enables robots to learn cooperative strategies aligned with human expectations by incorporating feedback into the learning process. However, existing RLHF methods rely on explicit query-based feedback, which is limited for complex, long-horizon tasks due to feedback sparsity and delay. Additionally, active feedback collection disrupts task execution and increases human cognitive load. To address these challenges, we propose Neural Trust-guided Reinforcement Learning (NeuTRL), a novel human-in-the-loop framework that utilizes human trust, derived from electroencephalography (EEG) signals, as implicit feedback for reward shaping and policy optimization. NeuTRL incorporates a deep neural network to map EEG to trust states, enabling high-frequency evaluations of the robot's behavior. Additionally, we propose a Long Short-Term Memory (LSTM)-based reward model to capture the temporal dynamics of trust evolution and generate dense rewards. NeuTRL enables robots to learn collaborative policies using dense EEG-based trust feedback, addressing the feedback sparsity challenge while maintaining uninterrupted interaction. Experimental results demonstrate that NeuTRL achieves a 74.5% improvement in average collaborative task performance compared to baselines without human feedback and a 26.9% improvement compared to query-based RLHF. Qualitative analysis further reveals that NeuTRL significantly enhances cooperative adaptability and coordination. These results highlight NeuTRL's potential to advance trust-aware, adaptive human-robot collaboration in dynamic environments.