Robot Adaptation Under Operator Cognitive Fatigue Using Reinforcement Learning

摘要

This paper presents the development and validation of a robot adaptation model to support human operators in Human-Robot Collaborative tasks when they are cognitively fatigued. A human-centered robot adaptation method for providing appropriate assistance to the operator is developed with a dual objective of task performance optimization and aiding human cognitive fatigue recovery. The problem is formulated as a Markov Decision Process (MDP) and solved using Q-learning. The implementation issues resulting from modeling the MDP and performing Q-learning for cognitive fatigue recovery, methods to mitigate those issues, and implications on the resulting optimal policies are discussed. The proposed approach is evaluated through a user study of sixteen participants performing a robotic surface polishing task under cognitive fatigue conditions. The MDP model is validated using subjective metrics (fatigue perception surveys) and objective metrics (Heart-Rate Variability (HRV), accuracy in trajectory tracking, and time efficiency of the task). Fatigue perceptions, accuracy, and time efficiency improved during the user-specific optimal adaptation policies. HRV analysis of time-domain features shows an overall improvement in fatigue conditions during the optimal adaptation policies. The results from this approach indicate that such human-centered robot adaptation can lead to efficient human-robot collaborations with robust interactions between robots and humans.