Investigating Inverse Reinforcement Learning during Rapid Aiming Movement in Extended Reality and Human-Robot Interaction

Abstract

Rapid aiming movement involves quick, accurate, pre-programmed motions used in the context of human-computer and human-robot interaction. It incorporates target forecasting to minimize the duration of tasks requiring rapid aiming. Applications include predicting target icons in UI design, driver intent in automotive technology, and human intent during human-robot collaboration. Conventional approaches often fail to capture human preferences accurately, leading to low prediction accuracy. This work explores an Inverse Reinforcement Learning (IRL)-based system for forecasting human hand movements and intended targets during rapid aiming. Sampling-based Maximum Entropy IRL (SMEIRL) with a sampler and Maximum Entropy Deep IRL (MEDIRL) algorithms were evaluated for prediction accuracy. The proposed sampler efficiently generates sample trajectories for rapid aiming tasks. User studies were conducted to assess target prediction during two tasks involving rapid aiming movement: (1) Pointing in Virtual Reality (VR) and Mixed Reality (MR), and (2) Human-robot handovers. A multimodal target prediction algorithm was analyzed for swift and accurate anticipation of the intended target, considering both hand and eye gaze. Results demonstrate that the proposed approach achieves a prediction accuracy of 98% in MR and 96% in VR for the pointing task using SMEIRL. During human-robot handover task, prediction accuracy using MEDIRL reached 99.9% when less than 20% and 40% of the task was left using only hand motion or both hand and eye gaze, respectively, surpassing state-of-the-art methods using Path Integral-IRL (PI-IRL), Recurrent Neural Network-Inverse Kinematics-Modified Kalman Filtering (RNNIK-MKF), Bayesian Predictor for Human Motion Trajectory (BP-HMT), and Classical kinematics of motion ( \(CM_{k=5}\) ).