Towards Immersive Bilateral Teleoperation Using Encountered-Type Haptic Interface

Abstract

Encountered-type haptics (ETH) is an emerging research field that enables unencumbered physical haptic interaction in virtual reality (VR). In this paper, we propose Encountered-type haptics (ETH) as an interaction medium for immersive remote teleoperation, facilitating intuitive bare-hand interaction with visuo-haptic feedback. Our system allows a human operator to control a remote robot immersively through the visual rendering of the VR environment, while interacting with a haptic robot at the user site. The ETH feedback rendering and the teleoperation at the remote site are both implemented using a 7 degrees-of-freedom (DoFs) Franka Emika Panda robot under Cartesian-impedance control. The Cartesian goal poses for each robot are determined based on the pose of the operator's proxy hand in the VR environment and the operator's interaction intention, estimated through hand gestures and gaze direction. The impedances of both robots are updated at runtime to provide the operator with bilateral haptic interaction forces. Our system was evaluated through a user study involving a door-opening task under three teleoperation conditions: (1) without haptic feedback, (2) ETH feedback with constant impedance, and (3) ETH feedback with variable impedance. The results highlight the advantages of using ETH, including the ability to regulate forces at both the user and robot sites. ETH conditions demonstrate lower peak forces and reduced force jittering at the remote site. Furthermore, the variable impedance condition within ETH shows improved task execution times and reduced exerted force. This paper demonstrates that ETH is an effective medium for immersive bare-hand bilateral teleoperation.