Enhancing Telecooperation Through Haptic Twin for Internet of Robotic Things: Implementation and Challenges

Abstract

The Internet of Robotic Things (IoRT) serves as a bridge for the progress of upcoming immersive interaction technologies like XR, holographic communications, and the metaverse, enabling smooth interaction between the tangible and virtual domains. It enriches the capabilities of sensors and controllers in both domains, enabling precise mapping and control. Moreover, immersive interaction technologies enhance IoRT by offering superior visualization and user experiences. Here, we propose a haptic-twin-enhanced telecooperation system (HTTS) delving into tactile interaction technology for IoRT, unveiling a telecooperation-enhanced haptic twin technology. Our goal is to construct haptic gloves devices and its haptic twin entity that are centered on virtual collaborations (i.e., virtual handshakes and collaborative tasks), deploying an experimental system for the acquisition, processing, transmission, reconstruction and interaction of multisensory data over wireless networks. We simulate and demonstrate the perception of touch, friction, and virtual gravity using the proposed haptic twin-based experimental testbed. Compared to commercial product, namely, HTC Vive Controller, our approach achieves a more reliable and precise construction improvement of haptic twin entity that is based on our implemented haptic gloves in terms of simulated human hand posture, ensuring consistent haptic feedback in virtual control and collaboration. This offers a more user-friendly and immersive experience with adaptive haptic effects, paving the way for novel telecooperation applications in the future IoRT.