A Digital Twin-Driven Immersive Teleoperation Framework for Robot-Assisted Microsurgery

Abstract

This paper presents a novel digital twin (DT)-driven framework for immersive teleoperation in the domain of robot-assisted microsurgery (RAMS). The proposed method leverages the power of DT with mixed reality (MR) technology to create an interactive, immersive teleoperation environment for surgeons to conduct RAMS with higher precision, improved safety, and higher efficiency. More specifically, the MR device can provide operators with the 3D visualization of a digital microsurgical robot mimicking the motions of the physical one as well as the 2D real-time microscopic images during microsurgical operation. We evaluated the proposed framework through user studies based on a Trajectory Following task and conducted comparisons between scenarios with and without using the proposed framework for RAMS. The NASA-TLX questionnaire, along with additional evaluation metrics such as total trajectory, time cost, mean velocity, and a predefined collision metric, were used to analyze the user studies. Results indicated that the proposed DT-driven immersive teleoperation framework could enhance the precision, safety, and efficiency of teleoperation, and provide a satisfactory user experience to operators during microsurgical operation.