Supervised Control for Robot-Assisted Surgery Using Augmented Reality

Abstract

In this paper, we propose an AR-based robotic system that can plan and execute a trajectory based on a 3D medical model and allow the surgeon to supervise the execution of surgical process. In order to achieve image-guided surgery, a hand-eye calibration procedure is developed by using AprilTag to obtain the transformation between the workspace coordinate and the robot coordinate, and perform the image navigation task of the surgical robot to complete a drilling task. A procedure of robot supervised control is proposed to assign or modify the robot trajectory based on AR visualization and 3D model. A specified AR marker is used to project virtual objects in the AR glasses. We developed a robot registration algorithm to match the AR virtual coordinate system and the workspace coordinate system, and convert the planned trajectory into robot trajectory. Experimental results on the lab-built robotic system show that a user can adjust the position and orientation of the insertion point on a bone model, and transmit the trajectory information to the robot for execution. The proposed visualization-based robot navigation method has the potential to enhance the safety of surgical operation.