Surgical Robotics Environment in NVIDIA Isaac Sim for Robot-Assisted Suturing

Abstract

Virtual simulations have served as important tools to design and test robotic frameworks in a variety of domains. Increasingly, they have been used to train and experiment on surgical robotics applications. Having seen the success of an open simulation environment for robot-assisted surgical suturing in the Asynchronous Multi-Body Framework (AMBF), we sought to reproduce the surgical suturing environment from AMBF in NVIDIA Isaac Sim, a new simulator with impressive visual fidelity and GPU-accelerated computation capabilities. We present a virtual scene for robotic suturing comprising two robotic manipulators, an endoscope, two surgical phantoms, and a needle attached to a thread. The thread is modeled using three different techniques. This work also provides interactive manipulation of the virtual scene via haptic input devices using the Robot Operating System interface. We measured performance speeds, ranging between 13 to 60 frames per second depending on the task, and approximately 0.5 seconds of delay between a haptic input device and the simulator. In this environment, we hope to provide a descriptive overview of the capabilities and limitations of NVIDIA Isaac Sim for surgical simulation applications.