Autonomous Robotic Ultrasound Approach for Fetoscope Tracking by Fusing Optical and 2D Ultrasound Data

Abstract

2D ultrasound (US) guidance is an essential tool in fetoscopic laser photocoagulation (FLP) to treat twin-to-twin transfusion syndrome (TTTS). During the procedure, the sonographer and endoscopic surgeon manage different image modalities each with its own field of view. Tacit collaboration is needed between them to visualize the right information and ensure smooth operation. Robotic approaches could simplify this interaction but would require robust localization tools to cope with the complex fetoscopic motion patterns. This letter proposes a method for robotic ultrasound (rUS) fetoscope tracking, fusing optical tracking system (OTS) and 2D US imaging. The Kalman filter is utilized to guarantee robust online registration and enhance fetoscope tracking. Real-time detection of the fetoscope tip is achieved using the You Only Look Once (YOLO v7) algorithm. Additionally, a US image-based searching strategy is proposed for situations where the optical camera is obstructed. Hybrid position-force control is employed to manipulate the US probe safely against the pregnant abdomen. Validation on a silicone phantom demonstrates accurate tracking results with a mean error below 2.59 mm in most experiments and tip visibility exceeding 90%. This system has the potential to significantly reduce surgeon workload and training requirements associated with FLP surgery.