A Multimodal Fusion Scheme for the Safe Puncture of Robotic-Assisted Flexible Needle

Abstract

Accurate tracking of flexible needles is crucial for the success of robotic-assisted puncture procedures in minimally invasive surgery. This paper presents a novel sensor fusion framework that integrates permanent magnet navigation (PMN) and X-ray imaging to enable robust and precise needle tip tracking in robotic-assisted flexible needle (RFN) systems. A magnetic flexible needle is designed by embedding a permanent magnet in a nitinol spring body, allowing wireless localization while supporting the continuous rotation required by duty-cycling steering. To address challenges such as ferromagnetic interference in PMN and occlusions in X-ray, we develop a decentralized extended Kalman filter-based fusion algorithm that dynamically adjusts sensor weights based on cross-covariance, ensuring reliable estimation even with partial sensor failure. Experimental validation on a self-developed RFN platform demonstrates that the proposed framework outperforms single-modality systems under both magnetic interference and visual occlusion conditions, showing improved robustness and tracking continuity. This approach offers a promising solution for image-guided interventions in complex anatomical environments, with the potential to reduce the number of puncture attempts and minimize patient discomfort.