Bimodal Tactile Tomography with Bayesian Sequential Palpation for Intracavitary Microstructure Profiling and Segmentation

Abstract

Robotic palpation for in situ tissue biomechanical evaluation is crucial for disease diagnosis, especially in luminal organs. However, acquiring real-time information about the tissue’s interaction state and physical characteristics remains a substantial challenge. While commercial surgical robotic systems have integrated tactile feedback, the absence of tactile intelligence and autonomous decision-making limits the surgeon’s ability to comprehensively assess tissue mechanics, hindering the efficient detection of abnormalities. Endoscopic optical coherence tomography has emerged as a promising technology for real-time, 3-dimensional visualization of tissue microstructures and subtle lesions in luminal organs. However, it does not address the tactile sensing required for lesion profiling and boundary identification. To bridge this gap, we developed a new robotic bimodal palpation technique that uses a previously proposed optical-coherence-tomography-based tactile sensor, ElastoSight. This technique utilizes circumferential and sliding B-scan modes along with Bayesian optimization for precise lesion center and boundary detection. In tumor phantom models, our technique achieves tumor localization within 30 iterations, with high F 1 scores over 0.976 and a centroid error below 0.032 mm. Using the sliding B-scan mode on the phantom surface, we achieve accurate segmentation of hard tissue inclusions from the surrounding soft tissue, with a precision rate of 0.983 and an area error below 0.25 mm 2 . These results show that the proposed technique effectively tackles real-time lesion localization and segmentation challenges, demonstrating strong performance in simulations and experiments. Our technique can potentially enhance tissue abnormality detection during robot-assisted minimally invasive surgery, improving the precision and efficiency of procedures like tumor removal.