World's first live AI‐assisted augmented reality (AR) robotic colon surgery with instrument deocclusion: Navigating the future of surgery

Abstract

Artificial intelligence (AI) and augmented reality (AR) are transforming the landscape of minimally invasive surgery. We present the world's first live integration of an AI-assisted AR overlay with real-time instrument deocclusion in robotic colorectal surgery. A real surgical case of robotic right colectomy was performed using a da Vinci Xi platform enhanced by an AI-driven AR system. The technology generated three-dimensional anatomical overlays while simultaneously applying instrument deocclusion algorithms to maintain continuous visualization of critical anatomy (Video 1). The procedure was successfully completed without conversion, demonstrating the technical feasibility of combining AI-based overlays and deocclusion in a complex intraoperative environment. The system allowed stable AR guidance despite dynamic camera movements and instrument interference. From the operating surgeon's perspective, the AR overlay was useful, non-disruptive to workflow, and enhanced anatomical orientation. These impressions are preliminary and require validation in prospective studies. This proof-of-concept highlights the feasibility of AI-assisted AR with instrument deocclusion in robotic colorectal surgery and represents an important step toward digital surgery of the future. While no formal outcomes or structured user assessments were collected in this initial demonstration, future studies will focus on systematic evaluation of usability, surgeon feedback, and clinical performance metrics. Thalia Petropoulou: Conceptualization; investigation; writing – original draft; methodology; validation; visualization; writing – review and editing; formal analysis. Jente Simoens: Methodology; validation; software. Andreas Polydorou: Investigation; validation; supervision. Pieter De Backer: Conceptualization; methodology; validation; software; formal analysis; data curation. Alex Mottrie: Methodology; validation; software; supervision. The use of the AI-assisted technology was observational and did not alter the standard treatment pathway or clinical decision-making. The patient provided written informed consent for the use of anonymized clinical data, images, and videos for scientific publication and educational purposes. No identifiable patient information is presented in this manuscript. The publication of this article in OA mode was financially supported by HEAL-Link. No funding was used. No conflict of interest between the authors. This study was conducted in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments. The data that support the findings of this study are available from the corresponding author upon reasonable request.