Deep-learning-based outcome prediction from endoscopic videos for robot-assisted radical prostatectomy

Abstract

Prostate cancer is a common type of cancer in men that can be treated by removing the prostate through Robot- Assisted Radical Prostatectomy (RARP). A key outcome is the preservation of the patient’s Early Urinary Continence (EUC). Predicting EUC and other outcomes could help clinicians plan the individual patient’s aftercare more effectively. Additionally, one could automatically analyze large numbers of recorded RARP procedures for quality assurance. This work presents the development of a fully automated Deep Learning (DL) pipeline designed to predict EUC recovery after RARP based solely on endoscopic video data. Together with preand post-processing algorithms, the pipeline utilizes DL technologies, including a feature extractor, a transformer, and dense model. An essential component of this pipeline is the localization and analysis of the anatomical structure of the external urinary sphincter muscle, which is correlated with the success of EUC recovery. Our pipeline achieves results comparable to existing methods but without the need for manual pre-processing steps or additional data beyond the raw video recordings. The localization component of the pipeline can find 80% of the sphincter sequences in the test data. The component for the EUC prediction achieves an accuracy of almost 70% on the annotated test data, which also corresponds to the accuracy of the entire pipeline. We conducted a comparative user study aiming to quantify the assumption that the representation of the anatomical structure of the sphincter muscle from endoscopic video data is sufficient for human experts to predict EUC. Hence, this work illustrates the potential of DL to improve medical video analysis, provides insight into the performance of such an automated system, and compares this pipeline to the performance of clinical experts on similar data.