ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools

Abstract

Real-time tool segmentation from endoscopic
\nvideos is an essential part of many computer-assisted robotic
\nsurgical systems and of critical importance in robotic surgical
\ndata science. We propose two novel deep learning architectures
\nfor automatic segmentation of non-rigid surgical instruments.
\nBoth methods take advantage of automated deep-learningbased
\nmulti-scale feature extraction while trying to maintain
\nan accurate segmentation quality at all resolutions. The two
\nproposed methods encode the multi-scale constraint inside the
\nnetwork architecture. The first proposed architecture enforces it
\nby cascaded aggregation of predictions and the second proposed
\nnetwork does it by means of a holistically-nested architecture
\nwhere the loss at each scale is taken into account for the
\noptimization process. As the proposed methods are for realtime
\nsemantic labeling, both present a reduced number of
\nparameters. We propose the use of parametric rectified linear
\nunits for semantic labeling in these small architectures to
\nincrease the regularization of the network while maintaining
\nthe segmentation accuracy. We compare the proposed architectures
\nagainst state-of-the-art fully convolutional networks.
\nWe validate our methods using existing benchmark datasets,
\nincluding ex vivo cases with phantom tissue and different robotic
\nsurgical instruments present in the scene. Our results show
\na statistically significant improved Dice Similarity Coefficient
\nover previous instrument segmentation methods. We analyze
\nour design choices and discuss the key drivers for improving
\naccuracy.