Aggregating Long-Term Context for Learning Laparoscopic and\n Robot-Assisted Surgical Workflows

Abstract

Analyzing surgical workflow is crucial for surgical assistance robots to\nunderstand surgeries. With the understanding of the complete surgical workflow,\nthe robots are able to assist the surgeons in intra-operative events, such as\nby giving a warning when the surgeon is entering specific keys or high-risk\nphases. Deep learning techniques have recently been widely applied to\nrecognizing surgical workflows. Many of the existing temporal neural network\nmodels are limited in their capability to handle long-term dependencies in the\ndata, instead, relying upon the strong performance of the underlying per-frame\nvisual models. We propose a new temporal network structure that leverages\ntask-specific network representation to collect long-term sufficient statistics\nthat are propagated by a sufficient statistics model (SSM). We implement our\napproach within an LSTM backbone for the task of surgical phase recognition and\nexplore several choices for propagated statistics. We demonstrate superior\nresults over existing and novel state-of-the-art segmentation techniques on two\nlaparoscopic cholecystectomy datasets: the publicly available Cholec80 dataset\nand MGH100, a novel dataset with more challenging and clinically meaningful\nsegment labels.\n