Loop Closure Detection and SLAM in Vineyards with Deep Semantic Cues

Abstract

Automation of vineyards cultivation necessitates for mobile robots to retain accurate localization system. The paper introduces a stereo vision-based Graph-Simultaneous Localization and Mapping (Graph-SLAM) pipeline custom-tailored to the specificities of vineyard fields. Graph-SLAM is reinforced with a Loop Closure Detection (LCD) based on semantic segmentation of the vine trees. The Mask R-CNN network is applied to segment the trunk regions of images, on which unique visual features are extracted. These features are used to populate the bag of visual words (BoVW s) retained on the formulated graph. A nearest neighbor search is applied to each query trunk-image to associate each unique feature descriptor with the corresponding node in the graph using a voting procedure. We apply a probabilistic method to select the most suitable loop closing pair and, upon an LCD appearance, the 3D points of the trunks are employed to estimate the loop closure constraint to the graph. The traceable features on trunk segments drastically reduce the number of retained BoVWs, which in turn expedites significantly the loop closure and graph optimization, rendering our method suitable for large scale mapping in vineyards. The pipeline has been evaluated on several data sequences gathered from real vineyards, in different seasons, when the appearance of vine trees vary significantly, and exhibited robust mapping in long distances.