3D Semantic Mapping in Greenhouses for Agricultural Mobile Robots with Robust Object Recognition Using Robots' Trajectory

Abstract

This paper describes a method of building a semantic map of a greenhouse for a robot path planning. Existing mapping methods only consider whether there are obstacles in a certain region. They are not sufficient for path planning in greenhouses where traversable regions are often covered by branches and leaves which are also recognized as obstacles. We propose a mapping method which generates a map with semantic information on the types of obstacles. By integrating RGB-D based visual SLAM (Simultaneous Localization And Mapping) and semantic segmentation by a deep neural network, we obtain a 3D map with semantic labels. In order to deal with the uncertainty of observations, we introduce a Bayesian label updating strategy which effectively utilizes the fact that the robot traverses a region. Through evaluations, we confirmed that the proposed method can perform a more accurate semantic labeling than the one only using SegNet.