3D Semantic Mapping based on RGB-D Camera and LiDAR Sensor in Beach Environment

Abstract

The increasing amount of sea garbage proving to be a huge problem not only for marine life but also causing a tremendous number of troubles to humans living along the coastlines. Especially Japan, that is surrounded by the ocean which has a long coastline that spans thousands of kilometers, and this provides ample opportunities for marine debris to accumulate. This paper proposes a 3D mapping system utilizing the semantic information from the RGBD sensor extracted from a specifically trained state-of-art deep learning neural network to merge with the spatial information from the LiDAR sensor using Max Fusion Technique. This paper also aims to contribute to the implementation of deep learning and autonomous driving in beach cleaning-related robots and spark interest and raise awareness of the public in beach clean-up activities. It is also shown that the proposed system can achieve a mean accuracy of 98% approximately with the uniquely trained deep learning model.