Semantic Slam: A New Paradigm for Object Recognition and Scene Reconstruction

Abstract

Simultaneous localisation and mapping (SLAM) is a technique studied in computer vision and robotics that, given measurements obtained from one or more sensors, allows incremental building of a map of the environment and simultaneous estimation of the position and orientation of the very same sensor used to acquire the input data.
\nVisual SLAM systems typically allow the generation of accurate reconstructions of the explored environment but, until very recently, did not provide high level informations on the contents of the reconstructed scenes, useful to foster high level reasoning by subsequent algorithms.
\n
\nIn this thesis we focus on the topic of Semantic SLAM, proposing techniques to obtain semantically accurate reconstructions of the explored environment by combining efficient SLAM systems with state-of-the-art semantic image segmentation algorithms.
\nWe show how, by relying on such semantic reconstructions, the accuracy of the localisation phase of a SLAM pipeline can improve, by accounting for the presence of semantic informations during the camera pose estimation step.
\nWe thus realise a "semantic loop", where the availability of high level clues betters the mapping process, in turn helping the subsequent localisation phase.
\nA full system, drawing inspiration from the presented research, allowing a real-time and automatic semantic mapping of large-scale environments is then presented.
\n
\nAn ancillary, but nevertheless important, component of simultaneous localisation and mapping systems is a technique to allow the estimation of sensor position separately from the main SLAM loop, to recover from failures in the localisation algorithm.
\nWe present a technique that, by exploiting the appearance of image patches, can reliably localise the likely position of the sensor used to acquire such images.
\nSuch relocalisation system can be easily included in a Semantic SLAM system to allow a more robust mapping process wherein camera tracking failures can be reliably recovered from.