3D scene description by pretrained features and ICP-based odometry

摘要

The goal of my study is to provide a relative location, posture change, and spatial features that provide a spatial description for applications that recognize space such as SLAM(Simultaneous Localization And Mapping), robotics, AR(Augmented Reality), VR(Virtual Reality) and etc. For odometry by using a depth camera, corresponded feature ICP (Iterative Closest Point) is employed. However, if a feature occurs at the edge of an object, the distance may change due to angle changes, so the features are initially extracted only on a single plane using PCA(Principal Component Analysis) globally. Additionally, ICP algorithm obtains rotation and translation by extracted features in which the agent moves. This process constitutes a sliding window with N-sets. In the description of space, the plane of the floor and ceiling is first semantically recognized with PCA. It is relatively easy to distinguish by the estimating of pose with the agent's IMU (Inertial Measurement Unit) and the agent's camera tilting. Expanding from tiles recognized as floors to unique vectors, colors, and textures to obtain occupancy (Whether it's an obstacle or not.). The recognition of objects module is inspired by PointNet, harnessing inputs as points and normal vectors, and classifies pre-trained features (box, cylinder, sphere, and cone etc.). The position of the recognized object is reused to correct the drift of the odometry. Stronger recognition is possible because the shape is recognized, not a specific class. The experiment is carried out in a rover which has a battery that can operate for 10-12 hours on a single charge, a depth camera including an IMU collect data and an edge device with Wi-Fi, and transmit the data to a server for continuous training. If the agent could keep training in the same place, semi-supervised learning is possible with several confirmations by the supervisor.