Synthetic to Real Gap Estimation of Autonomous Driving Datasets using Feature Embedding

Abstract

Recent advances in autonomous driving using deep learning have drawn immense attention from robotics and computer vision communities. Training generalized deep learning models for autonomous driving tasks like visual odometry, segmentation, and object detection requires large amounts of data. Acquiring real-world data with accurate annotations is time-consuming and expensive. Due to this challenge, synthetic datasets are increasingly being used for training and testing deep learning models. Synthetic data lacks the appearance and contextual properties of real-world datasets. Several works have been shown to reduce this gap between synthetic and real-world images. However, evaluating the gap between the synthetic and real-world datasets is a longstanding challenge because of its highly not deterministic nature. This research proposes the use of feature embedding techniques to address this synthetic to reality gap in the form of distance between different data clusters. From the experiments, the proposed approach estimated the distance between real-world to enhanced virtual datasets is 6-10 times the distance between real-world to virtual datasets.