A New Metric for Evaluating Semantic Segmentation: Leveraging Global and Contour Accuracy

Abstract

Semantic segmentation of images is an important issue for intelligent vehicles and mobile robotics because it offers basic information which can be used for complex reasoning and safe navigation. Different solutions have been proposed for this problem along the last two decades, where recent deep neural networks approaches have shown very promising results in the context of urban navigation. One of the main problems when comparing different semantic segmentation solutions is how to select an appropriate metric to evaluate their accuracy. On the one hand, classic metrics do not measure properly the accuracy on the object contours, which is important in urban driving to differentiate road from sidewalk for instance. On the other hand, contour-based metrics [1] disregard the information far from class contours. This paper explores the problem multi-modal image segmentation, and presents a new metric to leverage global and contour accuracy in a simple formulation. This metric is validated with the evaluation of several semantic segmentation solutions that exploit RGB-D images to rank these solutions taking into account the quality of the segmented contours. We also present a comparative analysis of several commonly used metrics together with a statistical analysis of their correlation.